With the increasing adoption of cloud native computing and container technologies, the number of machines within organisations is exploding. This is significantly impacting the security of machine-to-machine communications, as described in Venafi’s machine identity crisis whitepaper. Early identification of potential vulnerabilities and violations is crucial. In order to do this, companies must be able to make sense of this ever-increasing volume of data and how it is all connected. This also enables organisations to make informed decisions and react to cyber security challenges quickly and effectively.
The Machine Identity Protection Fund was set up to empower organisations to explore and develop effective tools for addressing these cyber security challenges. OpenCredo is proud to have contributed two complementary tools which focus on acquiring and using identity data more intelligently.
Here we introduce you to these developments, exploring the key features of the Kafka Connector for Venafi Platform, and the Machine Identity Network Explorer Tool. (MINEr).
A crucial step on the path to identify potential vulnerabilities, weaknesses or issues within an organisation relies on gaining timely, reliable access to the underlying security data and events in the first place.
Apache Kafka is the leading open source distributed streaming platform and is used in a number of financial services firms and other highly regulated industries. It has been employed to explore real-time data in order to combat fraud, address regulatory constraints and requirements, and to bring data together from disparate systems across the organisation.
The open source Kafka connector for the Venafi Platform provides the official way to integrate your Venafi and Kafka deployments. It works by leveraging the Apache Connect framework, extracting security events from the Venafi platform and making them available for further analysis and processing via Kafka. The connector takes advantage of the public HTTP Venafi API in order to gain access to key data, which is then placed onto appropriately configured Kafka topics. This enables flexible intelligent real-time processing of security and machine identity data by a variety of different Kafka consumers—regardless of whether the target is a downstream log aggregator, SIEM, real-time machine learning platform or some combination thereof.
Figure 1: Venafi Kafka Connect - High level architecture
The Kafka connector does not rely on any changes to the Venafi platform itself and opens up options for organisations who are already using, or want to use, Kafka as an integration or real-time analysis option moving forward. The connector is also available via the Venafi Marketplace.
Summarising the key benefits and features:
Full event coverage
Covers all events generated by the Venafi platform
Real-time data processing
Enables configurable, real-time analysis of Venafi security events
Easily send Venafi events downstream into any tool capable of consuming data from Kafka
Security analytics skills are in strong demand, but limited in supply. Spread thinly across multiple projects, security analysts must make sense of incomplete documentation, reams of reports and, with luck, some usable analytics. More likely than not, however, analysts will be presented with waves of information—the volume of which is only increasing. The effort required to derive insight from this tsunami of data is not insignificant.
One of the most defining features of a complex network or system is its connectivity. Business value comes from being able to discover as well as predict patterns, reveal hidden meaning and gain an intuitive understanding from this web of connected data. This is why OpenCredo developed the Machine Identity Network Explorer (MINEr). MINEr is a proof-of-concept which practically demonstrates these capabilities by exploring an organisation’s machine identity network. Going beyond simple statistical analysis, MINEr leverages innovative graph algorithms and approaches from the field of network science to advance the insights.
Exploration and visualization
The initial version of MINEr focuses on leveraging static SSH data from the Venafi platform to build up a Machine Identity Network model. This models the underlying network of machines, users, keys and accounts—and the connections between them. Data is stored in Neo4j, a graph native datastore, providing a solid foundation for further analysis. Basic visualisation and opportunity for guided ad-hoc exploration of the network space is provided by a reactive web interface.
Figure 2: Guided visual exploration of machine identify network
Within a network, “pivoting” describes a method whereby a person attempts to gain access to some machine in a network. This is often referred to as gaining an initial foothold, with the express purpose of using it to gain further access to other machines which might not otherwise be accessible through authorized routes. The ability to move laterally (usually illegally) throughthe network poses a significant security risk. MINEr provides the ability to identify and visualise potential pivoting paths within the network, and serves to demonstrate the broader capability of being able to find known patterns within a vast network of data.
Figure 3: Identification and visualization of a pivoting paths
Uncovering hidden patterns
Being able to uncover new patterns in the data which may previously have been unknown, is where genuine fresh insights becomes a reality. One such example shown below is that of (Louvain based) community detection. By way of an example, groups of machines and accounts are identified as belonging together based purely on the connections between them rather than any label data which may have been explicitly supplied. This can highlight related groupings of machine communities (e.g. Sales department, Development team, Unix boxes etc) as well as potentially unexpected accounts and pathways which exist between these groups. Further insights were uncovered via various anomaly detection techniques as well.
Figure 4: Detection and visualization of communities
The MVP focused on laying the foundation for the base network driven exploration platform through static SSH data. Progressing beyond this however, further insight is on offer by including and combining data from a variety of other sources, for example, TLS (certificate) data as well as dynamic usage data. Such data will move the tool closer towards providing genuine intelligence for example machine learned real-time anomaly detection.
The future of effective cyber security
The future of effective cyber security protection lies in the ability of users to obtain reliable and genuinely useful insights and information out of their systems. Insight which allows for informed decisions on what is important, what is noise, and where limited time and effort should be invested.
We see the flexible Kafka Connector for Venafi Platform acting as a conduit for gaining reliable access to key security and event data, allowing for both existing and future tools and processes to be able to ply their trade of interpreting and processing it.
The Machine Identity Network Explorer (MINEr) demonstratesalternative ways to more intuitively visualize network data, and provide the ability to make sense of complex scenarios hiding amongst relationships which make up the machine identity network.
Technology is moving fast! Flexible adaptable tools which are able to tackle the seen challenges of today, and evolve to handle the unseen challenges of tomorrow, will be a key differentiating factor when it comes to success. We are excited to be able to share our insights as well as make these tools available, as part of contributing to moving the industry forward in this space.
OpenCredo is a hands-on software development consultancy focused on helping clients effectively and pragmatically tackle complex problems effectively through emerging technologies and practices. More information on these tools is available via the appropriate links in this blog, but if you are looking for more info, or your own tailored proof-of-concept demonstrating the art of the possible for your organisation please do reach out to us at [email protected].