Skip to main content
banner image
venafi logo

Addressing Machine Identity Challenges with Intelligent Data

Addressing Machine Identity Challenges with Intelligent Data

Machine Identity Challenges and Intelligent Data
April 25, 2019 | Nicki Watt, CTO at OpenCredo

With the increasing adoption of cloud native computing and container technologies, the number of machines within organisations is exploding. This is significantly impacting the security of machine-to-machine communications, as described in Venafi’s machine identity crisis whitepaper. Early identification of potential vulnerabilities and violations is crucial. In order to do this, companies must be able to make sense of this ever-increasing volume of data and how it is all connected. This also enables organisations to make informed decisions and react to cyber security challenges quickly and effectively.

The Machine Identity Management Development Fund was set up to empower organisations to explore and develop effective tools for addressing these cyber security challenges. OpenCredo is proud to have contributed two complementary tools which focus on acquiring and using identity data more intelligently.

Here we introduce you to these developments, exploring the key features of the  Kafka Connector for Venafi Platform, and the Machine Identity Network Explorer Tool. (MINEr).


Kafka Connector for Venafi Platform

A crucial step on the path to identify potential vulnerabilities, weaknesses or issues within an organisation relies on gaining timely, reliable access to the underlying security data and events in the first place.

Apache Kafka is the leading open source distributed streaming platform and is used in a number of financial services firms and other highly regulated industries. It has been employed to explore real-time data in order to combat fraud, address regulatory constraints and requirements, and to bring data together from disparate systems across the organisation.

The open source Kafka connector for the Venafi Platform provides the official way to integrate your Venafi and Kafka deployments. It works by leveraging the Apache Connect framework, extracting security events from the Venafi platform and making them available for further analysis and processing via Kafka. The connector takes advantage of the public HTTP Venafi API in order to gain access to key data, which is then placed onto appropriately configured Kafka topics. This enables flexible intelligent real-time processing of security and machine identity data by a variety of different Kafka consumers—regardless of whether the target is a downstream log aggregator, SIEM, real-time machine learning platform or some combination thereof.

Figure 1: Venafi Kafka Connect - High level architecture

The Kafka connector does not rely on any changes to the Venafi platform itself and opens up options for organisations who are already using, or want to use, Kafka as an integration or real-time analysis option moving forward. The connector is also available via the Venafi Marketplace.

Summarising the key benefits and features:
  • Full event coverage
    Covers all events generated by the Venafi platform
  • Real-time data processing
    Enables configurable, real-time analysis of Venafi security events
  • Flexible integration
    Easily send Venafi events downstream into any tool capable of consuming data from Kafka
  • Open source, Confluent verified
    Available via the Confluent Hub, as open source under the popular Apache 2.0 license and officially gold verified by Confluent.
Machine Identity Network Explorer (MINEr)

Security analytics skills are in strong demand, but limited in supply. Spread thinly across multiple projects, security analysts must make sense of incomplete documentation, reams of reports and, with luck, some usable analytics. More likely than not, however, analysts will be presented with waves of information—the volume of which is only increasing. The effort required to derive insight from this tsunami of data is not insignificant.

One of the most defining features of a complex network or system is its connectivity. Business value comes from being able to discover as well as predict patterns, reveal hidden meaning and gain an intuitive understanding from this web of connected data. This is why OpenCredo developed the Machine Identity Network Explorer (MINEr). MINEr is a proof-of-concept which practically demonstrates these capabilities by exploring an organisation’s machine identity network. Going beyond simple statistical analysis, MINEr leverages innovative graph algorithms and approaches from the field of network science to advance the insights. 

Exploration and visualization

The initial version of MINEr focuses on leveraging static SSH data from the Venafi platform to build up a Machine Identity Network model. This models the underlying network of machines, users, keys and accounts—and the connections between them. Data is stored in Neo4j, a graph native datastore, providing a solid foundation for further analysis. Basic visualisation and opportunity for guided ad-hoc exploration of the network space is provided by a reactive web interface.

Figure 2: Guided visual exploration of machine identify network


Within a network, “pivoting” describes a method whereby a person attempts to gain access to some machine in a network. This is often referred to as gaining an initial foothold, with the express purpose of using it to gain further access to other machines which might not otherwise be accessible through authorized routes. The ability to move laterally (usually illegally) through the network poses a significant security risk. MINEr provides the ability to identify and visualise potential pivoting paths within the network, and serves to demonstrate the broader capability of being able to find known patterns within a vast network of data.


Figure 3: Identification and visualization of a pivoting paths

Uncovering hidden patterns

Being able to uncover new patterns in the data which may previously have been unknown, is where genuine fresh insights becomes a reality. One such example shown below is that of (Louvain based) community detection. By way of an example, groups of machines and accounts are identified as belonging together based purely on the connections between them rather than any label data which may have been explicitly supplied. This can highlight related groupings of machine communities (e.g. Sales department, Development team, Unix boxes etc) as well as potentially unexpected accounts and pathways which exist between these groups. Further insights were uncovered via various anomaly detection techniques as well.

Figure 4: Detection and visualization of communities

The MVP focused on laying the foundation for the base network driven exploration platform through static SSH data. Progressing beyond this however, further insight is on offer by including and combining data from a variety of other sources, for example, TLS (certificate) data as well as dynamic usage data. Such data will move the tool closer towards providing genuine intelligence for example machine learned real-time anomaly detection.

The future of effective cyber security

The future of effective cyber security protection lies in the ability of users to obtain reliable and genuinely useful insights and information out of their systems. Insight which allows for informed decisions on what is important, what is noise, and where limited time and effort should be invested.

We see the flexible Kafka Connector for Venafi Platform acting as a conduit for gaining reliable access to key security and event data, allowing for both existing and future tools and processes to be able to ply their trade of interpreting and processing it.

The Machine Identity Network Explorer (MINEr) demonstrates alternative ways to more intuitively visualize network data, and provide the ability to make sense of complex scenarios hiding amongst relationships which make up the machine identity network.

Technology is moving fast! Flexible adaptable tools which are able to tackle the seen challenges of today, and evolve to handle the unseen challenges of tomorrow, will be a key differentiating factor when it comes to success. We are excited to be able to share our insights as well as make these tools available, as part of contributing to moving the industry forward in this space.

About OpenCredo

OpenCredo is a hands-on software development consultancy focused on helping clients effectively and pragmatically tackle complex problems effectively through emerging technologies and practices. More information on these tools is available via the appropriate links in this blog, but if you are looking for more info, or your own tailored proof-of-concept demonstrating the art of the possible for your organisation please do reach out to us at


Related posts

Like this blog? We think you will love this.
Featured Blog

OpenCredo Venafi-Vault Wizard: Bringing InfoSec and Developers One Step Closer

Increasing visibility without slowing down developers

Read More
Subscribe to our Weekly Blog Updates!

Join thousands of other security professionals

Get top blogs delivered to your inbox every week

See Popular Tags

You might also like

TLS Machine Identity Management for Dummies

TLS Machine Identity Management for Dummies

Certificate-Related Outages Continue to Plague Organizations
White Paper

CIO Study: Certificate-Related Outages Continue to Plague Organizations

About the author

Nicki Watt, CTO at OpenCredo
Nicki Watt, CTO at OpenCredo
Read Posts by Author
get-started-overlay close-overlay cross icon
get-started-overlay close-overlay cross icon
Venafi Risk assessment Form Image

Sign up for Venafi Cloud

Venafi Cloud manages and protects certificates

* Please fill in this field Please enter valid email address
* Please fill in this field Password must be
At least 8 characters long
At least one digit
At last one lowercase letter
At least one uppercase letter
At least one special character
* Please fill in this field
* Please fill in this field
* Please fill in this field

End User License Agreement needs to be viewed and accepted

Already have an account? Login Here

get-started-overlay close-overlay cross icon

How can we help you?

Thank you!

Venafi will reach out to you within 24 hours. If you need an immediate answer please use our chat to get a live person.

In the meantime, please explore more of our solutions

Explore Solutions

learn more

Email Us a Question

learn more

Chat With Us

learn more