home shape
plural logo 300x104

Plural Technology: Graph thinking for a new part discovery platform

Using knowledge graphs and AI/ML to analyze products to power supply-chain efficiencies

Results:

  • Strengthened competitiveness of Fortune 500 global engineering and technology company
  • Enabled company to make rapid, informed manufacturing decisions by finding similarities and making recommendations
  • Greatly expedited queries for searches such as parts, bills of materials, and suppliers
  • Simplified complex architecture and improved with AI and machine learning
  • Effortless dynamic scaling to drive operational efficiency
right blob img min

The Scenario: The need for a relationship-centric store for PLM data

Plural Technology helps more than 200 of the world’s most prestigious companies improve product development, production, design, and management. The company started in 2007 with headquarters in Pune, India, and is known for its expertise in Product Lifecycle Management (PLM).

Clients come to Plural Technology to solve their most pressing issues, such as quickly offering customers broad product variety while reducing costs. This task is more challenging than ever due to unstable demand, globalization, increased lead times, and the massive supply chain and manufacturing disruptions caused by the COVID-19 pandemic.

“We are living in the age of personalization, from consumer preferences for personalized products to personalized production like engineered-to-order (ETO),” explains Plural Technology’s Sr. Program Manager, Deenu Gengiti. “In 2022, Apple launched its mobile phone, iPhone 14, with 36 variants of different sizes, colors, and storage. And Apple isn’t alone. The only way for manufacturers to survive is through mass customization, made possible by technologies such as PLM, ERP, and CRM.”

The ability to quickly search product information such as parts, drawings, change items, bills of materials, vendors, and production planning is imperative for manufacturers in the age of customization. Product data has become far more diverse and complex, and manufacturers need insights into the connections among bits of information. In many complex products, the number of potential configurations is often greater than the number of parts. Layered on top of that are entities such as people, companies, orders, and supply chain partners, making relationships the most crucial factor in PLM.

Another problem arose: relational databases, where product information is traditionally stored, failed to bring insights into known and inferred relationships. This shortcoming impedes their ability to provide intuitive visualization capabilities and ML-powered analytics.

“Are graph databases a radical overhaul for future enterprise applications?” asks Gengiti. “Probably. Advances in computer architecture and increases in parallelism and main memory allow us to rethink enterprise applications’ storage layer.”

The Requirements: A schema-less database with knowledge graphs, AI/ML, and easy scalability

In 2022, the need for a new PLM database architecture arose. A Fortune 500 global engineering and technology company approached Gengiti and his team. They had a tall order: they were receiving 40,000 incoming e-commerce orders daily, each with requests for price quotes. This customer wanted a system that could accelerate the closing of large orders by recommending similar product variants delivered in the past.

The client’s existing Product Information Management (PIM) system was ten years old, had a clunky interface, and relied on a SQL database and batch processing. As a result, it was causing delays due to its inability to scale to accommodate the company’s massive dataset comprising billions of data points and more than 160 million records from ERP, PLM, and other legacy systems.

Engineering, supply chain, customer service, and operations teams struggled with Microsoft Power BI, spreadsheets, and other manual methods for reporting and analytics. Their complex and outdated technology stack led to inefficiencies, a lack of real-time or past insights into product data, an inability to trace or fix data, and no way to handle incremental change.

The client needed a whole new state-of-the-art part discovery platform. The first step in creating it was a wholesale shift from relational to graph thinking. Plural Technology’s team started with questions such as, “What are the most frequent product queries?” and “What product questions do users need to answer most in day-to-day operations?”

Screenshot 2023 02 24 at 15.41.47 1536x622

Plural Technology transformed a relational database schema into a graph

The Plural Technology team then polled the client’s subject matter experts to help convert the relational data model to a graph that would intuitively show which vendors make and supply each part – and how each part is related to others within the bill of materials.

“We had to move away from a schematic structure to a schema-less one,” says Gengiti. “ArangoDB rose to the top of the list when we explored various options because of its multi-model capabilities.”

Why ArangoDB: Schema-less and flexible

Plural Technology chose ArangoDB because it is schema-less and offers robust graph functionality, including the ability to create relationships between data and quickly query them. As a native multi-model database, ArangoDB can represent data in documents, key/value pairs, and graphs.

Beyond the data representation, ArangoDB offers searches combining full-text and AQL queries, supercharged by built-in GraphML algorithms and analytics for operations like similarity searches. ArangoDB’s Kubernetes operator made it easy for Plural Technology to perform containerized distributed application management, one more feature that put ArangoDB at the top of Plural Technology’s list.

background img

The Implementation: Microservices on sharded ArangoDB on OpenShift clusters on Azure

Plural Technology built a proof of concept to overhaul the PIS system, including data ingestion and Extract, Transform, and Load (ETL) processes using Apache Kafka with Kafka Connect and Debezium source and ArangoDB sink connectors to stream data to ArangoDB. In addition, an API layer delivers product data to React, Quarkus, and Jupyter Notebook interfaces for presenting, searching, and analyzing products, bills of materials, and supply chain data. The system runs on Red Hat OpenShift clusters on Microsoft Azure.

To develop the graph data model, the Plural Technology team imported a subset of the records, approximately nine million from 12 locations business applications (ERP, CPQ, PLM), for initial testing and benchmarking. They used Python to transform the data for ArangoDB. The system replicates source files to JFrog, and the ArangoDB Foxx microservice framework manages API endpoints. They also built a sample React UI and extended it to Graphistry for deep graph analytics.

Screenshot 2023 02 24 at 15.44.04 1536x909

ArangoDB AQL and Foxx Microservices were markedly faster than
their previous relational database.

The Results: Robust graph functionality, near-instant scalability, fuzzy search, and other surprise benefits

Built-in microservices framework expedites product queries: Instead of manually designing and hosting a microservices architecture to orchestrate interactions between the database storage layer and front-end, the Foxx microservices framework is built into ArangoDB, significantly expediting queries for searches such as parts, bills of materials, and suppliers. In addition, the Foxx framework allows Plural Technology to restrict access to sensitive data by adding external authentications. Yadav adds that there’s no need for reskilling: anyone with Swagger UI experience can quickly grasp the ArangoDB Foxx framework.

Full-text, fuzzy search with ArangoSearch: Natural-language search capabilities built into ArangoDB allow the client to conduct accurate searches, even if the spelling is wrong. ArangoSearch supplies rich natural language processing tools, customizable relevance scoring, federated search, and schema-agnostic indexing. Plus, users can combine graph traversals or joins with ArangoSearch functions to find node information, such as suppliers.

“When our customers saw ArangoSearch, they were amazed and thrilled that they could retrieve the correct data even if the spelling was wrong. Also, the search response time was stunning,” says Yadav.

Powerful query language: According to Yadav, ArangoDB Query Language (AQL) is one of the most potent query languages Plural Technology has ever encountered. It provides a standard query language for all data models, is human-readable, delivers key/value pairs and adjacent documents, and allows Plural Technology to use the same language for all clients, regardless of their preferred programming languages. Another plus: it’s easy to understand for anyone who knows SQL.

According to Yadav, AQL is easy to read, interpret, and write. In addition, AQL has drastically reduced the time for queries to execute. And, with the low-code approach and machine learning component built-in, Plural Technology’s client can quickly retrieve any similarity search. AQL can convert nodes into a graph of low-dimensional, mathematical, or numerical vectors to find, for instance, the distances between the similarities and determine the top recommendations for bills of materials.

“The ability to find similarities and make recommendations is exactly the requirement our client asked for from day one,” says Yadav. “The customer was delighted to receive the result in one simple query instead of having us do extensive customization. It was quite an amazing experience.”

Fast, easy scalability: With ArangoDB’s smart sharding, dynamic system management, and the ArangoDB Kubernetes Operator, scaling servers and clusters up or down based on demand is a fast, effective, one-click process. Spinning up new clusters takes just five minutes. ArangoDB handles the rest, automatically realigning and re-indexing data on the back end.

Performance gains: Queries run against roughly nine million records showed 38% faster performance due to graph traversals, built-in ML capabilities, and enriched query-building functions.

“When our customers saw ArangoSearch, they were amazed and thrilled that they could retrieve correct data even if the spelling was wrong. Also, the response time was stunning.”

- Deenu Gengiti, Plural Technology

In summary, Plural Technology saw increased developer productivity through Foxx microservices, ArangoSearch, and AQL, along with improved operational efficiency via smart sharding, a Kubernetes operator, and fast query performance. All these improvements enabled Plura to deliver innovations in less time.

Customer Architecture

Below is the architecture of what Plural Technology built to analyze products, bills of materials, and supply chains. Apache Kafka with Kafka Connect handles ingest and ETL, and Debezium source and ArangoDB sink connectors stream data to ArangoDB. An API layer delivers product data to React, Quarkus, and Jupyter Notebook interfaces for presenting, searching, and analyzing product data. The solution runs on Red Hat OpenShift clusters on Microsoft Azure.

Screenshot 2023 02 24 at 15.46.04 1536x625

For more details, you can watch Deenu’s presentation from ArangoDB Summit 2022: