AI Notes

Learn a lot

Tag: json

  • Sample Project demonstrating moving Data from Kafka into Tableau

    Here we demonstrate connection from Tableau to Kafka using a most practical approach using a database as a sink via Kafka Connect and then connecting Tableau to that database. Here’s a breakdown with conceptual configuration and Python code snippets: Scenario: We’ll stream JSON data from a Kafka topic (user_activity) into a PostgreSQL database table (user_activity_table)… Read more

  • Building a Personalized Banking Chat Agent with React.js, RAG, LLM, and Redis with sample code

    Here we outline a more detailed structure with conceptual sample code snippets for each layer of a conceptual personalized bank FAQ chat agent. Keep in mind that this is a simplified illustration, and a production-ready system would involve more robust error handling, security measures, and integration logic. I. Knowledge Base Preparation: Step 1: Data Collection… Read more

  • Detail of Parquet

    The Parquet format is a column-oriented data storage format designed for efficient data storage and retrieval. It is an open-source project within the Apache Hadoop ecosystem. Here’s a breakdown of its key aspects: Key Characteristics: Advantages of Using Parquet: Disadvantages of Using Parquet: Parquet vs. Other Data Formats: In summary, Parquet is a powerful and… Read more

  • Building a Personalized Bank FAQ Chat Agent with React.js, RAG, LLM, and Redis

    Providing efficient and informative customer support is crucial for any financial institution. A well-designed FAQ chat agent can significantly enhance the user experience by offering instant answers to common queries. This article provides a comprehensive guide to building a personalized bank FAQ chat agent using React.js for the frontend, Retrieval-Augmented Generation (RAG) and a Large… Read more

  • Intelligent Chat Agent UI with Retrieval-Augmented Generation (RAG) and a Large Language Model (LLM) using Amazon OpenSearch

    In today’s digital age, providing efficient and accurate customer support is paramount. Intelligent chat agents, powered by the latest advancements in Natural Language Processing (NLP), offer a promising avenue for addressing user queries effectively. This comprehensive article will guide you through the process of building a sophisticated Chat Agent UI application that leverages the power… Read more

  • Simplistic implementation of Medallion Architecture (With Code)

    Here we demonstrate a simplistic implementation of Medallion Architecture. Medallion Architecture provides a structured and robust approach to building a data lakehouse. By progressively refining data through the Bronze, Silver, and Gold layers, organizations can ensure data quality, improve governance, and ultimately derive more valuable insights for their business Python Explanation of the Sample Code… Read more

  • Medallion Architecture

    The Medallion Architecture is a data lakehouse architecture pattern popularized by Databricks. It’s designed to progressively refine data through a series of layers, ensuring data quality and suitability for various downstream consumption needs. The name “Medallion” refers to the distinct quality levels achieved at each layer, similar to how medals signify different levels of achievement.… Read more

  • Data Lake vs. Data Lakehouse: Understanding Modern Data Architectures

    Organizations today grapple with ever-increasing volumes and varieties of data. To effectively store, manage, and analyze this data, different architectural approaches have emerged. Two prominent concepts in this landscape are the data lake and the data lakehouse. While both aim to provide a centralized data repository, they differ significantly in their design principles and capabilities.… Read more

  • Loading documents into OpenSearch for vector search

    Here’s how you can load documents into OpenSearch for vector search: 1. Create a k-NN Index First, you need to create an index in OpenSearch that is configured for k-Nearest Neighbors (k-NN) search. This involves setting index.knn to true and defining the field that will store your vector embeddings as type knn_vector. You also need… Read more

  • k-NN (k-Nearest Neighbors) search in OpenSearch

    To perform a k-NN (k-Nearest Neighbors) search in OpenSearch after loading your manuals (or any documents) as vector embeddings, you’ll use the knn query within the OpenSearch search API. Here’s how you can do it: Understanding the knn Query The knn query in OpenSearch allows you to find the k most similar vectors to a… Read more