Learning Hub

Learning Hub Post Type

How to Make a Website Accessible for the Blind Community

Introduction Imagine entering a store where every section is mislabeled, the signs are written in a language you don’t understand, and the staff are unable to answer even basic questions. That’s how a blind person can feel when navigating an inaccessible website. In today’s digital-first world, ensuring your website is accessible is far more than a courtesy—it’s a fundamental necessity. According to the World Health Organization, at least 2.2 billion people have a near or distance vision impairment, with at least 1 billion of these cases involving vision impairment that could have been prevented or has yet to be addressed. By designing with accessibility in mind, you’re not only adhering to compliance standards but also fostering inclusivity, reaching a broader audience, and building meaningful trust with your users. Why It’s Important to Build an Accessible Website for the Blind Legal responsibility: Laws such as the Rights of Persons with Disabilities (RPwD) Act in India, the Americans with Disabilities Act (ADA) in the U.S., and global standards like the Web Content Accessibility Guidelines (WCAG) mandate accessibility. Companies that fail to comply risk lawsuits, fines, and reputational damage. Thousands of ADA website accessibility lawsuits are filed annually in the U.S., demonstrating how seriously accessibility compliance is being enforced globally. Business benefit: Making your site accessible expands your potential customer base. For instance, if your e-commerce store isn’t screen-reader friendly, you may be shutting out thousands of willing buyers. Better overall usability: Accessibility best practices—such as clear navigation, structured content, and descriptive labeling—improve the experience for all users, particularly those using mobile devices, voice search, or browsing in challenging conditions like bright sunlight. Ethical responsibility: At its core, accessibility is about equal opportunity and digital inclusion. The web should be a place where everyone has the same access to information, services, and experiences, regardless of their abilities. Understanding How Blind Users Navigate the Web Blind and visually impaired users interact with the internet in ways that may not be obvious to sighted users. For example: Linear navigation: Unlike sighted users, who can scan a page visually, blind users often experience content line by line through a screen reader. If your content isn’t properly structured, it becomes a frustrating maze of disconnected information. Heading levels as a roadmap: Proper heading tags (<h1>, <h2>, <h3>) allow screen reader users to jump quickly between sections, creating a mental map of your content. When headings are misused for styling rather than structure, this navigation system breaks down entirely. Links and buttons: A button labeled only “Click here” tells a blind user nothing about what will happen next. However, “Download Product Guide” provides clarity and empowers users to make informed decisions. Real-World Example: Imagine a blind user visits an online clothing store. If product images lack alt text, the screen reader might announce “image123.jpg” instead of “Blue denim jacket, size medium, $59.99.” This forces users to rely on others for basic shopping tasks, eliminating their independence and potentially losing your sale. Tools the Visually Impaired Use to Navigate Websites Blind users rely on assistive technology to bridge the gap between web content and accessibility. Some common tools include: Screen readers: JAWS and NVDA (Windows), VoiceOver (Mac and iOS), and TalkBack (Android). These programs read aloud page content, navigate menus, announce form fields, and even describe text formatting and page structure. Braille displays: Hardware devices that convert on-screen text into refreshable braille patterns, allowing users to “read” websites through touch. These are particularly useful for users who are deaf-blind or prefer tactile reading. Screen magnifiers: Software like ZoomText or built-in magnification tools used by individuals with low vision to enlarge text, images, and interface elements up to 16x their original size. Voice navigation software: Programs like Dragon NaturallySpeaking allow users to browse websites, fill forms, and click links using voice commands alone. Keyboard navigation: Many users rely entirely on keyboard shortcuts (Tab, Enter,Arrow keys) to navigate websites when mouse interaction isn’t feasible. Developer Tip: You can experience your website from a blind user’s perspective by installing NVDA (free for Windows) or testing with VoiceOver on Mac. Close your eyes and try navigating your site using only these tools—you’ll quickly discover accessibility gaps you never noticed. Key Steps to Ensure Your Website is Accessible for the Blind Community Here’s a detailed breakdown with do’s and don’ts for each step: 1. Use Semantic HTML Do: Use headings in proper hierarchy (<h1> for main title, <h2> for major sections, <h3> for subsections). Don’t: Use headings solely for styling purposes (e.g., making text large and bold without semantic meaning) or skip heading levels 2. Add Meaningful Alt Text to Images Do: Write concise, descriptive alt text that conveys the image’s purpose (“Golden retriever puppy playing with a red ball in a park”). Don’t: Over-describe, add redundant phrases like “Image of…” or “Picture showing…” or leave alt text empty for informative images. 3. Provide Text Alternatives for Multimedia Add complete transcripts for audio content and podcasts. Include captions and audio descriptions for videos. Example: A cooking tutorial should describe not just the dialogue, but also visual actions (e.g., “The chef dices onions into quarter-inch cubes using a sharp knife”). 4. Ensure Keyboard Navigation Works Users must be able to navigate through all interactive elements—menus, buttons, forms, and links—using Tab, Shift+Tab, Enter, and Arrow keys. Add visible focus indicators so users can clearly see which element is currently selected. Ensure no “keyboard traps” where users get stuck on elements. 5. Label Forms and Buttons Clearly Do: Use descriptive, action-oriented labels (e.g., “Subscribe to Monthly Newsletter,” “Download PDF Guide”). Don’t: Use vague labels like “Submit,” “Click Here,” or “Learn More” without context. 6. Implement ARIA Landmarks Wisely ARIA attributes (Accessible Rich Internet Applications) enhance meaning where standard HTML falls short. Example: aria-label=”Search products” on a search input, or aria-describedby=”password-help” for form instructions. Important: Overusing ARIA can confuse screen readers—always prefer semantic HTML when available. 7. Prioritize Contrast and Resizable Text Use contrast checkers (like WebAIM’s tool) to ensure text meets WCAG AA standards (4.5:1 ratio

Smarter, Faster, Scalable: Designing a Selenium Automation Framework with Bitbucket

Because QA shouldn’t be about repetition. It should be about consistency, innovation, and impact. Repetitive workflows are common in enterprise applications: logging into portals, filling forms, uploading documents, validating data, and waiting for confirmations. Performed manually, these tasks are not only time-consuming but also prone to errors. For QA teams, this often meant repeating the same checks across different environments or re-validating minor changes on developer requests, effort that slowed down delivery without truly adding value. By automating these repetitive tasks, the framework not only saves time but also allows QA to focus on higher-impact activities like exploratory testing and quality improvements. To address this, I built a Java Selenium automation framework that is modular, scalable, and version-controlled via Bitbucket. While the initial use case focused on tender workflows, the framework itself is application-agnostic and can be adapted to streamline any enterprise process with similar challenges. This blog shares how the framework was designed, the technical challenges overcome, and how the solutions create a strong foundation for smarter, reusable automation across business-critical workflows. Section 1: Why Automation Was Needed  Enterprise workflows often involve repetitive, multi-step tasks such as:  Logging into secure portals. Entering unique identifiers and references  Selecting portal types, categories, and configurations  Uploading critical documents  Scheduling processes for execution  Validating confirmation popups and status updates  Manually repeating these steps across multiple configurations is not only time-consuming but also increases the risk of human error and delays overall delivery.  By introducing automation, we achieved:  Significant time savings in regression testing  100% consistency and accuracy across configurations  Scalability to handle future features and workflows  Seamless integration with Bitbucket, paving the way for CI/CD pipelines  Section 2: Framework Design  I kept the framework modular, reusable, and scalable, using Java, Selenium, JUnit 5, and Maven.  Here’s the compact structure: project-qa-automation │── src/main/java/com/project/automation │   ├── locators/   # Application locators │   ├── models/     # Tender configuration model │   ├── pages/      # Page Objects │   ├── utils/      # Config, TenderUtils, FilePaths │ │── src/test/java/com/project/automation │   ├── BaseTest.java   # Driver setup & teardown │   ├── LoginTest.java  # Login test │   └── ProcessTender.java # End-to-end tender workflow │ │── pom.xml         # Maven dependencies │── chromedriver.exe # WebDriver binary   This modular design ensures readability, reusability, and scalability, while making onboarding seamless, new team members can quickly understand the structure and start contributing.  Section 3: Key Features Implemented  Unique Identifier Generation  Tender IDs and Ref IDs are generated inside each test iteration, ensuring uniqueness across runs. This prevents duplication issues and keeps execution conflict-free.  Memory Cleanup for Generated IDs To maintain a clean runtime environment and avoid memory bloat, all generated Tender IDs are stored in a Set during execution. After the test completes, this Set is cleared, ensuring that no leftover IDs remain in memory. This small but important step improves efficiency and reinforces robust test design. End-to-End Workflow Automation The framework automates the full cycle, from login and document uploads to scheduling and confirmation validation. It supports multiple platforms and process categories (e.g., Public vs. Limited, Works vs. Goods vs. Services), ensuring broad coverage and consistency.  Smart Dynamic Config Selector One of the most distinctive features of this framework is its ability to adapt at runtime. Instead of hardcoding options, users are presented with a clean multi-select modal (TestConfigSelector.java) that prompts: “Select Configurations (Use Ctrl + Click for Multiple Selection)”  Implementation of the Smart Config Selector (TestConfigSelector.java) that dynamically builds the runtime configuration modal. At runtime, the modal allows:  Selecting one or more configurations dynamically  Graceful exit if Cancel is clicked (no dangling browsers)  Eliminating hardcoding for flexibility across workflows  Making automation accessible even to non-technical members  The interactive configuration modal shown at runtime, allowing users to select one or more options or cancel gracefully.  This feature strikes the perfect balance between automation power and user-driven adaptability.  Error Handling & Recovery Robust exception handling ensures smooth execution:  Graceful shutdown if a user cancels config selection  Driver quits safely on runtime errors, avoiding resource leaks  Scheduling Automation Integrated with the “Schedule Now” functionality. The script waits for a “Created Successfully” confirmation before proceeding, ensuring backend processes are triggered reliably.  Bitbucket Integration The framework is version-controlled with a branch → PR → merge-to-master strategy. This maintains a clean commit history, proper author/email configuration, and supports smooth team collaboration.  Section 4: Challenges and Solutions  Ensuring Stable Login Flow During early test runs, the browser sometimes launched to a blank page. Solution: Explicitly navigating to Config.BASE_URL before login actions, which stabilized execution across environments.  Maintaining Unique Identifiers Each iteration required a fresh Tender ID and Ref ID. Solution: Moving ID generation inside the test loop so every run produces unique identifiers without conflicts.  Graceful Execution Control When users cancelled the configuration modal, sessions remained open. Solution: Implemented exception handling to shut down the driver cleanly, avoiding dangling browsers and resource leaks.  Scalable Version Control Practices Early commits in Bitbucket suffered from mismatched authorship and messy history. Solution: Enforced proper Git user/email configuration and adopted a branch → PR → merge workflow, ensuring collaboration and traceability.  Streamlined Onboarding for New Contributors New team members were often unsure about “publish vs push” during their first commit. Solution: Standardized documentation and defined a clear workflow (git push origin branch → Pull Request), making onboarding seamless.  Section 5: Impact  The framework has delivered measurable benefits:  End-to-End Automation: Processes that once took hours now complete in minutes.  Multi-Configuration Support: A single run can handle multiple workflows without code duplication.  Scalable & Reusable Codebase: Built on modular principles, making it easy to extend for new features.  Reduced Errors, Increased Speed: Consistency across runs eliminates human error while accelerating delivery cycles.  Seamless Collaboration: Bitbucket integration with branch → PR → merge workflows ensure clean version control and smoother teamwork.  In practice, this means repetitive, error-prone processes have been transformed into fast, reliable, and scalable systems, freeing teams to focus on higher-value work while building a strong foundation for

Load Testing a Chatbot API A Journey with Apache JMeter

Load Testing a Chatbot API: A Journey with Apache JMeter

It started with a simple question from our QA lead: “How fast is our chatbot when 200 users start talking to it at the same time?”  It was a good question. We had tested the chatbot for correctness, covered edge cases, and made sure it could respond sensibly. But we hadn’t tested it under pressure — the kind of pressure it might face during a product launch or a marketing campaign.  There are many load testing tools available like K6, Apache JMeter and LoadRunner each with its own strengths. But for our needs, JMeter stood out. It’s open-source, easy to configure, has a graphical interface, supports scripting when needed, and integrates well into CI/CD pipelines. More importantly, it allowed us to simulate realistic user behavior without writing a single line of code.  So, we turned to Apache JMeter, a powerful tool that helps simulate real-world usage patterns and understand how your system performs under load.  Why JMeter? Before diving in, we asked ourselves — why JMeter?  For starters, it’s free, supports multiple protocols like HTTP, REST, and FTP, and offers a graphical interface that doesn’t require much coding. For those who want more control, scripting is available through BeanShell and JSR223. It’s also compatible with CI/CD tools like Jenkins, making it perfect for agile and DevOps workflows.  With JMeter, we could simulate hundreds of virtual users, mimic realistic behavior, analyze response times, and quickly identify bottlenecks.  Setting the Stage – The Test Plan The chatbot API’s interaction model was simple: A user starts a new chat session (GET request).  Then, the user sends a query (another POST request).  The Thread Group is configured with 200 virtual users to simulate concurrent user activity, a ramp-up period of 1 second, and a loop count set to 1. This setup mimics how multiple users would interact with the chatbot API in a real-world scenario.  But simulating real-world behavior is rarely simple. Each interaction needed to: Extract and pass dynamic session IDs using JSON Extractors. Simulate real user thinking time with Constant Timers. Support multi-language payloads, like English (“en”), but adaptable for more. Gradually ramp up user load to avoid spiking the system instantly.  Example Payload: Crafting the Simulation We broke the test plan into two core components:  1. Start a New Session – A GET request that returns a session ID, which we extract using JMeter’s JSON Extractor.  2. Send a User Query – A POST request using that session ID, simulating a real question from the user.  We added an HTTP Header Manager to include:  To simulate user traffic, we configured a Thread Group with 200 virtual users, each performing the above actions with random delays (using Gaussian and Constant Timers). This made the simulation more life-like and less robotic. Execution and Discovery Once our test was ready, we hit Start.  As the virtual users swarmed the chatbot API, we used JMeter’s Summary Report, Aggregate Report, and Graph Results to monitor how the system held up.  Key Observations:  New session response time: Averaged 904 ms — fast and consistent. Query response time: Averaged 22 seconds, with spikes up to 30 seconds. Failures: Zero, which was reassuring.  But there was a clear performance gap between starting a session and processing a query. The backend, likely due to complex chatbot logic or third-party API calls, was struggling to keep up when the load increased.  Using the View Results Tree, we confirmed that the API was responding correctly, just not fast enough.  What We Learned (and What’s Next) Our load test journey didn’t just validate system stability—it offered valuable insights into the internal behavior of our chatbot API under pressure.  Key Learnings  Session Creation Performed Well  When simulating hundreds of users starting conversations, the chatbot API maintained excellent response times. The session creation averaged around 904 ms and scaled predictably even as the number of users increased. This gave us confidence that the authentication and initial session handling mechanisms are well-architected and production-ready.  Query Handling Showed Performance Bottlenecks  The true test of the system came when users started sending queries.  Response times surged to an average of 22 seconds, with some outliers touching 30 seconds. Although no failures were detected, such long response times would lead to poor user experience in real scenarios. We investigated possible causes:  Backend Processing Overhead: As query volume increased, backend services struggled to maintain speed. This points to potentially non-optimized workflows or excessive internal API calls.  Absence of Caching: Identical or repeated queries from multiple users consistently hit backend systems without any response caching, adding unnecessary load.  Heavy NLP Model Processing: Natural Language Processing (NLP) models or large language models (LLMs) are computationally expensive. The simultaneous invocation by many users could easily explain the sharp increase in latency.  Overall, while the system passed the “survivability test,” it failed the “speed test” under high concurrency for query handling.  Turning Learnings into Action Having identified the weak points, we were able to outline clear next steps to enhance performance:  Introduce Caching Mechanisms: Applying caching for repetitive queries will drastically reduce backend load and improve response times.  Refactor Backend Processes: Streamlining internal API workflows, minimizing database calls, and using asynchronous processing where possible will cut down backend response delays.  Optimize LLM/NLP Operations: Investigate batch processing of queries, reduce model calls when not strictly necessary, or explore lightweight fallback mechanisms.  Improve Database Indexing: Optimizing data indexing structures will accelerate search and retrieval tasks, which likely contributes to delays during query resolution. Expand Test Scenarios: Plan additional load tests with variations in network conditions, payload sizes, and concurrency levels to further harden the system.  The Bigger Picture Performance testing is not just about breaking the system; it’s about understanding its limits and preparing for the real world.  Tools like JMeter empower teams to simulate large-scale traffic, pinpoint bottlenecks, and deliver fast, scalable applications — especially critical for chatbots where users expect immediate responses.  Final Thoughts What began as a curious question turned into a journey of exploration and insight. With Apache JMeter, we could

Delving deeper into interaction with AI - Communicating with AI

Delving Deeper Into Interaction With AI – Communicating With AI

Building on our introduction to prompt engineering (the previous blog), this blog delves into advanced strategies that can further enhance your interactions with large language models. Prompt engineering is rapidly evolving with AI’s increasing relevance in our day-to-day lives. The latest developments of prompt engineering include: adaptive prompting, multimodal prompting, AI-assisted prompt optimisation, automated prompt generation with self-refinement, etc. The role of prompt engineering is gaining more importance; it’s becoming a vital skill for effective human-machine interaction by improving user experience.  Recent Developments In Prompt Engineering Adaptive prompting– AI models are being programmed for improved personalisation. They adapt to the input style and user preferences and accordingly adjust responses.  Multimodal Programming– This type of prompting includes crafting instructions that can include text, images and even audio, allowing for more nuanced and contextual answers. For example, providing an image alongside a text prompt can guide a graphic-generating AI more effectively.  Automated Prompt Generation and Refinement– AI models that rewrite the prompts to refine the input and add clarity, making it easier for users to get desired results even without knowledge of prompt engineering.  AI-Assisted Prompt Optimisation– This technology provides feedback on the prompts in real time with suggestions to increase effectiveness.  Integration with domain-specific models– Specialised models are trained on industry-specific data to increase accuracy and relevance of responses to prompts based on varied fields. These models are tailored to enhance utility and precision using specialisation.  Advanced Prompt Engineering Techniques The art of crafting sophisticated prompts to elicit optimal responses from LLMs requires advanced prompt engineering techniques. This is an extension to the practices explored in the previous blog, like role-playing, contextualising, few-shot prompting and chain of thought(CoT) reasoning to guide AI towards specificity and efficiency. Tree-of-Thought (ToT) Prompting: This goes beyond CoT as the AI uses various logics and reasoning before providing the final answer. Similar to the tree diagram in probability theory, several paths branch out from the user input, each with a different approach to the prompt, ultimately leading to a final input.  Self-Refinement: The AI can be instructed to evaluate and improve its own answers based on feedback. This is an advancement from the normal refinement method, where you ask the AI for feedback and accordingly improve the prompt.  Prompt Chaining: A complex prompt is broken down into simpler prompts and is sequenced in a way where the output of one prompt becomes the input for the next. This way, tasks that include multiple steps are answered with sufficient explanation. This is not to be confused with task breakdown. Task breakdown focuses on dividing complex tasks into smaller, easier tasks, whereas prompt chaining focuses on creating a sequential pipeline of prompts.  Image credits: IBM  Meta Prompting: Focuses on the structure of the content rather than the content itself. Instead of prioritising specific details, the goal is to emphasise the format and pattern of the information, creating an abstract method of communication with LLMs, certainly distant from the traditional content-based methods. It employs abstract examples as frameworks, uses syntax as a guiding template, follows type theory by categorisation and is useful across multiple domains, giving it the ability to solve a wide range of problems. Image Source: Zhang et al.2024  Self-consistency: aims “to replace the naive greedy decoding used in chain-of-thought prompting.”(Wang et al. 2022) It applies few-shot CoT to create diverse reasoning paths and select the most consistent answer. This is an improved version of CoT prompting and is especially used for tasks requiring mathematical logic or common sense. Image source: PromptHub  Generated Knowledge Prompting: Models can be used to generate knowledge before making predictions. If there is information that might be necessary to answer your question, it can be generated and integrated into the answer.  Image Source: Liu et al.2022  Directional Stimulus Prompting: A tuneable language model is trained to generate a hint to make the response more specific and relevant. The model is presented with direct instructions acting as a stimulus to produce highly specific and contextual responses. The stimulus controls the model’s generation process.  The figure below shows Directional Stimulus Prompting compared with standard prompting.  Image source: Liu et al.2022 Benefits Of Advanced Prompt Engineering Large language models are probability machines and not fact-based machines. AI has the ability to produce outputs by predicting the probability of the next token (a unit of text). It does not always give correct and reliable answers. A perfect example is the phenomenon of AI hallucination, where the AI “makes things up” but makes it look as if it were true. The fabrication of information might not be obvious because of the AI’s tone and confidence, which makes it easier for users to believe the model. If you point out that the model has made a mistake, it will most likely agree with you and accept your suggested answer. Prompt engineering can help with the prevention of such issues.  Image source: Outshift by Cisco  Advanced Prompting Results In: Improved Accuracy and Relevance Context-based answers that directly answer the question, avoiding hallucination Enhanced Reasoning and Problem-Solving ToT prompting enables AI to tackle complicated problems and provide justified and rational answers.  Increased Efficiency Optimising prompts can lead to faster processing times. Reduced Bias Carefully crafted prompts and feedback loops can help prevent bias and provide objectivity in responses. Exploring the potential of AI models  Skills Used In Prompt Engineering A prompt engineer needs to be acquainted with certain knowledge fields and have a technical skillset to be good. You can also develop these skills as you try and learn prompting.  Understanding of AI and Machine Learning– Knowing how Artificial Intelligence functions, how models are trained, and the underlying architecture and technology is crucial for designing quality prompts.  Natural Language Processing- NLP is used for creating LLMs; it helps AI models understand, process and generate human language.  Familiarity with LLMs- Experience with models like GPT, PaLM2, etc. Knowing how a model behaves means you can predict the model’s response to a prompt and accordingly customise it.  Data analysis- Analysing model

Introduction to Prompt Engineering- Communicating with AI

In today’s rapidly progressing world, the use of artificial intelligence has been integrated into the daily lives of many, be it while working, studying or researching. Large language models (LLMs) such as Chat GPT, Perplexity, Gemini, Deepseek, etc, are easily accessible on the web, making our lives much more convenient. The widespread usage of Generative AI has led to the rise of prompt engineering, which has become the interface between humans and AI.  What is Prompt Engineering? While using AI models, you must have noticed that you don’t always get the desired results and have had to modify your prompt to guide the AI. Doing so, you unknowingly make use of prompt engineering. Prompt engineering is a relatively new discipline; it’s the practice of crafting your prompt to provide context in a way that helps AI models generate useful responses. A well-written prompt can steer a large language model towards your desired output.  The Importance of Prompt Engineering  If you’re wondering why prompt engineering is used, it helps in understanding the limitations and capacity of large language models. These skills are important to effectively interact with the model to get better responses. Prompting is the core of interaction with AI. The aim of prompt engineering is to: Direct the AI models’ behaviour, unlock their full potential, and ensure accurate results.  Techniques in Prompt Engineering Basic Prompt Engineering A good prompt requires the user to provide the task, context, and examples. A prompt such as “Write about GenAI” is too vague to get a proper response. Instead, a prompt such as “Write a concise, engaging haiku with the word ‘farduddle’. To do a ‘farduddle’ means to jump up and down really fast. Examples of sentences that use farduddle: When we won the game, we all started to farduddle in celebration. Seeing the artist perform my favourite song live made me want to farduddle out of joy.” This is an example of few shot prompting. There are different types of prompting useful in different situations: zero shot, one shot, and few shot. When you simply type a question and hit the enter key, it’s known as zero shot prompting since you don’t provide any examples to assist the AI. An example of zero shot prompting would be “Write an article about digital marketing”. With the same logic, one shot prompting refers to providing only one example along with your instruction and a desired output format, and few shot prompting translates to providing 2/more examples.  Improved Prompting The user plays an integral role in an AI model’s output as they write the prompt. Under prompt engineering, there are different features and techniques to be applied to prompts that can substantially elevate the standard of the result you receive.  Clear communication is essential. If your question is vague, the answer will be vague too. Hence, it’s necessary to provide context (additional background information) and frame concise, detailed and direct questions to produce a relevant response.  Providing the LLM with a tone and persona also enhances answers. This requires instructing the AI to assume a role/identity and specifying the emotion and attitude that should be conveyed through the language of the text. Assigning a tone and persona helps the AI adopt a perspective and style to craft the response.  After receiving the initial response, you can refine it by iteratively improving the prompt based on the previous interaction. Another effective method is asking the AI for feedback on how to provide better prompts and information you could include to better its contextual understanding. This tells you the issues the AI is facing and gives you a precise resolution as well.  Advanced Techniques Although these strategies are meant to enrich AI models’ replies, we need to consider the limitations of large language models and be careful about things such as context overload (overwhelming the AI by providing too much information). Keep in mind that AI assistants have constraints on processing abilities.  Tactics such as task breakdown, chain of thought, and rephrasing help us work with these limitations. If you have a complex request that requires the AI to execute multiple tasks, it’s best to break your request into smaller, more manageable prompts, each prompt building on the previous response. In addition to this, making use of chain of thought – prompting AI to explain its reasoning in steps – will give the AI time to think and support it in navigating through the question and finding the right answer. This method is especially useful for questions that require reasoning, e.g., “If there are 3 red balls and 5 blue balls in a bag, and you randomly draw two balls without replacement, what is the probability that both are red? Let’s think step by step.” It’s important to mention that you want the AI to go through the problem stepwise. You can even specify some of the steps to further direct the model.  If the AI is not able to generate a proper response or is overloaded, try simplifying the task. A suggestion for long prompts would be not to enter all the information at once. First, inform the AI about the goal to be achieved and the type of task you want it to perform; in a separate prompt, provide it with the tone, persona and context. An alternative method is to include all the information in one prompt; however, repeat the main aim/ instruction at the end. Sometimes, the AI tends to read and answer only the latter part of the prompt.  Final Thoughts Prompt engineering is a medium for talking to AI; it is the interface between human thinking and machine understanding. This subtle art has given us the opportunity to unleash the true potential of AI models and maximise their value. Prompt engineering will only develop and evolve further, with new prompting techniques aiming not only for effective communication but also for amalgamating the realm of AI with our reality.  There is much more to prompt engineering that hasn’t been covered in

The Future of Lazy Loading: AI, Edge Computing, and Beyond

Introduction In today’s digital landscape, where users expect lightning-fast and seamless web experiences, performance optimization is critical. Lazy loading has been a cornerstone technique in web development, enabling faster page loads by deferring the initialization of non-critical resources—such as images, videos, or JavaScript modules—until they are needed. This approach reduces initial page weight, conserves bandwidth, and enhances user satisfaction, particularly on resource-constrained devices or networks. However, as user expectations and technological capabilities evolve, lazy loading is undergoing a transformation. The integration of Artificial Intelligence (AI), Edge Computing, and emerging technologies like WebAssembly is redefining lazy loading, making it smarter, more adaptive, and user-centric. This comprehensive guide analyzes the evolution of lazy loading, provides practical code implementations with conceptual outputs, and explores future trends that promise to shape its next frontier(policy) frontier. Section 1: Traditional Lazy Loading Lazy loading, at its core, is about efficiency. By delaying the loading of non-essential resources until they are needed—typically when they enter the user’s viewport—developers can significantly reduce initial page load times. This is particularly effective for content-heavy websites, such as e-commerce platforms or media galleries. Code Implementation: Traditional Lazy Loading Here’s a classic example of lazy loading images using the IntersectionObserver API: How It Works The data-src attribute stores the actual image URL, while a lightweight placeholder (or no src) is used initially. The IntersectionObserver detects when the image enters the viewport, triggering the load of the actual image. Once loaded, the image is removed from observation to optimize performance. Output (Conceptual) When the page loads, only placeholders (e.g., low-resolution images or empty spaces) are visible. As the user scrolls, images load seamlessly as they enter the viewport, reducing initial load time and improving perceived performance. Limitations Traditional lazy loading relies on static rules, such as viewport visibility, which do not account for user behavior, device capabilities, or network conditions. This limitation has spurred the development of more intelligent approaches. Section 2: AI-Enhanced Lazy Loading AI is revolutionizing lazy loading by introducing predictive and context-aware capabilities. By analyzing user behavior, preferences, and contextual data, AI can preload resources that are likely to be needed, prioritize content based on relevance, and adapt to varying conditions like network speed or device type. Key Benefits Predictive Preloading: AI predicts user actions, such as scrolling to a specific section, and preloads relevant content. Intelligent Prioritization: Content is loaded based on engagement metrics or user intent, ensuring the most relevant resources are available first. Adaptive Loading: AI adjusts loading strategies based on real-time factors like bandwidth or battery level. Code Implementation: AI-Enhanced Lazy Loading Below is a simplified example simulating AI-driven lazy loading by prioritizing content based on a predicted interest score: How It Works Each content card (e.g., a product or article) is assigned an interest score based on factors like time of day and user region. Cards are sorted by score, and higher-priority content is loaded first with a slight delay to stagger resource demands. In a real-world scenario, predictInterestScore would leverage a machine learning model trained on user interaction referenced data. Output (Conceptual) Content cards with higher predicted interest scores (e.g., entertainment content in the evening for US users) load first, displaying richer content like high-resolution images or detailed descriptions. Lower-priority content loads later, optimizing resource usage and enhancing user experience. Real-World Example Companies like Netflix (Netflix) use AI-driven lazy loading to preload preview thumbnails or video player modules only when users hover or click, minimizing initial page load and tailoring content delivery based on user interaction. Section 3: Edge Computing and Lazy Loading Edge computing brings data processing and storage closer to the user through geographically distributed servers, significantly reducing latency. When combined with lazy loading, edge computing enables faster content delivery and smarter loading decisions based on user location, device type, and network conditions. Key Benefits Ultra-Fast Load Times: Content is served from nearby edge servers, reducing latency. Context-Aware Decisions: Edge servers can tailor content delivery based on real-time data, such as serving compressed images for slow connections. Scalability and Resilience: Edge-enabled lazy loading ensures efficient content delivery even under high traffic or network strain. Code Implementation: Edge-Aware Lazy Loading Using Next.js with Vercel’s Edge Middleware for context-aware lazy loading: How It Works The middleware sets headers for user location and device type. The React component dynamically imports either a lightweight or high-resolution component based on these headers. This ensures optimized content delivery tailored to the user’s context. Output (Conceptual) A mobile user in India loads a lightweight component, reducing data usage and improving speed. A desktop user in the US loads a high-resolution component, leveraging faster network capabilities. The result is a tailored, efficient user experience with minimal latency. Real-World Example Airbnb (Airbnb) employs edge-aware lazy loading to dynamically load listing details, maps, and images only when needed, optimizing performance for users on slower mobile networks. Section 4: WebAssembly and Lazy Loading WebAssembly (WASM) is a binary instruction format that enables high-performance code (e.g., written in C++ or Rust) to run in the browser at near-native speed. When integrated with lazy loading, WASM allows performance-critical modules, such as image decoders or video players, to be loaded on demand, reducing initial page weight. Potential Application High-Performance Modules: Lazy loading WASM modules for tasks like image processing or 3D rendering in gaming and visualization apps. Efficient Resource Use: Loading only the necessary modules when triggered by user actions, such as activating a feature. While specific WASM examples are less common, the potential for lazy-loaded, high-performance modules is significant, particularly for resource-intensive applications. Section 5: Future Trends in Lazy Loading The future of lazy loading is poised to be transformative, driven by advancements in AI, edge computing, and related technologies. Key trends include: Trend Description AI-Powered Preloading Predicting user behavior to preload relevant content proactively. Edge-Aware Lazy Loading Dynamic content delivery from edge servers based on user context. WASM Lazy Modules Efficient, high-performance binary modules loaded on demand. Personalization at Runtime Tailoring content loading based on user preferences and behavior. Lazy APIs & Microservices

Google Kubernetes

Deploying Your First Application To The Google Kubernetes Engine

Deploying your application from a Bitbucket repository to production in Google Kubernetes Engine (GKE) typically involves setting up a Continuous Integration/Continuous Deployment (CI/CD) pipeline. Google Cloud Build is a popular choice for this, as it integrates well with Bitbucket and GKE. Here’s a breakdown of the common steps and concepts involved: 1. Prerequisites: Google Cloud Project: You need an active Google Cloud project. Google Kubernetes Engine (GKE) Cluster: A running GKE cluster where you’ll deploy your application. Google Cloud APIs Enabled: Cloud Build API Kubernetes Engine API Artifact Registry API – for storing your Docker images. Secret Manager API Bitbucket Repository: Your application code should be in a Bitbucket repository. Service Account: A Google Cloud service account with the necessary permissions for Cloud Build to: Build and push Docker images to Artifact Registry. Deploy to your GKE cluster (e.g., Kubernetes Engine Developer role). Access any other Google Cloud resources your build process needs. Kubernetes Manifests: You’ll need Kubernetes YAML files (Deployment, Service, Ingress, etc.) that define how your application should run on GKE. 2. Core Steps for the CI/CD Pipeline: A. Connect Bitbucket to Google Cloud Build: Google Cloud Console: Go to the Cloud Build Triggers page. Click “Create trigger”. Select Bitbucket Cloud (or Bitbucket Server if you’re using a self-hosted instance) as your source. Follow the prompts to authorize Cloud Build to access your Bitbucket repositories. Select the specific Bitbucket repository you want to connect. Trigger Configuration: Event: Choose the event that triggers your build (e.g., “Push to a branch” for master or main branch for production deployments, or “Push new tag” for versioned releases). Source: Select your connected Bitbucket repository and specify the branch or tag pattern. Configuration: Choose “Cloud Build configuration file (yaml or json)”. Location: Specify the path to your cloudbuild.yaml file in your repository (e.g., cloudbuild.yaml). B. Define Your cloudbuild.yaml (Build Configuration File): This file, located in your Bitbucket repository, instructs Cloud Build on how to build your application, containerize it, push the image, and deploy it to GKE. A typical cloudbuild.yaml for GKE deployment would include the following steps: Build Docker Image: Use the docker builder to build your application’s Docker image. Tag the image with a unique identifier (e.g., Git commit SHA, branch name, or a version tag). Push Docker Image to Artifact Registry: Use the docker builder to push the newly built image to your chosen registry. Get GKE Cluster Credentials: Use the gcloud builder to authenticate and get credentials for your GKE cluster. This allows subsequent kubectl commands to interact with your cluster. Deploy to GKE and Expose service: Using kubectl directly: This provides more fine-grained control C. Manage Service Account Permissions: Ensure the Cloud Build service account (which is [PROJECT_NUMBER]@cloudbuild.gserviceaccount.com by default) has the necessary IAM roles: Cloud Build Editor (or Cloud Build Service Account) Storage Admin (for pushing to GCS if needed for intermediate artifacts) Artifact Registry Writer or Storage Object Admin (for pushing Docker images) Kubernetes Engine Developer (to deploy to GKE) Service Account User (if you’re using other service accounts during the build) 3. Example cloudbuild.yaml: 4. Example deployment.yaml: 5. Example service.yaml: Key Considerations: Environments (Dev, Staging, Production): It’s highly recommended to have separate GKE clusters or namespaces for different environments. Your Bitbucket pipeline can be configured to deploy to different environments based on branches (e.g., develop to dev, main to production) or tags. Rollback Strategy: Understand how to roll back deployments in GKE in case of issues. kubectl rollout undo is your friend. Monitoring and Logging: Integrate with Google Cloud’s monitoring (Cloud Monitoring) and logging (Cloud Logging) to observe your application’s health and troubleshoot problems. Testing: Implement automated tests (unit, integration, end-to-end) in your CI/CD pipeline before deployment to ensure the quality of your application. Helm or Kustomize: For more complex Kubernetes deployments, consider using Helm charts or Kustomize to manage your Kubernetes manifests, making them more modular and configurable. Network Access: If your GKE cluster is private, ensure Cloud Build has network access to the cluster’s control plane. This might involve setting up a private pool for Cloud Build. By following these steps, you can establish a robust CI/CD pipeline to automatically deploy your application from Bitbucket to Google Kubernetes Engine.

ai in security and surveillance featured

AI-Enabled Surveillance for Urban Safety

As India’s cities continue to grow faster than many Asian countries, law enforcement agencies are facing mounting pressure. Rapid urbanization, combined with a rising population and an increasing number of vehicles, has created a complex environment for police forces in cities like Delhi, Mumbai, and Bengaluru. Between 2018 and 2022, India witnessed over 500,000 fatalities, with approximately 92% occurring in urban areas. Major cities are struggling with severe traffic violations and congestion, which directly contribute to fatalities, rising crime rates, and the difficulty of maintaining public order in overcrowded, fast-moving environments. The challenges of urban infrastructure—narrow roads, chaotic traffic, and insufficient public transportation—complicate policing efforts even further. Additionally, the burden on law enforcement is intensified by India’s police-to-population ratio, which stands at 1:800, far below the global standard of 1:450. In practice, this means that police officers are often tasked with managing hundreds or even thousands of individuals per shift. As a result, response times slow, workloads increase, and officers experience burnout. In high-pressure situations, officers are expected to balance multiple roles—traffic management, crime prevention, and investigation. Traditional policing methods, which rely heavily on manpower, are no longer sufficient to meet the growing demands of urban life. For Easiest understanding of the how traffic Voilation is interconnected with its factor are shown below. Vision Analytics: Transforming Urban Policing As part of India’s Smart City Mission, technologies like Vision Analytics have the potential to transform law enforcement. Vision Analytics leverages AI technologies such as machine learning, computer vision, and real-time data analysis to streamline policing operations. This technology automates critical processes, improves operational efficiency, and enhances overall safety. Here’s how Vision Analytics can impact various aspects of urban safety: Use Case Impact Monitor Traffic Identifies traffic bottlenecks and congestion. Tracks vehicles involved in illicit activities like smuggling or hit and run accidents. Detects wrong way driving and sends real-time alerts to prevent accidents. RealTime Crime Detection Enables faster emergency response. Provides immediate alerts for crime prevention. Prevent Traffic Violations Reduces human error. Automates fines and penalties, ensuring timely enforcement. Track Criminal Activities Tracks suspicious vehicles in real-time. Aids in investigations by providing valuable data. Identify Road Hazards Prevents accidents by alerting authorities. Reduces roadkill and helps fix road damage faster. Monitor Public Spaces Prevents crimes before they escalate. Enhances safety by alerting police to potential threats. Automate Traffic Law Enforcement Reduces physical patrol needs. Ensures consistent enforcement of traffic laws. Support Emergency Services Reduces response time for emergency services. Improves coordination between law enforcement and emergency teams. Monitor Pedestrian Safety Increases pedestrian safety. Alerts pedestrians and traffic authorities to unsafe crossings. Building the Infrastructure for Vision Analytics For Vision Analytics to reach its full potential, India must strengthen its technological infrastructure. Key steps include: High-Resolution Surveillance Systems: Installing cameras that can capture high-quality footage to monitor urban areas effectively. Data Storage and Processing: Ensuring systems can process large volumes of data in real-time for rapid decision-making. Training for Police Personnel: Equipping law enforcement officers with the necessary skills to operate AI-driven tools and interpret the data generated. Reliable Data Networks: Establishing strong and consistent networks for seamless data transmission between surveillance systems and central control rooms. To visualize how Vision Analytics works, think of it as an interconnected network where each component, like real-time monitoring, AI surveillance, and automated law enforcement, works together to improve urban safety. Whether it’s optimizing traffic flow, detecting wrong-way driving, or tracking suspicious vehicles, the system continuously processes data, sends alerts to officers, and enables quicker responses. All components complement each other, making Vision Analytics a powerful tool for preventing crimes and accidents. Below is an image that shows how these components are interrelated, helping to create a safer environment for everyone. Key Benefits of Vision Analytics Responsiveness: Vision Analytics enables real-time monitoring and instant alerts, allowing law enforcement to respond quickly to incidents. This helps prevent crimes and accidents before they escalate. Auditability: The system offers a transparent and auditable record of all activities, making it easier to track, review, and ensure accountability in law enforcement operations. This improves oversight and supports compliance. Data-Driven Analytics: By continuously collecting and analyzing data, Vision Analytics provides actionable insights. These data-driven analytics enable law enforcement to make informed decisions, improving crime prevention, traffic management, and urban safety strategies. Enhanced Public Safety: Vision Analytics helps detect suspicious activity and hazardous conditions in real time. This reduces response times and allows for proactive intervention in high-risk situations, ultimately improving public safety. Challenges in Implementing Vision Analytics in India While Vision Analytics presents significant potential, several challenges exist. However, these challenges can be addressed with careful planning and execution: Infrastructure Limitations: Many cities lack the necessary surveillance cameras and internet infrastructure. Phased investments can bridge these gaps, allowing cities to adopt AI-powered systems effectively. Financial Constraints: The initial cost of AI systems may be high, but a cost-benefit analysis shows that long-term savings through improved efficiency and reduced crime outweigh the investment. Training Gaps: Many officers may lack the technical expertise to operate AI-driven tools. However, proper training programs can address this skill gap, enabling officers to fully leverage the power of Vision Analytics. Privacy Concerns: Surveillance technologies, especially facial recognition, raise privacy issues. Implementing transparent policies and strict data protection regulations will help build public trust and ensure the responsible use of data. System Integration: Integrating AI with existing law enforcement systems can be complex. Focusing on interoperability will ensure seamless integration and maximize the benefits of AI technologies. Real-World Impact of Vision Analytics Despite these challenges, Vision Analytics has already proven successful in cities globally. For example, in China’s Smart City Initiative, AI systems integrated with surveillance cameras monitor traffic and detect crimes in real time. This proactive approach has reduced road accidents and improved public safety. In India, imagine a crowded market in Noida. A man snatches a woman’s mobile phone and tries to flee. Thanks to Vision Analytics, the AI detects the suspicious movement and immediately alerts nearby officers. Within moments, the police respond and intercept the

Optimizing Frontend Performance with Throttling and Debouncing

Imagine you’re typing in a search bar, and with every keystroke, a flood of API calls bombard the backend. Or worse—you’re scrolling through a page, and an event listener fires hundreds of times per second, causing lag. The result? A sluggish UI, frustrated users, and an overwhelmed server. That’s where throttling and debouncing come to the rescue! These two performance optimization techniques are the unsung heroes of frontend development. Let’s break them down. What is Throttling? Think of throttling as a traffic cop directing cars at a busy intersection—it controls the flow, allowing only a certain number of cars to pass at regular intervals. Similarly, throttling limits how often a function executes over time. Even if an event is triggered multiple times in rapid succession, throttling ensures that the function runs at fixed intervals instead of executing every single time the event occurs. How Does It Work? Throttling works by enforcing a cooldown period. When an event occurs: The function executes immediately (or after an initial delay). During the cooldown period, additional event triggers are ignored. Once the cooldown period ends, the function can execute again if the event is triggered. This prevents excessive function calls, optimizing performance and reducing unnecessary computations. Where to Use Throttling? 1. Infinite Scrolling – Prevent Excessive API Calls When Users Scroll Infinite scrolling is a common UI pattern where more data loads as the user scrolls down. However, if an API request is triggered on every single scroll event, it can overload the server and slow down the user experience. How Throttling Helps: By throttling the scroll event, we ensure that the API request fires at fixed intervals (e.g., once every second), even if the user scrolls continuously. This reduces unnecessary network calls while still ensuring a seamless scrolling experience. Example: Social media feeds, e-commerce product listings. 2. Window Resize Events – Avoid Unnecessary Layout Recalculations When a user resizes a browser window, the resize event fires many times per second. If we execute expensive computations (like recalculating layout positions) on each event, the UI can become laggy. How Throttling Helps: By throttling the resize event, layout recalculations only happen at set intervals (e.g., every 200ms), improving browser performance and avoiding unnecessary re-renders. Example: Responsive web designs, dynamic UI resizing. 3. Button Clicks – Prevent Accidental Multiple Requests Users sometimes click buttons multiple times, either due to impatience or accidental double-clicks. If each click triggers an API call (like submitting a form or processing a payment), this can lead to duplicate requests. How Throttling Helps: By throttling button clicks, we ensure that only one request is processed within a set period (e.g., once every 500ms), ignoring rapid consecutive clicks. Example: Payment processing buttons, form submission buttons. Example: Throttle in Action Using lodash.throttle, we can ensure that an event fires at most once every X milliseconds: Now, even if the user scrolls aggressively, the function executes at most once per second. What is Debouncing? Imagine you’re in a conversation, and someone keeps interrupting. Instead of responding to every word they say, you wait until they finish speaking before replying. This is exactly how debouncing works in programming! Debouncing ensures that a function only executes after a certain delay and resets the timer if the event occurs again within that delay. This means that the function only runs when the event stops happening for a specified period. How Does It Work? When an event (like keypress, input, or resize) occurs: A timer starts for the function to execute after a delay. If the event happens again within that delay, the timer resets. The function only runs after the event stops triggering for the entire delay duration. This prevents unnecessary function executions and optimizes performance, especially in situations where rapid user inputs or frequent events occur. Where to Use Debouncing? 1. Search Input Fields – Avoid Making API Calls for Every Keystroke When a user types in a search bar, each keystroke can trigger an API request to fetch matching results. If an API call is made on every single keypress, it can overload the server and slow down performance. How Debouncing Helps: By using debouncing, the API request is only sent after the user stops typing for a specified delay (e.g., 500ms). This ensures that unnecessary API calls are prevented, and only the final search query is processed. Example: Live search suggestions in Google, e-commerce product searches. 2. Form Validations – Wait for the User to Stop Typing Before Validating Input Validating input fields (e.g., checking if an email is valid or if a username is available) on every keystroke can cause performance issues and unnecessary server requests. How Debouncing Helps: By waiting until the user finishes typing, the validation check is only triggered once instead of multiple times, reducing redundant checks and improving responsiveness. Example: Email verification forms, username availability checks. 3. Auto-Save Features – Save Data Only After the User Stops Typing In applications where user input is auto-saved (e.g., note-taking apps, online editors), saving on every keystroke can lead to excessive writes to the server or database. How Debouncing Helps: With debouncing, auto-save is triggered only after the user stops typing for a set period (e.g., 1 second). This prevents unnecessary writes and improves efficiency. Example: Google Docs auto-save, note-taking apps, online surveys. Example: Debounce in Action Using lodash.debounce, we delay execution until the user pauses for a specified time: Now, API calls only happen when the user stops typing for 500ms—improving performance! Throttling & Debouncing Performance Optimization & Complexity Analysis Both throttling and debouncing are essential optimization techniques that help control the execution of frequently triggered functions, improving performance and efficiency. In terms of time complexity, without these optimizations, an event handler executes O(N) times, where N represents the number of event triggers. With throttling, execution is reduced to O(N/K) (where K is the throttle interval), ensuring controlled execution at fixed intervals. Debouncing, on the other hand, reduces execution to O(1) or O(N/K) by ensuring the function runs only after a

Centralized Logging using Elasticsearch & Kibana

Centralized Logging using Elasticsearch & Kibana

In today’s world, when everything is moving to the digital ecosystem, maintaining robust applications is always a challenge. Because of critical applications and use cases, it’s always tough to have an application which has zero downtime. Even big giants couldn’t save us from the famous Y2K problem.  Effectively, to have this minimal downtime, it’s always important to determine the cause of the problem as quickly as possible. For Web applications, there are always two phases from which a problem can be traced, i.e, frontend and backend applications.  In this article, we’ll primarily focus on how to trace backend application logs running on Kubernetes server.  Why choose Elasticsearch for backend logging? So, when we talk of enterprise level solutions, we always move to tools which are best suited for safety and security. Elasticsearch is the most famous multitenant-capable full text search engine which comes with a user-friendly interface and gives other important features as well. Elasticsearch is mainly used for large microservice based applications, where concepts and logic are distributed across multiple microservices.  We always talk of ELK as a techstack, which stands for E- Elastic, L- Logstash and K-Kibana. Elastic, logstash and kibana are actually main pillars for Elasticsearch. We have an agent called Filebeat as well which is responsible for acting as a bridge between Kubernetes servers and Logstash.  Getting Started with ELK Stack Let’s go over how to get started with the ELK stack on Kubernetes before getting into the configuration files and setup. To build up your logging system, you must do the following:  Prerequisites  A running Kubernetes cluster  Helm package manager installed  kubectl command-line tool configured  Access to pull images from Elastic’s container registry  Installation Steps  Add the Elastic Helm repository helm repo add elastic https://helm.elastic.co helm repo update  Install Elasticsearch # Create a namespace for ELK stack kubectl create namespace logging # Install Elasticsearch using Helm helm install elasticsearch elastic/elasticsearch \ –namespace logging \ –set minimumMasterNodes=1 \ –set replicas=1400;”>  Install Kibana helm install kibana elastic/kibana \ –namespace logging  Install Filebeat helm install filebeat elastic/filebeat \ –namespace logging  Install Logstash helm install logstash elastic/logstash \ –namespace logging  Verifying the Installation  Check if all pods are running correctly: kubectl get pods -n logging You should see pods for Elasticsearch, Kibana, Filebeat, and Logstash in a Running state. Accessing Kibana By default, Kibana is exposed through a Kubernetes service. You can access it by: 1. Port forwarding:  kubectl port-forward service/kibana-kibana 5601:5601 -n logging 2. Visit http://localhost:5601 in your browser  Initial Configuration  Once you have access to Kibana:  Create an index pattern to match your logs ○ Navigate to Stack Management → Index Patterns ○ Click “Create index pattern” ○ Use “filebeat-*” as your index pattern ○ Select “@timestamp” as the time field Basic Security Setup ○ Use the default elastic user credentials ○ It’s recommended to change the default password ○ Set up role-based access control for your team Now that you have the basic installation and configuration complete, you can proceed with the detailed configuration files mentioned in the following sections.  How does the solution work? So as I explained above, Elasticsearch as a solution can be used in terms of microservice based architecture. When different microservices are running on different pods in a kubernetes server, sending logs to a logging workspace like Kibana is actually a challenge. But Elasticsearch has a proper solution to tackle this challenge using different tools of ELK techstack.  Kubernetes Servers → Filebeat → Logstash → Kibana 1. Reading logs from Kubernetes server : An agent is required to bring the logs from Kubernetes server to Logstash. Filebeat acts as an agent in this case, we can install filebeat inside our kubernetes server and it will be responsible for reading logs from the server. In kubernetes, logs are stored inside /var/log/pods and /var/log/containers. This filebeat agent continuously reads logs from these locations and then dispatches logs to logstash.  There are three files that need to be created for configuring filebeat within Kubernetes server. a. Configmap.yaml : it specifies the configurations of file beat agent and specifies the output location to logstash pod as well. b. Daemonset.yaml : it specifies the specifications for file beat agent pod, like memory size etc. c. Sa.yaml : this file is responsible for storing service account settings of the server.  2. Logstash : Logstash is responsible for consuming logs from filebeat agents and then sends logs to Kibana dashboard. We connect to Elasticsearch inside logstash itself. Also, logstash is responsible for organizing the logs into different indexes so that we can easily classify logs from different microservices or we can organize them into high, low or medium priority logs.  There are three files that needs to be created for configuring logstash within Kubernetes server a. Configmap.yaml : It specifies the configurations of logstash and also specifies the format in which logs are being organized inside the Kibana dashboard. b. Daemonset.yaml : It is responsible for integration with elasticsearch cluster and also collects log streams from filebeat.  c. Sa.yaml : this file is responsible for storing service account settings of the server.  At this stage, we have made all required configurations in kubernetes server and now the configurations needs to be made inside Elasticsearch cluster. 3. Kibana : Kibana is the user interface for viewing logs. It stores logs in different indices which helps in organizing and visualizing the logs in the more structured way. As Elasticsearch operates on non relational data, we can store data in any format inside Elasticsearch clusters. When not to choose Elasticsearch? Elasticsearch is an expensive tool and it should be used only if we are designing an enterprise level solution or if our application is following microservice based architecture.  But if we follow monolithic architecture, then using elasticsearch for storing centralized logs will be an expensive choice. Because monolithic applications often generate a large amount of log data, especially if it’s logging every request, error, or event we hardly emphasize on mapping logs in different indices. As monolithic applications generate too

Scroll to Top