Learning Hub

Deploying your application from a Bitbucket repository to production in Google Kubernetes Engine (GKE) typically involves setting up a Continuous Integration/Continuous Deployment (CI/CD) pipeline. Google Cloud Build is a popular choice for this, as it integrates well with Bitbucket and GKE. Here’s a breakdown of the common steps and concepts involved: 1. Prerequisites: Google Cloud Project: You need an active Google Cloud project. Google Kubernetes Engine (GKE) Cluster: A running GKE cluster where you’ll deploy your application. Google Cloud APIs Enabled: Cloud Build API Kubernetes Engine API Artifact Registry API – for storing your Docker images. Secret Manager API Bitbucket Repository: Your application code should be in a Bitbucket repository. Service Account: A Google Cloud service account with the necessary permissions for Cloud Build to: Build and push Docker images to Artifact Registry. Deploy to your GKE cluster (e.g., Kubernetes Engine Developer role). Access any other Google Cloud resources your build process needs. Kubernetes Manifests: You’ll need Kubernetes YAML files (Deployment, Service, Ingress, etc.) that define how your application should run on GKE. 2. Core Steps for the CI/CD Pipeline: A. Connect Bitbucket to Google Cloud Build: Google Cloud Console: Go to the Cloud Build Triggers page. Click “Create trigger”. Select Bitbucket Cloud (or Bitbucket Server if you’re using a self-hosted instance) as your source. Follow the prompts to authorize Cloud Build to access your Bitbucket repositories. Select the specific Bitbucket repository you want to connect. Trigger Configuration: Event: Choose the event that triggers your build (e.g., “Push to a branch” for master or main branch for production deployments, or “Push new tag” for versioned releases). Source: Select your connected Bitbucket repository and specify the branch or tag pattern. Configuration: Choose “Cloud Build configuration file (yaml or json)”. Location: Specify the path to your cloudbuild.yaml file in your repository (e.g., cloudbuild.yaml). B. Define Your cloudbuild.yaml (Build Configuration File): This file, located in your Bitbucket repository, instructs Cloud Build on how to build your application, containerize it, push the image, and deploy it to GKE. A typical cloudbuild.yaml for GKE deployment would include the following steps: Build Docker Image: Use the docker builder to build your application’s Docker image. Tag the image with a unique identifier (e.g., Git commit SHA, branch name, or a version tag). Push Docker Image to Artifact Registry: Use the docker builder to push the newly built image to your chosen registry. Get GKE Cluster Credentials: Use the gcloud builder to authenticate and get credentials for your GKE cluster. This allows subsequent kubectl commands to interact with your cluster. Deploy to GKE and Expose service: Using kubectl directly: This provides more fine-grained control C. Manage Service Account Permissions: Ensure the Cloud Build service account (which is [PROJECT_NUMBER]@cloudbuild.gserviceaccount.com by default) has the necessary IAM roles: Cloud Build Editor (or Cloud Build Service Account) Storage Admin (for pushing to GCS if needed for intermediate artifacts) Artifact Registry Writer or Storage Object Admin (for pushing Docker images) Kubernetes Engine Developer (to deploy to GKE) Service Account User (if you’re using other service accounts during the build) 3. Example cloudbuild.yaml: 4. Example deployment.yaml: 5. Example service.yaml: Key Considerations: Environments (Dev, Staging, Production): It’s highly recommended to have separate GKE clusters or namespaces for different environments. Your Bitbucket pipeline can be configured to deploy to different environments based on branches (e.g., develop to dev, main to production) or tags. Rollback Strategy: Understand how to roll back deployments in GKE in case of issues. kubectl rollout undo is your friend. Monitoring and Logging: Integrate with Google Cloud’s monitoring (Cloud Monitoring) and logging (Cloud Logging) to observe your application’s health and troubleshoot problems. Testing: Implement automated tests (unit, integration, end-to-end) in your CI/CD pipeline before deployment to ensure the quality of your application. Helm or Kustomize: For more complex Kubernetes deployments, consider using Helm charts or Kustomize to manage your Kubernetes manifests, making them more modular and configurable. Network Access: If your GKE cluster is private, ensure Cloud Build has network access to the cluster’s control plane. This might involve setting up a private pool for Cloud Build. By following these steps, you can establish a robust CI/CD pipeline to automatically deploy your application from Bitbucket to Google Kubernetes Engine.

As India’s cities continue to grow faster than many Asian countries, law enforcement agencies are facing mounting pressure. Rapid urbanization, combined with a rising population and an increasing number of vehicles, has created a complex environment for police forces in cities like Delhi, Mumbai, and Bengaluru. Between 2018 and 2022, India witnessed over 500,000 fatalities, with approximately 92% occurring in urban areas. Major cities are struggling with severe traffic violations and congestion, which directly contribute to fatalities, rising crime rates, and the difficulty of maintaining public order in overcrowded, fast-moving environments. The challenges of urban infrastructure—narrow roads, chaotic traffic, and insufficient public transportation—complicate policing efforts even further. Additionally, the burden on law enforcement is intensified by India’s police-to-population ratio, which stands at 1:800, far below the global standard of 1:450. In practice, this means that police officers are often tasked with managing hundreds or even thousands of individuals per shift. As a result, response times slow, workloads increase, and officers experience burnout. In high-pressure situations, officers are expected to balance multiple roles—traffic management, crime prevention, and investigation. Traditional policing methods, which rely heavily on manpower, are no longer sufficient to meet the growing demands of urban life. For Easiest understanding of the how traffic Voilation is interconnected with its factor are shown below. Vision Analytics: Transforming Urban Policing As part of India’s Smart City Mission, technologies like Vision Analytics have the potential to transform law enforcement. Vision Analytics leverages AI technologies such as machine learning, computer vision, and real-time data analysis to streamline policing operations. This technology automates critical processes, improves operational efficiency, and enhances overall safety. Here’s how Vision Analytics can impact various aspects of urban safety: Use Case Impact Monitor Traffic Identifies traffic bottlenecks and congestion. Tracks vehicles involved in illicit activities like smuggling or hit and run accidents. Detects wrong way driving and sends real-time alerts to prevent accidents. RealTime Crime Detection Enables faster emergency response. Provides immediate alerts for crime prevention. Prevent Traffic Violations Reduces human error. Automates fines and penalties, ensuring timely enforcement. Track Criminal Activities Tracks suspicious vehicles in real-time. Aids in investigations by providing valuable data. Identify Road Hazards Prevents accidents by alerting authorities. Reduces roadkill and helps fix road damage faster. Monitor Public Spaces Prevents crimes before they escalate. Enhances safety by alerting police to potential threats. Automate Traffic Law Enforcement Reduces physical patrol needs. Ensures consistent enforcement of traffic laws. Support Emergency Services Reduces response time for emergency services. Improves coordination between law enforcement and emergency teams. Monitor Pedestrian Safety Increases pedestrian safety. Alerts pedestrians and traffic authorities to unsafe crossings. Building the Infrastructure for Vision Analytics For Vision Analytics to reach its full potential, India must strengthen its technological infrastructure. Key steps include: High-Resolution Surveillance Systems: Installing cameras that can capture high-quality footage to monitor urban areas effectively. Data Storage and Processing: Ensuring systems can process large volumes of data in real-time for rapid decision-making. Training for Police Personnel: Equipping law enforcement officers with the necessary skills to operate AI-driven tools and interpret the data generated. Reliable Data Networks: Establishing strong and consistent networks for seamless data transmission between surveillance systems and central control rooms. To visualize how Vision Analytics works, think of it as an interconnected network where each component, like real-time monitoring, AI surveillance, and automated law enforcement, works together to improve urban safety. Whether it’s optimizing traffic flow, detecting wrong-way driving, or tracking suspicious vehicles, the system continuously processes data, sends alerts to officers, and enables quicker responses. All components complement each other, making Vision Analytics a powerful tool for preventing crimes and accidents. Below is an image that shows how these components are interrelated, helping to create a safer environment for everyone. Key Benefits of Vision Analytics Responsiveness: Vision Analytics enables real-time monitoring and instant alerts, allowing law enforcement to respond quickly to incidents. This helps prevent crimes and accidents before they escalate. Auditability: The system offers a transparent and auditable record of all activities, making it easier to track, review, and ensure accountability in law enforcement operations. This improves oversight and supports compliance. Data-Driven Analytics: By continuously collecting and analyzing data, Vision Analytics provides actionable insights. These data-driven analytics enable law enforcement to make informed decisions, improving crime prevention, traffic management, and urban safety strategies. Enhanced Public Safety: Vision Analytics helps detect suspicious activity and hazardous conditions in real time. This reduces response times and allows for proactive intervention in high-risk situations, ultimately improving public safety. Challenges in Implementing Vision Analytics in India While Vision Analytics presents significant potential, several challenges exist. However, these challenges can be addressed with careful planning and execution: Infrastructure Limitations: Many cities lack the necessary surveillance cameras and internet infrastructure. Phased investments can bridge these gaps, allowing cities to adopt AI-powered systems effectively. Financial Constraints: The initial cost of AI systems may be high, but a cost-benefit analysis shows that long-term savings through improved efficiency and reduced crime outweigh the investment. Training Gaps: Many officers may lack the technical expertise to operate AI-driven tools. However, proper training programs can address this skill gap, enabling officers to fully leverage the power of Vision Analytics. Privacy Concerns: Surveillance technologies, especially facial recognition, raise privacy issues. Implementing transparent policies and strict data protection regulations will help build public trust and ensure the responsible use of data. System Integration: Integrating AI with existing law enforcement systems can be complex. Focusing on interoperability will ensure seamless integration and maximize the benefits of AI technologies. Real-World Impact of Vision Analytics Despite these challenges, Vision Analytics has already proven successful in cities globally. For example, in China’s Smart City Initiative, AI systems integrated with surveillance cameras monitor traffic and detect crimes in real time. This proactive approach has reduced road accidents and improved public safety. In India, imagine a crowded market in Noida. A man snatches a woman’s mobile phone and tries to flee. Thanks to Vision Analytics, the AI detects the suspicious movement and immediately alerts nearby officers. Within moments, the police respond and intercept the

Imagine you’re typing in a search bar, and with every keystroke, a flood of API calls bombard the backend. Or worse—you’re scrolling through a page, and an event listener fires hundreds of times per second, causing lag. The result? A sluggish UI, frustrated users, and an overwhelmed server. That’s where throttling and debouncing come to the rescue! These two performance optimization techniques are the unsung heroes of frontend development. Let’s break them down. What is Throttling? Think of throttling as a traffic cop directing cars at a busy intersection—it controls the flow, allowing only a certain number of cars to pass at regular intervals. Similarly, throttling limits how often a function executes over time. Even if an event is triggered multiple times in rapid succession, throttling ensures that the function runs at fixed intervals instead of executing every single time the event occurs. How Does It Work? Throttling works by enforcing a cooldown period. When an event occurs: The function executes immediately (or after an initial delay). During the cooldown period, additional event triggers are ignored. Once the cooldown period ends, the function can execute again if the event is triggered. This prevents excessive function calls, optimizing performance and reducing unnecessary computations. Where to Use Throttling? 1. Infinite Scrolling – Prevent Excessive API Calls When Users Scroll Infinite scrolling is a common UI pattern where more data loads as the user scrolls down. However, if an API request is triggered on every single scroll event, it can overload the server and slow down the user experience. How Throttling Helps: By throttling the scroll event, we ensure that the API request fires at fixed intervals (e.g., once every second), even if the user scrolls continuously. This reduces unnecessary network calls while still ensuring a seamless scrolling experience. Example: Social media feeds, e-commerce product listings. 2. Window Resize Events – Avoid Unnecessary Layout Recalculations When a user resizes a browser window, the resize event fires many times per second. If we execute expensive computations (like recalculating layout positions) on each event, the UI can become laggy. How Throttling Helps: By throttling the resize event, layout recalculations only happen at set intervals (e.g., every 200ms), improving browser performance and avoiding unnecessary re-renders. Example: Responsive web designs, dynamic UI resizing. 3. Button Clicks – Prevent Accidental Multiple Requests Users sometimes click buttons multiple times, either due to impatience or accidental double-clicks. If each click triggers an API call (like submitting a form or processing a payment), this can lead to duplicate requests. How Throttling Helps: By throttling button clicks, we ensure that only one request is processed within a set period (e.g., once every 500ms), ignoring rapid consecutive clicks. Example: Payment processing buttons, form submission buttons. Example: Throttle in Action Using lodash.throttle, we can ensure that an event fires at most once every X milliseconds: Now, even if the user scrolls aggressively, the function executes at most once per second. What is Debouncing? Imagine you’re in a conversation, and someone keeps interrupting. Instead of responding to every word they say, you wait until they finish speaking before replying. This is exactly how debouncing works in programming! Debouncing ensures that a function only executes after a certain delay and resets the timer if the event occurs again within that delay. This means that the function only runs when the event stops happening for a specified period. How Does It Work? When an event (like keypress, input, or resize) occurs: A timer starts for the function to execute after a delay. If the event happens again within that delay, the timer resets. The function only runs after the event stops triggering for the entire delay duration. This prevents unnecessary function executions and optimizes performance, especially in situations where rapid user inputs or frequent events occur. Where to Use Debouncing? 1. Search Input Fields – Avoid Making API Calls for Every Keystroke When a user types in a search bar, each keystroke can trigger an API request to fetch matching results. If an API call is made on every single keypress, it can overload the server and slow down performance. How Debouncing Helps: By using debouncing, the API request is only sent after the user stops typing for a specified delay (e.g., 500ms). This ensures that unnecessary API calls are prevented, and only the final search query is processed. Example: Live search suggestions in Google, e-commerce product searches. 2. Form Validations – Wait for the User to Stop Typing Before Validating Input Validating input fields (e.g., checking if an email is valid or if a username is available) on every keystroke can cause performance issues and unnecessary server requests. How Debouncing Helps: By waiting until the user finishes typing, the validation check is only triggered once instead of multiple times, reducing redundant checks and improving responsiveness. Example: Email verification forms, username availability checks. 3. Auto-Save Features – Save Data Only After the User Stops Typing In applications where user input is auto-saved (e.g., note-taking apps, online editors), saving on every keystroke can lead to excessive writes to the server or database. How Debouncing Helps: With debouncing, auto-save is triggered only after the user stops typing for a set period (e.g., 1 second). This prevents unnecessary writes and improves efficiency. Example: Google Docs auto-save, note-taking apps, online surveys. Example: Debounce in Action Using lodash.debounce, we delay execution until the user pauses for a specified time: Now, API calls only happen when the user stops typing for 500ms—improving performance! Throttling & Debouncing Performance Optimization & Complexity Analysis Both throttling and debouncing are essential optimization techniques that help control the execution of frequently triggered functions, improving performance and efficiency. In terms of time complexity, without these optimizations, an event handler executes O(N) times, where N represents the number of event triggers. With throttling, execution is reduced to O(N/K) (where K is the throttle interval), ensuring controlled execution at fixed intervals. Debouncing, on the other hand, reduces execution to O(1) or O(N/K) by ensuring the function runs only after a

In today’s world, when everything is moving to the digital ecosystem, maintaining robust applications is always a challenge. Because of critical applications and use cases, it’s always tough to have an application which has zero downtime. Even big giants couldn’t save us from the famous Y2K problem.  Effectively, to have this minimal downtime, it’s always important to determine the cause of the problem as quickly as possible. For Web applications, there are always two phases from which a problem can be traced, i.e, frontend and backend applications.  In this article, we’ll primarily focus on how to trace backend application logs running on Kubernetes server.  Why choose Elasticsearch for backend logging? So, when we talk of enterprise level solutions, we always move to tools which are best suited for safety and security. Elasticsearch is the most famous multitenant-capable full text search engine which comes with a user-friendly interface and gives other important features as well. Elasticsearch is mainly used for large microservice based applications, where concepts and logic are distributed across multiple microservices.  We always talk of ELK as a techstack, which stands for E- Elastic, L- Logstash and K-Kibana. Elastic, logstash and kibana are actually main pillars for Elasticsearch. We have an agent called Filebeat as well which is responsible for acting as a bridge between Kubernetes servers and Logstash.  Getting Started with ELK Stack Let’s go over how to get started with the ELK stack on Kubernetes before getting into the configuration files and setup. To build up your logging system, you must do the following:  Prerequisites  A running Kubernetes cluster  Helm package manager installed  kubectl command-line tool configured  Access to pull images from Elastic’s container registry  Installation Steps  Add the Elastic Helm repository helm repo add elastic https://helm.elastic.co helm repo update  Install Elasticsearch # Create a namespace for ELK stack kubectl create namespace logging # Install Elasticsearch using Helm helm install elasticsearch elastic/elasticsearch \ –namespace logging \ –set minimumMasterNodes=1 \ –set replicas=1400;”>  Install Kibana helm install kibana elastic/kibana \ –namespace logging  Install Filebeat helm install filebeat elastic/filebeat \ –namespace logging  Install Logstash helm install logstash elastic/logstash \ –namespace logging  Verifying the Installation  Check if all pods are running correctly: kubectl get pods -n logging You should see pods for Elasticsearch, Kibana, Filebeat, and Logstash in a Running state. Accessing Kibana By default, Kibana is exposed through a Kubernetes service. You can access it by: 1. Port forwarding:  kubectl port-forward service/kibana-kibana 5601:5601 -n logging 2. Visit http://localhost:5601 in your browser  Initial Configuration  Once you have access to Kibana:  Create an index pattern to match your logs ○ Navigate to Stack Management → Index Patterns ○ Click “Create index pattern” ○ Use “filebeat-*” as your index pattern ○ Select “@timestamp” as the time field Basic Security Setup ○ Use the default elastic user credentials ○ It’s recommended to change the default password ○ Set up role-based access control for your team Now that you have the basic installation and configuration complete, you can proceed with the detailed configuration files mentioned in the following sections.  How does the solution work? So as I explained above, Elasticsearch as a solution can be used in terms of microservice based architecture. When different microservices are running on different pods in a kubernetes server, sending logs to a logging workspace like Kibana is actually a challenge. But Elasticsearch has a proper solution to tackle this challenge using different tools of ELK techstack.  Kubernetes Servers → Filebeat → Logstash → Kibana 1. Reading logs from Kubernetes server : An agent is required to bring the logs from Kubernetes server to Logstash. Filebeat acts as an agent in this case, we can install filebeat inside our kubernetes server and it will be responsible for reading logs from the server. In kubernetes, logs are stored inside /var/log/pods and /var/log/containers. This filebeat agent continuously reads logs from these locations and then dispatches logs to logstash.  There are three files that need to be created for configuring filebeat within Kubernetes server. a. Configmap.yaml : it specifies the configurations of file beat agent and specifies the output location to logstash pod as well. b. Daemonset.yaml : it specifies the specifications for file beat agent pod, like memory size etc. c. Sa.yaml : this file is responsible for storing service account settings of the server.  2. Logstash : Logstash is responsible for consuming logs from filebeat agents and then sends logs to Kibana dashboard. We connect to Elasticsearch inside logstash itself. Also, logstash is responsible for organizing the logs into different indexes so that we can easily classify logs from different microservices or we can organize them into high, low or medium priority logs.  There are three files that needs to be created for configuring logstash within Kubernetes server a. Configmap.yaml : It specifies the configurations of logstash and also specifies the format in which logs are being organized inside the Kibana dashboard. b. Daemonset.yaml : It is responsible for integration with elasticsearch cluster and also collects log streams from filebeat.  c. Sa.yaml : this file is responsible for storing service account settings of the server.  At this stage, we have made all required configurations in kubernetes server and now the configurations needs to be made inside Elasticsearch cluster. 3. Kibana : Kibana is the user interface for viewing logs. It stores logs in different indices which helps in organizing and visualizing the logs in the more structured way. As Elasticsearch operates on non relational data, we can store data in any format inside Elasticsearch clusters. When not to choose Elasticsearch? Elasticsearch is an expensive tool and it should be used only if we are designing an enterprise level solution or if our application is following microservice based architecture.  But if we follow monolithic architecture, then using elasticsearch for storing centralized logs will be an expensive choice. Because monolithic applications often generate a large amount of log data, especially if it’s logging every request, error, or event we hardly emphasize on mapping logs in different indices. As monolithic applications generate too

Consider Django. Why do we use it? Django serves as a powerful back-end framework that handles client-side requests, processes them, and returns responses. But what happens when a request takes longer to process? Delayed responses can impact the client-side experience, leading to frustration and poor performance for your web application. This is where scheduling tools come into play. A scheduler can help offload time-consuming tasks by handling them periodically or asynchronously, improving overall user experience and reducing the server’s load. While Django doesn’t come with built-in scheduling capabilities, you can rely on external libraries like Celery Beat, AP Scheduler, and Cloud Scheduler to effectively manage and schedule tasks, ensuring smoother operations and better client-side responsiveness. Key Considerations: Complexity of Tasks: For simple tasks, Django Q or APScheduler might suffice. For complex workflows and distributed processing, Celery Beat and Cloud Scheduler is often preferred. Scalability: If you need to distribute tasks across multiple workers, Celery Beat is a strong contender. Integration: Choose a scheduler that integrates well with your existing Django project and infrastructure. Maintainability: Consider the ease of configuration, monitoring, and maintenance of the chosen scheduler. When to use schedulers: Schedulers are an essential tool in computing that allows you to automate repetitive tasks and tasks that we have to perform at specific times. Automate repetitive tasks: Consider the following scenario: we need to create reports at predetermined intervals. To accomplish this, we must utilise schedulers, which are simple to use and manage. Executing tasks at specific times and specific intervals: Schedulers help you save time and money by automating tasks. Schedulers can be used to run jobs automatically rather than by hand. For instance, you can use an interval scheduler such as AP Scheduler to set a 10-minute interval if you need to add data to a table every ten minutes. After that, there will be no need to manually execute the script because the scheduler will automatically input the data every ten minutes. This guarantees that the task is completed precisely and consistently while also freeing up your time. Triggering Tasks Based on Events: Imagine a system that processes uploaded images. When a user uploads a photo, it’s saved to a specific folder. A scheduled task, perhaps using a file system watcher, could be triggered as soon as a new file arrives in that folder. The task would then automatically resize the image, create thumbnails, and store the processed versions, ready for display on the website. When NOT to use schedulers: While Schedulers are powerful tool but they are not the best solution always. Real-time tasks: if a task needs to respond immediately to user, then the scheduler is not a good choice. Schedulers are designed for background or delayed processing. For example, when a user clicks a button, the response should be immediate, not scheduled. One-off tasks: If you only need to run any task only once, and this is a simple operation, manually executing is simpler than a scheduled task. Creating a scheduled task adds overhead, so it’s not worth it for very simple, one-time operations. Over-complex scheduling requirement: if your process depends on failure/success of other tasks then the scheduler is not the best choice. In such cases, a dedicated workflow management system might be more appropriate. Let’s start with AP Scheduler and Cloud Scheduler: AP Scheduler: A package called AP Scheduler, or Advanced Python Scheduler, lets you plan when Python functions will run at particular intervals or times. It has two main scheduler types: Background Scheduler, which runs tasks in the background without interfering with the main program, and Blocking Scheduler, which stops the current thread until scheduled activities are finished. Because it ensures that the web server stays responsive while managing scheduled tasks, Background Scheduler is especially well-suited for Django applications. Using AP Scheduler in Django for Firestore Data Management: We are taking an example in which we are using Django+apscheduler+firestore. In an application where we are dealing with large numbers of incident data, it’s very difficult to ensure that the database remains optimised and clutter-free. If we are storing incident data in a Firestore collection named incident but only need to keep records for the last 30 days, we have to implement a mechanism in which we archive older incidents. A practical solution is to use AP Scheduler and schedule a background job that can automatically move older data to an archived incident collection every day at a specific time. This approach ensures that our Incident collection remains lightweight, improving performance and query efficiency. Project Setup (if not already done): Ensure you have a Django project set up. Configure your DB Setting: Make sure you have Google Firestore set up in your Django project. Add the Firestore credentials in settings.py. Create a .py file in apps (if not already created): Create a new file named scheduler.py in your apps. Install apscheduler: Install the apscheduler library using pip: Write the APScheduler Job: Create a scheduler.py file in your Django apps and here write your scheduler and its job. Here we write a function archive_old_incidents in which we define we have to move incidents older than 30 days to the ‘Archived_Incidents’ collection. Modify apps.py to Start APScheduler: In your Django app, open apps.py and update it: How this task operates: In this solution, incident data older than 30 days is automatically moved to an archive collection in Firestore using the apscheduler in Django. The start_scheduler() function from scheduler.py is called by the ready() method in apps.py when Django starts, initialising a background scheduler. A cron task is then configured by the scheduler to run daily at midnight (00:00). The archive_old_incidents() function runs at the appointed time, retrieving all incidents older than 30 days from the Incidents collection. After that, these events are removed from the original collection and copied to the Archived_Incidents collection. Cloud scheduler: Google Cloud’s Cloud Scheduler managed cron job service, you can schedule tasks by setting up scheduled triggers for HTTP/S endpoints, Pub/Sub topics, or App Engine services.   Using

When we talk about India’s growing consumerism and manufacturing strength, it’s important to note that the manufacturing sector employed over 1.53 crore workers as of 2022. This growth gives us reasons to be optimistic about the country’s economy. Government programs like Make in India and the Production-Linked Incentive (PLI) schemes have played a big role in this success by attracting investments and increasing exports. These initiatives aim to make India a global manufacturing hub, with the industry expected to reach $1 trillion in value by 2025. However, this rapid growth comes with challenges. We need to keep growing while also taking care of the workers’ welfare. The manufacturing sector includes many players, from factory owners and suppliers to the millions of workers who are essential to this industrial boom. These workers often work in high-risk environments, such as chemicals, textiles, and heavy manufacturing, and they face daily challenges that go beyond just their jobs. The Composition of the workforce in India is shown in the below image- The Current Challenges in Indian Manufacturing The manufacturing sector, especially in high-risk areas like heavy machinery, automotive, and pharmaceuticals, faces several challenges: Worker Safety Risks: The industry has a high rate of workplace accidents. Between 2017 and 2021, over 6,500 workers lost their lives in industrial accidents in India. Issues like non-compliance with safety protocols and machinery accidents pose serious threats. Quality Control Issues: Consistent product quality is vital, yet manual inspections often fail to catch defects early, resulting in higher reject rates, potential recalls, and dissatisfied customers. Operational Inefficiencies: Manual processes slow down the production line, increasing operational costs and reducing profitability. Inventory Mishandling: This leads to lost items, inaccurate stock levels, and increased costs due to human errors and inefficiencies. Lean Manufacturing: Lean manufacturing practices streamline production processes, reduce waste, and improve efficiency, making it essential in competitive markets. These challenges have a tangible impact on profit margins, employee morale, and brand reputation. The need for an effective, technology-driven solution that improves both safety and quality standards is pressing. Computer Vision: A Transformative Solution for Manufacturing In today’s manufacturing landscape, advanced technologies play a critical role in addressing key challenges such as worker safety, quality control, and efficiency. Vision analytics emerges as a particularly transformative solution. It enables manufacturers to monitor and analyze production processes in real time, identifying safety hazards, quality issues, and process inefficiencies with high precision. Techniques such as object detection, OCR (Optical Character Recognition), OCV (Optical Character Verification), and instance segmentation allow it to instantly detect anomalies, verify product standards, and ensure compliance with safety standards. The rationale for choosing Computer Vision over other technologies lies in its ability to interpret complex visual data rapidly and adaptively. Unlike traditional sensors, CV can analyze visual details and provide instant feedback, making it an invaluable tool for proactive risk management and consistent quality assurance across production lines. Computer Vision AI vs Other Latest Manufacturing Technologies Capability Computer Vision AI IoT-Based Monitoring Robotic Process Automation (RPA) Augmented Reality (AR) Real-Time Monitoring Yes, monitors machines, workers, and processes in real time. Yes, tracks machine performance via sensors but lacks detailed visual analysis. No, RPA automates predefined tasks, not real-time monitoring. No, AR enhances human interaction, not real-time monitoring. Predictive Maintenance Yes, identifies potential equipment failures through visual and sensor data, preventing costly downtime. Yes, uses sensor data to predict maintenance needs but lacks visual inspection. No, RPA automates processes but doesn’t predict or monitor equipment health. No, AR supports maintenance but doesn’t predict or monitor equipment health. Safety Monitoring Constant monitoring of hazardous areas and unsafe behaviors in real-time, preventing accidents. Basic safety monitoring via sensors but cannot detect unsafe behavior. No, RPA doesn’t focus on safety monitoring or accident prevention. Yes, can be used for safety training but not real-time hazard detection. Cost Efficiency Proactive maintenance, reduced downtime, and optimized resource allocation lead to cost reduction. Costs can be high in terms of installation and ongoing management but provides data for cost optimization. Reduces labor costs by automating repetitive tasks but cannot monitor equipment health or safety. Costly for implementation but can save on training costs and improve worker efficiency in the long run. Integration with Existing Systems Easily integrates with existing manufacturing systems, providing valuable insights into production. Integrates well with IoT platforms but may need additional tools for full functionality. Can be integrated with existing workflows but not with real-time monitoring or visual data analysis. Limited integration capabilities; often used alongside other technologies for operational improvement. How Computer Vision Works in Manufacturing: Addressing Key Challenges 1. Spotting Issues on the Production Line In manufacturing, maintaining product quality is essential. Computer Vision (CV) helps detect and address defects as they happen, ensuring a smooth production flow: Real-Time Defect Detection: CV scans each product on the line, identifying flaws or irregularities instantly, preventing faulty items from advancing in the process. Pattern Recognition for Quality Assurance: By learning from production patterns, CV can detect quality dips early, supporting continuous quality improvements. 2. Checking Inventory Materials Automates quality and quantity checks by using AI-powered image recognition to detect defects, verify counts, and ensure inventory meets required standards in real-time. 3. Ensuring Safety Rules Are Followed In potentially hazardous manufacturing environments, CV helps ensure a safe workplace by monitoring compliance with safety protocols: PPE Compliance Monitoring: CV tracks video feeds to verify that workers are wearing proper safety gear; any violations trigger immediate alerts. Unsafe Action Detection: CV detects risky behaviors or unsafe postures in real time, enabling supervisors to intervene quickly and prevent accidents. 4. Noticing Machine Problems Early Although many safety features are installed on machinery, detecting equipment issues in real-time through Computer Vision (CV) systems can enhance virtual inspections and minimize downtime. By analyzing machine behavior, CV systems can spot early signs of malfunction, enabling timely maintenance and reducing costly disruptions. This is shown below in the image- How this works? 1. Watching and Collecting Data Real-Time Video Monitoring: Cameras continuously capture video footage of workers, machines, and work

In today’s digital landscape, applications frequently begin with a single tenant emphasis. However, when businesses grow, they frequently need to integrate several tenants, each with their own data requirements, security concerns, and regulatory compliance obligations. Supporting numerous tenants necessitates isolating each tenant’s data to maintain privacy and security, especially in industries like finance, healthcare, and e-commerce where data security is crucial. As applications grow in complexity, managing several databases becomes increasingly difficult. The Increasing Challenge of Multi-Tenancy Systems A single database may be sufficient for a small user base, but as the number of tenants increases, so does the need for data segregation, governance, and scalability. Each new tenant increases complexity, generating concerns about data partitioning, resource allocation, and performance. Using a single shared database might result in various hazards and inefficiencies, including: Data Security and Compliance: In order to comply with industry rules or privacy laws, different tenants may need different degrees of data isolation. Without intensive engineering, a single database finds it difficult to enforce such isolation.Data Security and Compliance: In order to comply with industry rules or privacy laws, different tenants may need different degrees of data isolation. Without intensive engineering, a single database finds it difficult to enforce such isolation. Performance bottlenecks: Slow queries, lock contention, and restricted scalability are some of the ways that increased data volume can impair database performance. Tenant-Specific Customization: Applications often need to deliver custom features or configurations for each tenant, complicating management in a single, monolithic database. Why the Issue May Emerge Abruptly A single-tenant platform can soon give way to a multi-tenant one. For instance, a growing SaaS business might have to quickly integrate a number of tenants. Data segmentation is immediately needed since each tenant contributes more data and frequently needs particular adaptations. It becomes difficult to scale for each new tenant without a multi-database strategy, endangering the performance and integrity of data for current renters. The Solution: Master-Slave Database Architecture An effective solution to this issue is provided by a master-slave database architecture. According to this paradigm, control functions are managed by a master database, and each tenant can have their own slave database with separate data storage and dedicated resources. The master-slave configuration allows: Data Isolation: To maintain privacy and compliance, each tenant’s data can be kept apart. Performance Scaling: By offloading tenant-specific data to several databases, bottlenecks are eliminated and performance is enhanced. Easier Maintenance and Customization: Individual tenants can handle maintenance duties and customizations more easily when data is segregated. By introducing new databases for every tenant, this design allows enterprises to grow effectively while maintaining the stability, security, and performance of the core system. Solution: Database routers with the Django Rest Framework (DRF) Custom database routing is made possible by the Django Rest Framework (DRF), which allows for dynamic management of several databases. Although settings.py in Django permits multiple database connections, a clear method is needed to switch between databases according to particular requirements. Using Django’s DATABASE_ROUTERS The Django feature DATABASE_ROUTERS specifies which database to utilize for certain tasks, such as reading, writing, and data migration. Usually, a database router file specifies three main purposes: db_for_read: Identifies the database to be read. db_for_write: Selects the database to be used for writing. allow_migrate: It is used for migrating models in the database. These features provide flexible, dynamic database management according to operation and tenant requirements by enabling the router file to be altered to guarantee that the appropriate database is utilized for every action. Database Switching Scenarios Database switching in a multi-tenant system can be predicated on: Django App: Give every Django application a database. Login as a user: Change databases dynamically according to the information of the logged-in user. 1.  Changing Databases Using the Django App It is possible to map every Django application to a certain database. For example, the router will switch to root-db if a model from the root app is visited if there is an application called root and a database called root-db. This method creates an organized method for managing database routing by directing requests to the relevant database according to the app environment. 2. Database Switching Upon User Login When a user logs in in this case, the default database is chosen. A database_mapper table, which contains fields like parent_database, child_database, and email, keeps track of several databases within this default database, which is frequently referred to as the “root” database. The procedure is as follows: A user’s email address is used to retrieve the parent_database and child_database associated with their login from the database_mapper table. After that, the system changes to the proper database, guaranteeing that the session runs in the right data context. This method makes it possible to manage databases dynamically for each user session according to the location of their data. Thread-Local Storage Implementation for Dynamic Database Switching Thread-local storage can be used to keep track of the right database name across requests. Thread-local storage in Python is helpful for handling numerous databases since it enables data to be saved uniquely for each thread. The database name can be stored in a thread-local storage object that is accessible during each API call without causing cross-request conflicts by using Django middleware. How It Works Save Database Name in Middleware: To guarantee that every request makes use of the appropriate database context, middleware saves the necessary database name in thread-local storage. Database names can be accessed in router functions: The router determines if a certain database is connected to the Django application within db_for_read and db_for_write. Otherwise, the database name is retrieved from thread-local storage by the system. This configuration allows for smooth database switching for every request, allowing many databases to be used in a single API call without running the risk of data leaks between sessions. How These Scenarios Work Together Combining login-based and app-based database switching provides a reliable and adaptable way to manage several databases dynamically within a single API. In order to allow scalable database administration across an expanding

International Tigers Day, celebrated annually on July 29th, is a day dedicated to raising awareness about tiger conservation and promoting efforts to protect this magnificent species. Tigers, symbols of strength, power, and grace, are facing numerous threats that endanger their existence. Habitat loss, poaching, and human-wildlife conflicts are some of the critical issues that need to be addressed to ensure the survival of tigers. On this important day, it is essential to explore innovative solutions that promote coexistence between humans and tigers, ensuring the safety of both. One pioneering initiative making significant strides in this direction is the Human Animal Conflict Mitigation System (HACMS) developed by Valiance Solutions. This advanced technological solution is designed to help forest officers, local communities, and conservationists effectively address the challenges of human-wildlife conflict, creating a safer environment for both humans and tigers. The Challenge of Human-Tiger Conflict Human-tiger conflicts are a growing concern, particularly in regions where human settlements are encroaching on tiger habitats. As natural habitats shrink, tigers are often forced to venture into human-dominated areas in search of prey or territory. This can lead to tragic outcomes for both humans and tigers. Tigers pose significant risks to human safety and livestock when they stray into villages and towns, while humans often retaliate, leading to injury or death for these magnificent creatures. Such conflicts exacerbate the threats faced by tigers and complicate conservation efforts. The Role of HACMS in Mitigating Conflict HACMS is a sophisticated system designed to bridge the gap between technology and wildlife conservation. By leveraging cutting-edge techniques like computer vision, edge computing, and real-time alerts, HACMS helps monitor and manage human-tiger interactions more effectively. 1. Real-Time Alerts: HACMS employs custom AI cameras with internet and edge capabilities to monitor areas prone to tiger movements. These cameras capture and send images to the cloud, where advanced algorithms classify the presence of tigers. Forest officers and local communities receive real-time alerts via the Valiance Eye mobile app, enabling them to take immediate precautionary measures. This proactive approach helps prevent potential conflicts before they escalate. 2. Proactive Measures:By providing timely information about tiger movements, HACMS empowers communities to safeguard their homes, livestock, and loved ones. Alerts are also disseminated through hooters in high-risk areas, raising awareness and ensuring that people remain vigilant. This system not only protects human lives but also minimizes retaliatory actions against tigers, promoting their conservation. 3. Data-Driven Insights: HACMS collects and analyzes data on tiger movements and behavior, offering valuable insights to conservationists and wildlife authorities. This data-driven approach helps in devising effective strategies for habitat management, conflict resolution, and conservation planning. By understanding tiger patterns and preferences, conservationists can create targeted interventions to reduce human-wildlife conflicts. Promoting Coexistence The success of HACMS lies in its ability to foster coexistence between humans and tigers. By minimizing the risk of conflict, the system not only protects human lives but also ensures that tigers can thrive in their natural habitats without fear of retaliation. This harmonious coexistence is essential for maintaining ecological balance and preserving biodiversity. When communities and tigers coexist peacefully, it creates a sustainable environment where both can flourish. A Vision for the Future As we celebrate International Tigers Day, it is imperative to recognize the potential of innovative solutions like HACMS in creating a safer world for both humans and wildlife. The integration of technology in conservation efforts marks a significant step forward in addressing the complex challenges of human-wildlife conflict. Valiance Solutions remains committed to leveraging technology to protect our natural heritage. By promoting awareness, fostering collaboration, and driving innovation, we can ensure that tigers continue to roam freely in their habitats, contributing to the richness of our planet’s biodiversity. On this International Tigers Day, let us reaffirm our commitment to conserving these majestic animals and embracing solutions that promote peaceful coexistence. Together, we can create a future where tigers and humans live in harmony, safeguarding our shared environment for generations to come.

Understanding the Lending Landscape Imagine you’re the captain of a ship navigating through the vast and sometimes stormy seas of the Indian lending market. Your crew consists of various lenders – public sector banks, private banks, housing finance companies (HFCs), and microfinance institutions in housing loans. Each has its way of steering through the waters. For ease of understanding below image is self-explanatory- Shifting Landscape of Loan Originations Home Loans Dominate: Home loans account for 40.7% of retail loans in 2023. Market Share: Public sector banks hold 41.7% of home loans, while private banks have 45.9% of active loans. Larger Home Loans: Loans above ₹75 lakh are growing, showing a trend toward higher ticket sizes. Declining HFCs: Housing Finance Companies (HFCs) are losing share to public and private banks. Personal and Microfinance Loans: Private banks lead in personal loans; NBFCs-MFIs and SFBs dominate microfinance. Analysis: Rise in High-Value Loans: Higher demand for large home loans due to rising property prices. Competitive Home Loan Market: Public and private banks are competing fiercely. HFCs Losing Ground: Traditional banks are taking over the home loan market. Diverse Personal Loan Market: Private banks excel in offering tailored financial products. Microfinance Growth: Essential for financial inclusion and supporting underserved communities. The Indian lending market is evolving, driven by a shift towards higher value loans and increased competition among banks and it also comes with challenges like NPA’s or Non-Performing Assets Enter Early Warning Systems In the dynamic world of banking and financial services, managing credit risk is crucial for maintaining portfolio health and ensuring sustainable growth. With the increasing complexity of financial markets and the growing volume of credit transactions, traditional risk management approaches are no longer sufficient. This is where early warning systems (EWS) come into play, offering a proactive solution to identify potential credit risks before they escalate into significant issues. An effective early warning system leverages advanced analytics, machine learning, and real-time data monitoring to detect early signs of credit deterioration. By providing timely alerts, these systems enable financial institutions to take preventive actions, optimize their risk management strategies, and safeguard their portfolios from potential defaults and losses.In this blog, we will delve into the importance of early warning systems in credit risk management. We will explore how these systems can enhance decision-making processes, improve portfolio performance, and ultimately contribute to a more resilient financial ecosystem. Join us as we uncover the key benefits and best practices for implementing an early warning system, and learn how it can transform your approach to credit risk management. Key Data Sources- Banks gather information from various places to keep an eye on their loans and borrowers. Here are some of the main sources: Loan Management Systems (LMS)/Re-Payment Track Report (RTR): These systems hold all the details about current loans and repayments, helping banks track how each loan is doing. Credit Bureau Scrubs: These are detailed credit reports showing a borrower’s credit history, including their credit score and outstanding debts. Macroeconomic Variables: These are big-picture economic factors like interest rates and unemployment rates, which can impact a borrower’s ability to repay loans. Customer touchpoint- It is a touchpoint is any interaction or communication between a customer and a official bank employee throughout the customer journey Important Variables-From these data sources, banks create some key indicators to help assess risk. Some of them are- Oncycle– Tracks if borrowers are making their payments on time. Normax– Measures the maximum debt levels of a borrower compared to what is considered normal for similar profiles. Score– A numerical value representing the risk of a borrower defaulting on a loan. Default Probability– The likelihood that a borrower will default on their loan obligations within a specific time frame. Delinquency Rate-The percentage of loans in a portfolio that are past due. Repayment History- Historical record of the borrower’s loan repayments. This score helps in categorizing borrowers into different risk buckets and determining the level of monitoring required, Risk Scores and Buckets. Each borrower gets a risk score based on these variables. This score helps banks categorize borrowers into different risk levels, or “buckets”: Low Risk: These borrowers are likely to repay on time, so they need only standard monitoring. Medium Risk: These borrowers show some signs of potential trouble, requiring closer monitoring and possible intervention. High Risk: These borrowers are likely to default on their loans, requiring immediate action to mitigate potential losses. How it looks like-           Cash Flow Forecast Cash flow forecasting is another important part of EWS. It predicts how much money will come in and go out of a borrower’s accounts in the future. This helps banks see if a borrower will have enough cash to make their future loan payments, providing a clearer picture of their financial health. The Role of EWS EWS acts as an advanced radar system, detecting potential fraud early. It identifies “Red Flagged Accounts” (RFAs) where warning signals suggest issues. Banks must monitor these signals, investigate suspicious accounts, and report findings to the RBI. This proactive approach allows banks to act quickly, reducing major loss risks and maintaining trust in the financial system. Benefits of EWS Minimize Default Rate: Identifies high-risk borrowers early to prevent defaults. Assist in Loan Approval: Provides insights for informed lending decisions. Enable Stringent Agreements: Structures loan agreements with safeguards for higher-risk borrowers. Determine Portfolio Composition: Optimizes loan portfolio decisions on sizes and borrower profiles. Facilitate Securitization: Improves risk assessment for efficient securitization of loan assets. As the sector continues to evolve, the integration of sophisticated EWS will likely become a key differentiator between banks that thrive and those that struggle with asset quality. For a robust and resilient banking sector, embracing these technological solutions is not just an option – it’s a necessity. At Valiance Solutions, we are leading the charge with our cutting-edge AI and data analytics solutions. We’ve partnered with credit rating companies to create a credit risk scoring model for micro and small consumer loans using AI. We’ve also developed

Background Forests have been a source of life and a backbone to the economy for us from the ancient ages. Even today, the need and the importance of forests needs no elaboration. While we humans migrated out of the forests, it’s still a home to a huge fraction of flora and fauna, a lot of which need our assistance to sustain in the coming generations. Tigers face the issues of poaching, shrinking habitats and other similar human-wildlife conflicts which has adversely affected its population in history. Even in the reserved and protected areas, we can observe deforestation for economic purposes. Challenge When it comes to detecting land usage or deforestation in a protected area, there is a general lack of solutions which can alert in real time. This results in severe and sustained losses, and sometimes the damage is beyond recovery. This not only affects the wildlife and ecology of the area, but it’s also a loss of culture, history and socio-economic value of the preserved land. To counter this challenge, we’ve come up with a solution to detect deforestation and categorise it based on the land usage(farming or residence) in a specific area, and an alert system for the stakeholders. With the help of this deforestation detection system, actions can be taken in due time and we can move towards a sustainable future and stay true to our goals of nature and wildlife conservation. Solution Approach Tech Stack Shapefile Objects Google Earth Engine Python SQL Flask API Shapefile Objects : The shapefile format is a digital vector storage format for storing geographic location and associated attribute information. It is possible to read and write geographical datasets using the shapefile format with a wide variety of software[1]. The shapefile format stores the geometry as primitive geometric shapes like points, lines, and polygons. These shapes, together with data attributes that are linked to each shape, create the representation of the geographic data. The term “shapefile” is quite common, but the format consists of a collection of files with a common filename prefix, stored in the same directory. The three mandatory files have filename extensions .shp, .shx, and .dbf. The actual shapefile relates specifically to the .shp file, but alone is incomplete for distribution as the other supporting files are required[1]. Mandatory files .shp — shape format; the feature geometry itself {content-type: x-gis/x-shapefile} .shx — shape index format; a positional index of the feature geometry to allow seeking forwards and backwards quickly .dbf — attribute format; columnar attributes for each shape, in dBase IV format Fig 1 : Sample shapefile for an area(it is different from a normal jpeg image as it can store coordinates(latitude and longitude) of every point within the file) Google Earth Engine : Google Earth Engine provides a vast catalogue of map datasets which can be used to observe various features of any point of interest. One such dataset is the Dynamic World V1 which is a 10m near-real-time (NRT) Land Use/Land Cover (LULC) dataset that includes class probabilities and label information for nine classes. These classes refer to different land usage such as water, trees, bare land and so on. Fig 2 : A sample image from Dynamic World V1 data set, different colours indicate different land usage The solution leverages the Dynamic World V1 dataset to get the images of the land piece of interest. Procedure The whole area we want to observe is distributed into smaller rectangular pieces to be analysed individually. For the current implementation, we’re dividing the sample shapefile(Fig 1 in 20 equal pieces). This results in higher quality images from Earth Engine and a more accurate result. In order to detect deforestation, we can compare an image of a place with an image of the same place from a certain time period back(let’s say we want to check the difference for a month). First thing we do is we use the Earth Engine API to get the following 3 images: forest cover for the previous month forest cover for the current month crop cover for the current month The area covered in each box is ~132 km2 (11.88km x 11.16km) and the size of each image obtained from Earth Engine for a box is 1512×1335 pixels. Fig 3 : Forest cover for a piece in Jan 2020 (left) vs Feb 2020 (right) Fig 4 : Crop cover for the same piece in Feb 2020 Processing the Images In order to get the area where forests have been cut and the land is being used for farming, we do the following steps: We first find the area deforested by subtracting the forest cover for the two months Then we intersect that with the crop cover to find out the area where there has been deforestation and crops are planted Fig 5 : For the above scenario, the processed image looks like above Next challenge is to decide which coordinates to point out, as we might only need to identify the locations with a certain area threshold of deforestation for practicality. We use the below criterion to figure out those coordinates: We create a square of n pixels and if all the pixels in the square satisfy our criterion, we give the centre of that square as one of the outputs. Area per pixel for this image is 65.67 sq. metres. So basically this sets a threshold of deforestation to 65.67* sq. metres for being reported. Fig 6 : As seen in the above image, the squares with yellow boundary are the areas to be reported Generating Output Till now we have the pixel values of the image denoting the area of deforestation. To convert them into latitude and longitude coordinates, following steps are followed: To get the latitude and longitude of the centre of these squares, we use unitary method on the pixel location of the centre Compare it with the edge coordinates of the piece to get the coordinates of the centre. Last thing is to make sure that we are