News Aggregator

T-Shape Engineer, An Introduction

Aggregated on: 2023-04-25 23:00:21

Hello, and welcome to the T-Shape engineer journey! It will be a long, and I hope, interesting ride because we’re going to talk about a lot of things. This won’t be just dry theory about programming languages, databases, etc. We’re gonna create a full-stack web application from scratch. From the very beginning, step by step, we’ll build that application and will try to become a T-Shape engineer.

SmartStoreNET: Malicious Message Leading To E-Commerce Takeover

Aggregated on: 2023-04-25 22:30:21

SmartStoreNET is the leading open-source e-commerce platform for .NET, which makes it suitable for companies running Windows Server. Next to the operation of an online business, it offers advanced features, such as CRM tools, a blog, and a forum. As a result, a SmartStoreNET instance handles highly sensitive data such as credit card, financial, and personally identifiable information that have to be protected from attackers. During recent security research, my team and I discovered two vulnerabilities that could allow attackers to gain control of the server where SmartStoreNET is installed by sending one malicious message to the instance's administrator. In this article, I present the root cause analysis of two Cross-Site Scripting bugs and then describe how they could be exploited by attackers. Finally, I will describe the patches applied by the maintainers and the limitations of those patches.

Feature Interaction Metrics

Aggregated on: 2023-04-25 21:45:21

In software development, feature interaction occurs when the behavior of a software system is affected by the combination of features or inputs. Feature interaction can cause unexpected and undesirable behavior in software systems, which can be challenging to detect and resolve. Feature interaction matrices are one of the techniques used to identify and analyze feature interaction in software systems. These metrics can help identify potential problems caused by feature interactions and guide software developers in designing better systems. What Is a Feature Interaction Matrix? A feature interaction matrix is a table that lists all possible combinations of features in a software system and identifies any interactions between them. Each row and column in the matrix represents a feature, and the cells represent the interactions between the features. The matrix is populated by testing the software system with different combinations of features to identify any interactions that may cause unexpected behavior.

Azure Data Box

Aggregated on: 2023-04-25 21:45:21

Azure Data Box is a product offered by Microsoft Azure that helps organizations transfer large amounts of data securely and efficiently to and from Azure. It is similar to AWS Snowball Edge or/ Google Transfer Appliance. The purpose of the Azure Data Box is to provide a way for organizations to move large amounts of data to Azure without relying on traditional data transfer methods that can be slow, costly, and unreliable. Data Box devices can be ordered using the Azure portal. Using the local web UI, one can set up this as per the requirement. Data Box is a device designed to securely transport up to 100TB of data to and from Azure. It is ideal for organizations that need to transfer large amounts of data but have limited network bandwidth.

Connection Pool High Availability With CockroachDB and PgCat

Aggregated on: 2023-04-25 21:15:21

Today, I'd like to cover pooler high availability as this is one aspect of connection pooling I've never focused on. Due in large part because my previous attempts were based on PgBouncer, and it standing up PgBouncer is not easy. Luckily, it's been an absolute pleasure working with PgCat as it has a lightweight installation process. High-Level Steps Start CockroachDB, PgCat, and HAProxy in Docker Run a workload Demonstrate fault tolerance Conclusion Step-By-Step Instructions Start CockroachDB, PgCat, and HAProxy in Docker I have a Docker Compose environment with all of the necessary services here. Primarily, we have to set up two instances of PgCat.

Spring Boot Debugging With Aspect-Oriented Programming (AOP)

Aggregated on: 2023-04-25 20:45:21

Aspect-Oriented Programming (AOP) is a programming paradigm that aims to increase modularity by allowing the separation of cross-cutting concerns. In simple terms, it helps you keep your code clean, modular, and easy to maintain. We can leverage AOP to debug Spring Boot applications more effectively and seamlessly. In this post, we will explore how to use AOP to debug a Spring Boot application effectively. AOP Concepts Before diving into debugging with AOP, it is essential to understand the core concepts. AOP lets us write code that executes before or after a method. It includes the following common terms:

Frequently Faced Challenges in Implementing Spark Code in Data Engineering Pipelines

Aggregated on: 2023-04-25 19:30:21

Pyspark has become one of the most popular tools for data processing and data engineering applications. It is a fast and efficient tool that can handle large volumes of data and provide scalable data processing capabilities. However, Pyspark applications also come with their own set of challenges that data engineers face on a day-to-day basis. In this article, we will discuss some of the common challenges faced by data engineers in Pyspark applications and the possible solutions to overcome these challenges. 1. Serialization Issues in Pyspark Serialization issues in PySpark are a common problem that can lead to slow and inefficient processing. This article will discuss what serialization is, why it is important, and how to identify and resolve serialization issues in PySpark.

GPX Routes From Apps to a Watch for Phone-Free Navigation

Aggregated on: 2023-04-25 19:15:21

Smart wearables are incredibly useful when it comes to outdoor workouts, especially in situations where you don't want to be carrying your mobile phone. A smartwatch that tracks your real-time exercise records and routes monitors your health status, and even supports maps and real-time navigation is practically a must-have tool for outdoor sports enthusiasts nowadays. However, not every smartwatch supports independent, real-time navigation on the watch. Fortunately, for watches without such a feature, it is still possible to use offline maps to navigate. Fitness apps can take advantage of offline maps to provide users with navigation features on smartwatches. The problem is, how can the offline maps generated on a fitness app be synced to a smartwatch?

Building a RESTful API With Java Spring Boot

Aggregated on: 2023-04-25 18:45:21

Spring Boot is a popular framework for creating powerful RESTful APIs, and in this tutorial, we will use it to develop a simple API that simulates a credit score rating. The API endpoint we will create will allow a user to retrieve a credit score rating by sending a request to the API. However, it is important to note that we will not be linking up to any actual backend systems to pull a real credit score. Instead, we will use a random number generator to generate the score and return it to the user. Although this API is relatively simple, it will demonstrate the basics of REST API development using Java and Spring Boot. This tutorial will provide a hands-on introduction to building RESTful APIs with Java Spring Boot. Prerequisites Before we start, we must ensure that a couple of prerequisites are completed. To follow along and run this tutorial, you will need the following:

KubeCon EU: Summary of Observability Day Europe

Aggregated on: 2023-04-25 17:30:21

The KubeCon + CloudNativeCon event kicked off with a slew of off-site events. I dropped in on Observability Day Europe and wanted to share a few things I found interesting. This event was set up to foster collaboration, discussion, and knowledge sharing of cloud-native observability projects (including but not necessarily limited to Prometheus, Fluentd, Fluent Bit, OpenTelemetry, and OpenMetrics), as well as vendor-neutral best practices for addressing observability challenges. It was intended both for audiences that are new to observability as well as for seasoned practitioners. Observability Day enabled you to spend a day peeking under the hood of major Cloud Native Computing Foundation observability-related projects and broadening your knowledge of observability.

HLS Streaming With AVKit and Swift UI for iOS and tvOS

Aggregated on: 2023-04-25 17:30:21

HLS (HTTP Live Streaming) is a popular video streaming protocol that is widely used for delivering video content over the internet. In this response, we'll discuss how to implement HLS streaming with AVKit and SwiftUI for iOS and tvOS. What Is HLS? HLS, which stands for HTTP Live Streaming, is an internet protocol that enables the delivery of video content. Developed by Apple Inc., it was first introduced in 2009 as part of their QuickTime X and iPhone software. Today, HLS is extensively used for streaming live and on-demand video to a broad range of devices and platforms, including smartphones, tablets, smart TVs, and desktop computers. These systems can operate seamlessly with existing caching infrastructure, including Content Delivery Networks (CDNs), and can effectively navigate standard firewalls and routers.

Build a Web3 Ticketing System and Disrupt Online Ticketing

Aggregated on: 2023-04-25 17:15:21

A popular and practical use case for web3 is generating tickets to live events. Blockchains such as Ethereum can guarantee the ownership, originator, and authenticity of a digital item, effectively solving the problem of counterfeit tickets. While major players such as Ticketmaster struggle to mitigate scalpers (trying desperately to control who can resell tickets, where, and for how much) and ticket fraud—web3 already has a solution. The ticketing industry is ripe for disruption. In this tutorial, we’ll look at how to create such a ticketing solution using ConsenSys Truffle, Infura, and the Infura API. We’ll deploy a smart contract that acts as a ticketing service and creates tickets as ERC-20/ERC-721 digital tokens. We’ll also walk through a few architectures of potential frontends that could interface with the contract, and together function as an integrated, full-stack, web3 ticketing system.

A Critical Detail About Kafka Partitioners

Aggregated on: 2023-04-25 15:30:21

Apache Kafka is the de facto standard for event streaming today. Part of what makes Kafka so successful is its ability to handle tremendous volumes of data, with a throughput of millions of records per second, not unheard of in production environments. One part of Kafka's design that makes this possible is partitioning. Kafka uses partitions to spread the load of data across brokers in a cluster, and it's also the unit of parallelism; more partitions mean higher throughput. Since Kafka works with key-value pairs, getting records with the same key on the same partition is essential.

Potential Benefits of Using Generative AI in the Development and Operations of IT Systems

Aggregated on: 2023-04-25 15:15:21

Generative AI can automatically generate code or models used in IT systems. This can help to speed up the development process and reduce the amount of manual labor required. Generative AI can also create new designs or solutions for IT systems that human developers may not have considered. By analyzing large amounts of data and identifying patterns, generative AI can develop novel solutions to complex problems.

How To Use Cypress Intercept for Handling Network Requests

Aggregated on: 2023-04-25 14:30:21

With cy.intercept(), you can intercept HTTP requests and responses in your tests, and perform actions like modifying the response, delaying the response, or returning a custom response. When a request is intercepted by cy.intercept() the request is prevented from being sent to the server, and instead, Cypress will respond with the mock data you provide. This allows you to test different scenarios and responses from a server without actually having to make requests to it.

How To Install CMAK, Apache Kafka, Java 18, and Java 19 [Video Tutorials]

Aggregated on: 2023-04-25 12:45:21

This article explains how to install the following in video tutorials: CMAK (Cluster manager for Apache Kafka) or Kafka manager Apache Kafka on Windows or Windows 10/11 Java 18 on Windows 10/11, JDK installation Java 19 on Windows 10/11, JDK installation Java JDK 19 on Amazon EC2 Instance or Linux operating system Apache Kafka on Amazon EC2 Instance or Linux operating system How to Install and Use CMAK (Cluster Manager for Apache Kafka) or Kafka Manager Learn how to install and use CMAK. This is a tool for managing Apache Kafka clusters that allow us to view all the topics, partitions, numbers of offsets, and which are assigned to what and all topics, etc.

AWS ECS vs. AWS Lambda: Top 5 Main Differences

Aggregated on: 2023-04-24 23:15:21

Cloud computing has revolutionized the software industry in the last 10 years. Today, most organizations prefer to host applications and services on the cloud due to ease of deployment, high security, scalability, and cheap maintenance costs over on-premise infrastructure. In 2006, Amazon launched its cloud services platform, Amazon Web Services (AWS), one of the leading cloud providers to date. Currently, AWS offers over 200 cloud services, including cloud hosting, storage, machine learning, and container management.

Automated App Testing Using Appium With TestNG

Aggregated on: 2023-04-24 23:15:21

In recent times, many web applications have been ported to mobile platforms, and mobile applications are also created to support businesses. However, Android and iOS are the major platforms because many people use smartphones compared to desktops for accessing web applications. As per Statcounter, Mobile is the clear leader with 61.16% market share, while Desktop has 38.84% market share.

Streaming Forex With Python SocketIO

Aggregated on: 2023-04-24 23:15:21

This tutorial takes you through a Python SocketIO implementation. Understanding the difference between SocketIO and WebSocket is essential. However, that is not the scope of this article. Please go through this tutorial to explore more about it. To realize the reason behind the increasing popularity of SocketIO, I would like to request you work along with this tutorial. We will use the client code and SocketIO server responding with TraderMade’s Forex data. I am clarifying these aspects in the beginning as it is easy to work on one side of the example before diving deeper into the topic. At the same time, you will experience a practically used, real-life example.

How To Place Text and Shapes on an Image in Java

Aggregated on: 2023-04-24 22:45:21

Coding new visual elements into an image file begins with a basic understanding of how image files are normally displayed. When an image file is loaded for display on any of our devices, software from that device must first decode the file and store the result of that decoding in a temporary block of memory called a buffer. Buffers become responsible for communicating the color information stored by each individual pixel within an image (in each instance, the file is opened), and from there, that color information can be rendered by our device’s lighting (usually LCD or LED) display. When we want to layer new text or shapes which display on top of an image, we need to access the file in memory and create our own temporary image buffer to work within. This new buffer will give us control over an entirely new layer of pixels in our image, allowing us to temporarily influence — and eventually save changes to — our image’s final display. How our image subsequently implements our new buffer depends on the original file format. If we begin with a PNG file, for example, we’ll leverage built-in transparency features to layer new content on top of the original file. If we begin with a JPG file, which doesn’t offer transparency features, we’ll end up with a single, blended image layer that effectively mixed the new and original content together. In either case, the result of the operation will retain the encoding of the original file, allowing us to easily write our resulting (slightly larger sized) file encoding to a new file entirely.

Integration Architecture Guiding Principles, A Reference

Aggregated on: 2023-04-24 22:15:21

The Integration Architecture guiding principles are guidelines to increase the consistency and quality of technology decision-making for the integration solutions. They describe the big picture of the enterprise within the context of its technology intent and impact on the organization. In the context of Integration Architecture, the guiding principles drive the definition of the target state of the integration landscape. Each principle will contain a description, rationale, and implications. 1. Centralized Governance Statement Any integration-related initiative should conform to the standards and recommendations of a centralized Integration Centre of Excellence (CoE); any design decision or strategy related to integrations needs to be driven, reviewed, and approved by the Integration CoE.

Self-Management

Aggregated on: 2023-04-24 22:15:21

Is self-management an essential building block on an organization’s path to business agility or a nice-to-have cultural twist to, for example, keep teams happy and attract new talent? While many people, particularly at the management level, are skeptical about the concept, I am convinced that organizations need to descale and regroup around aligned, autonomous, self-managing teams in a complex environment. Ultimately, only the people closest to the customers’ problems can solve those within the given constraints while contributing to an organization’s sustainability.

Get an In-Depth Look at the Best Ways To Store Application Parameters in AWS

Aggregated on: 2023-04-24 21:30:21

Many applications are now being hosted on public cloud platforms, and it becomes imperative to leverage the cloud to store their data and application parameters. And of the most popular cloud providers, Amazon Web Services (AWS) is the most widely used. While AWS offers many solutions for storing application parameters, understanding which option best fits your application and use case can be difficult. In this article, we’ll dive into the best ways to store your application parameters in AWS. Overview of Application Properties Storage Let us take the example of AWS Lambda — a popular compute service that allows developers to run code without provisioning or managing servers. When writing code for Lambda functions, it's common to use configuration properties to define how the function should operate. Configuration properties can include things like environment variables, database connection strings, and other settings that are specific to your application.

Getting Started With Gradle on MacOS: A Step-By-Step Guide Using Homebrew

Aggregated on: 2023-04-24 21:00:21

What Is Gradle? Gradle is a powerful build automation tool that is widely used in the Java and Android development communities. It allows developers to automate the process of building, testing, and deploying applications, making it an essential tool in modern software development workflows. What Is Homebrew? Homebrew is a package manager that allows you to easily install and manage software packages on macOS.

Best Practices and Hacks: Data Loading in Snowflake From Stages

Aggregated on: 2023-04-24 20:30:21

Continuing our Snowflake blog series, after learning about setting up a Snowflake account using System-defined roles, we will explore the best practices for loading data from a file into Snowflake. Let’s begin!

Java Synchronized Block

Aggregated on: 2023-04-24 20:30:21

In my earlier post, we learned about the Java synchronized method. In this post, let’s learn about the synchronized block. Video: To see the visual walk-through of this post, click below:

Remote Testing Mobile Devices and Embedded Platforms Using Squish

Aggregated on: 2023-04-24 19:45:21

Automation testing tools rely on object identification based on object models or object properties such as name, type, and position. We have many automation tools for web and desktop applications. While testing mobile devices, hardware, and electronic equipment poses unique challenges, which can be sorted out with remote system access. Where the local and remote system is not geographically co-located, interactive script recording and debugging can become cumbersome and time-consuming. Existing remote desktop solutions bring problems of their own: interoperability of the controlled and controlling platform, compatibility of the test framework and remote control application, setup and configuration issues, and others. Accessing specific embedded or mobile devices may not be possible. Squish is a cross-platform testing tool that supports many desktop, mobile, and embedded platforms. In addition, Squish allows automation of the testing of graphical user interfaces (GUI) for applications written in various programming languages, including Java, .NET, and Qt.

API Gateway Pattern: Features and the AWS Implementation

Aggregated on: 2023-04-24 19:15:21

API Gateway is an architectural pattern to create a façade that exposes the internal system’s data to external clients in a decoupled layer, reducing the dependency between frontend and backend components. Nowadays, that pattern is widely spread in the computing field being used in business applications and integration solutions for small, medium, and large enterprises. The value of breaking the dependency between clients and backend implementations is very substantial. Many paramount functions of modern distributed systems can be implemented in the API gateway. The list of functions is quite large and will be elucidated in the next sections of this article. Currently, we see API gateways usage in critical mission applications and use cases such as microservices architecture, legacy monoliths, monolith migration to microservices projects, and integration layers. Additionally, the API gateways are involved in different deployment models, since the bare-metal servers running on-premises, virtual machines like Amazon EC2, and containerized solutions such as Kubernetes on Amazon EKS, even in the modern serverless computing platform, covering all possible deployment spectrum available today. The issues solved using the API Gateway pattern are explored in section 2.

10 Best Practices for Using Kubernetes Network Policies

Aggregated on: 2023-04-24 16:45:21

As more applications are deployed in Kubernetes clusters, ensuring that traffic flows securely and efficiently between them becomes increasingly important. Kubernetes Network Policies are a powerful tool for controlling traffic flow at the IP address or port level, but implementing them effectively requires following best practices. In this article, we will explore ten best practices for using Kubernetes Network Policies to enhance the security and reliability of your applications. 1. Use Namespaces and Labels for Granular Policy Enforcement YAML apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: backend-policy namespace: backend spec: podSelector: matchLabels: app: backend ingress: - from: - podSelector: matchLabels: app: frontend

Make E2e Testing Easier With the Right Tools

Aggregated on: 2023-04-24 15:30:21

End-to-end (e2e further in the text) testing is immensely important since there are issues that can only be discovered while testing the system as a whole, testing it from one end to another. That also adds a greater level of complexity to e2e tests, making them longer, slower to execute, and with more points of potential failure compared to lower-level tests with a smaller scope, especially when compared to tiny and blazing-fast unit tests. Since e2e tests bring a lot of value, we don’t want to neglect them. We also want to make our lives easier when doing e2e testing, which we can do with the right tools. Apart from automating our e2e tests, we can speed up our e2e tests if when they are not automated. Let us dig deeper into what exactly we can do to make e2e testing easier and faster to execute.

A Complete Guide to Cordova Testing

Aggregated on: 2023-04-24 14:00:20

When we think about the idea of an application, one central question constantly trembles: What would be the best platform to develop it for initially? The reason for this question is the highly fragmented market today we live in. Currently, a user has two primary options to operate an application. For example, a ticket booking application can either be opened as a web application on a desktop or a mobile device can be considered for the same process. Users can also download the mobile application if they choose a mobile device. After this, these options divide further. Let’s say they prefer a mobile device to download the application. But then, which operating system are they using? Since Android and iOS have APIs and structure that is poles apart, they need separate applications.

Generics in Java and Their Implementation

Aggregated on: 2023-04-24 13:30:20

Generics in Java In Java programming, language generics are introduced in J2SE 5 for dealing with type-safe objects. It detects bugs at the compile time by which code is made stable. Any object type is allowed to be stored in the collection before the generic introduction. Now after the generic introduction in the Java programming language, programmers are forced to store particular object types. Advantages of Java Generics Three main advantages of using generics are given below:

Conditionals In JavaScript

Aggregated on: 2023-04-24 12:30:20

Conditional statements are used for making decisions based on different conditions. Also, conditionals in JavaScript allow you to execute different blocks of code based on whether a certain condition is true or false. Conditions can be implemented using the following ways:

The Art of Using Checklists In Jira: Best Practices and Examples

Aggregated on: 2023-04-23 22:00:20

You probably have your favorite recipe you like to treat yourself to every once in a while. Would it still be your go-to dish if you were to miss a couple of ingredients while cooking? A good checklist in Jira can ensure that you and your team are following the recipe to a tee every time.

IBM App Connect Enterprise Unit and Component Tests

Aggregated on: 2023-04-23 22:00:20

Integration flows often interact with multiple external services such as databases, MQ queue managers, CICS regions, etc., and testing the flows has historically required all of the services to be available when running tests. This provides a high degree of confidence that the flows behave correctly for the tested scenarios, but the number of scenarios that can be tested this way is often too small to provide sufficient confidence that the overall solution will behave correctly in all (or even most) circumstances. Unit testing with mocked services is a common solution to this problem in the application development world, but integration solutions may require an in-between style of testing due to the large number of service interactions and the common re-use patterns seen in the integration world. App Connect Enterprise development started calling these “component tests” some time ago: Unit tests that test slightly larger sections of code and (unlike pure unit tests) are allowed to communicate with external services. This article will attempt to illustrate this approach using a database as an example service.

Zero Trust Network for Microservices With Istio

Aggregated on: 2023-04-23 21:30:20

Security was mostly perimeter-based while building monolithic applications. It means securing the network perimeter and access control using firewalls. With the advent of microservices architecture, static and network-based perimeters are no longer effective. Nowadays, applications are deployed and managed by container orchestration systems like Kubernetes, which are spread across the cloud. Zero trust network (ZTN) is a different approach to secure data across cloud-based networks. In this article, we will explore how Istio, with ZTN philosophy, can help secure microservices.

What Is Argo CD? The GitOps Tool for Kubernetes

Aggregated on: 2023-04-23 17:45:20

Argo CD is a continuous delivery (CD) tool that has gained popularity in DevOps for performing application delivery onto Kubernetes. It relies on a deployment method known as GitOps. GitOps is a mechanism that can pull the latest code and application configuration from the last known Git commit version and deploy it directly into Kubernetes resources. The wide adoption and popularity are because Argo CD helps developers manage infrastructure and applications lifecycle in one platform. To understand Argo CD in-depth, we must first understand how continuous delivery works for most application teams and how it differs from GitOps.

API Gateway For ChatGPT Plugins

Aggregated on: 2023-04-23 16:45:20

OpenAI has recently launched a new version of ChatGPT which now allows plugins inside ChatGPT. These plugins can be added directly to the chatbot, providing it with access to a wide range of knowledge and information from its third-party partners through the APIs. ChatGPT plugins can extend its functionality and enhance its capabilities to access up-to-date information such as research travel costs, find out discount information, or help you book flights and order food. You can also build your own plugin that allows ChatGPT to call your API data intelligently. Yes, that’s right! To make your data accessible through a ChatGPT custom plugin, ChatGTP requires you to build a new API or use an existing one that can be used to query it and receive its responses. Then it generates a user-friendly answer by combining the API data and its natural language capabilities. In this case, API Gateway can help with improving security, usability, and efficiency. This post explores how API Gateway can be beneficial for ChatGPT plugin developers to expose, secure, manage, and monitor their API endpoints.

Custom Training of Large Language Models (LLMs): A Detailed Guide With Code Samples

Aggregated on: 2023-04-22 19:15:19

In recent years, large language models (LLMs) like GPT-4 have gained significant attention due to their incredible capabilities in natural language understanding and generation. However, to tailor an LLM to specific tasks or domains, custom training is necessary. This article offers a detailed, step-by-step guide on custom training LLMs, complete with code samples and examples. Prerequisites Before diving in, ensure you have:

How to Handle Secrets in Jenkins

Aggregated on: 2023-04-22 18:30:19

Jenkins is a popular open-source automation server that is widely used for building, testing, and deploying software. It allows developers to automate many aspects of their software development process, including continuous integration and continuous deployment. As with any continuous integration software, managing secrets in Jenkins is crucial to ensure the security and integrity of the applications being built and deployed. In this article, we will be discussing the best practices for managing secrets in Jenkins, including configuring and managing secrets and how to handle potential security breaches.

How Midjourney and Other Diffusion Models Create Images from Random Noise

Aggregated on: 2023-04-22 18:15:19

Colossal hype ensues whenever there's progress or even the slightest shift in the world of machine learning (ML). When it comes to AI, the craze always ends up growing out of proportion and insane claims invariably follow. You've probably noticed this recently as more people are beginning to use Midjourney, ChatGPT, Copilot, and so on. The truth about these systems, however, is always more prosaic. Machine learning is mathematical, and the real implications of the ML models are not nearly as profound as some bloggers would have you believe. They could be beneficial and, in some cases, transform large portions of workflows in specific areas, but only if the user, whether an organization or an individual, has a sufficient understanding of their inner workings, limitations, capabilities, and potential.

MLOps for Beginners: Getting Started With MLOps

Aggregated on: 2023-04-22 17:30:19

Machine learning (ML) has revolutionized various industries by enabling data-driven decision-making along with the automation of certain tasks. For instance, many banking institutions deploy advanced machine-learning models to detect fraudulent transactions. These models need to evolve constantly otherwise there will be a steep rise in false positives. However, deploying new machine learning models in production can be challenging. Training the model on production data, deploying it, and maintaining it isn’t easy. Many a time, machine learning models in production fail to adapt to the changing data and environment. And doing all of it manually isn’t efficient.

Producing and Consuming Avro Messages With Redpanda Schema Registry

Aggregated on: 2023-04-21 21:45:19

If you’re familiar with Apache Kafka®, then you might have encountered a Kafka-compatible schema registry—a separate component that you deploy outside of your Kafka cluster, since Kafka itself doesn’t have one built-in. Essentially, a schema is a logical description of how your data is organized, and so a schema registry provides a central repository for those schemas, allowing producers and consumers to seamlessly send and receive data between them. For event-driven architectures, this can become complex and difficult to manage as you scale, since data schemas can change and evolve over time (potentially breaking things down the line).

Lambda Architecture: A Powerful Approach to Data Engineering

Aggregated on: 2023-04-21 20:15:19

In the world of big data and analytics, processing and managing vast amounts of data efficiently and effectively is a critical challenge. Data engineers play a pivotal role in designing and implementing solutions to handle this data deluge. One such approach that has gained popularity in recent years is the Lambda Architecture, a powerful framework for building scalable and robust data processing pipelines. In this article, we will explore the Lambda Architecture in detail, understanding its key concepts, benefits, and challenges. What Is Lambda Architecture? The Lambda Architecture is a data processing architecture that combines batch processing with real-time/stream processing to handle large volumes of data in a distributed and fault-tolerant manner. It was introduced by Nathan Marz in his book "Big Data: Principles and best practices of scalable real-time data systems" and has since become a widely adopted approach in the field of data engineering.

What Is Platform Engineering? How To Get Started

Aggregated on: 2023-04-21 20:15:19

Platform engineering is the discipline of building and maintaining a self-service platform for developers. The platform provides a set of cloud-native tools and services to help developers deliver applications quickly and efficiently. The goal of platform engineering is to improve developer experience (DX) by standardizing and automating most of the tasks in the software delivery lifecycle (SDLC). Instead of context switching like provisioning infrastructure, managing security, and learning curve, developers can focus on coding and delivering the business logic using automated platforms. Platform engineering has an inward-looking perspective as it focuses on optimizing developers in the organization for better productivity. Organizations benefit greatly from developers working at the optimum level because it leads to faster release cycles. The platform makes it happen by providing everything developers need to get their code into production so they do not have to wait for other IT teams for infrastructure and tooling. The self-service platform that makes developers' day-to-day activities more effortless and autonomous is called an internal developer platform (IDP).

Why Use Kong API Gateway

Aggregated on: 2023-04-21 19:45:19

Kong API Gateway is a cloud-native API gateway that is based on the Nginx reverse proxy. It is a simple, fast, and lightweight solution that enables you to control, set up, and direct application-to-dedicated server request routing. The Kong API Gateway helps regulate who can access the services and data that are managed behind it. It ensures data security by allowing only authorized users and apps to access the data. The Kong API Gateway is highly performant and offers the following features: Request/Response Transformation: Kong can transform incoming and outgoing API requests and responses to conform to specific formats. Authentication and Authorization: Kong supports various authentication methods, including API key, OAuth 2.0, and JWT, and can enforce authorization policies for APIs. Traffic Management: Kong provides traffic management features, such as rate limiting, request throttling, and IP whitelisting, to maintain the reliability and stability of APIs. Monitoring and Logging: Kong offers detailed metrics and logs to help monitor API performance and identify issues. Plugins: Kong has a vast and continuously growing ecosystem of plugins that provide additional functionality, such as security, transformations, and integrations with other tools. Microservice Architecture: Kong is designed to work with microservice architecture, providing a central point of control for API traffic and security. Scalability: Kong is designed to scale horizontally, allowing it to handle large amounts of API traffic. Advantages of Using Kong as an API Gateway Kong API Gateway is an efficient solution for managing APIs that offer advanced routing and management capabilities. With flexible request routing, automatic service discovery, advanced load balancing, comprehensive API management, and real-time analytics and monitoring, organizations can effectively route API traffic, discover and register APIs, distribute traffic across backend services, manage APIs throughout their lifecycle, and gain insight into API performance and usage. These capabilities make Kong a highly effective solution for managing APIs at scale and are essential for organizations looking to build and maintain a robust API infrastructure.

Okta + SAML + JBoss EAP 6.4.x + Picketlink

Aggregated on: 2023-04-21 19:15:19

In this article, I explain how to provide Okta authentication to your web application hosted on a JBoss EAP 6.4.x server. In this example, we provide authentication to internal users (who are logged in in the customer network) and external users (from the internet) who may be third-party collaborators such as carriers, partners, etc.

What Is Blue-Green Software Delivery Deployment?

Aggregated on: 2023-04-21 17:30:19

A blue-green deployment model is a software delivery release strategy based on maintaining two separate application environments. The existing production environment running the current production release of the software is called the blue environment, whereas the new version of the software is deployed to the green environment. As part of testing and validation of the new version of the software, application traffic is gradually re-routed to the green environment. If no issues are found, then the green environment becomes the new blue environment. The former blue environment can be taken down, and a new green environment can be established for the next release. Why Is Blue-Green Deployment Useful? The primary benefits of implementing a blue-green strategy are 1) minimal or zero application downtime and 2) no negative impact on end-users when switching users to a new software release or when rolling back a release in the event of unforeseen issues with the new release or deployment.

How to Stream Sensor Data to Apache Pinot for Real Time Analysis

Aggregated on: 2023-04-21 15:30:19

This is part 2 of a multi-part sample application that I've been building using Kafka and Apache Pinot. In the first part, we built an NFC Badge Reader to record when badges are swiped. In this part, we will begin ingesting some environmental data from a sensor into the mix. I'm not going to spoil the fun by telling you where this is ultimately headed, but feel free to reach out to me on Twitter and see if you can guess! As always, all of the code is available on GitHub.

Test Tool Tutorial: A Comprehensive Guide With Examples and Best Practices

Aggregated on: 2023-04-21 14:30:19

Test tools are software or hardware designed to test a system or application. Various test tools are available for different types of testing, including unit testing, integration testing, and more. Some test tools are intended for developers during the development process, while others are designed for quality assurance teams or end users. In addition to automating testing tasks, test tools can produce test data, monitor system performance, and report on test results.