News Aggregator

Docker Use Cases: 15 Most Common Ways to Use Docker

Aggregated on: 2022-11-26 02:43:51

Containerizing applications instead of hosting them on virtual machines is a concept that has been trending in the last few years, making container management popular. Docker sits at the heart of this transition, helping organizations seamlessly adopt containerization technology. Recently, Docker use cases can be seen across all industries, regardless of size and nature. What Is Docker? Docker is a containerization technology that enables developers to package a service into a container along with its dependencies, libraries, and operating system. By separating the apps from the infrastructure, Docker allows you to seamlessly deploy and move apps across a variety of environments.

Testing the Untestable and Other Anti-Patterns

Aggregated on: 2022-11-25 20:28:51

Books on bad programming habits take up a fraction of the shelf space dedicated to best practices. We know what good habits are – or we pay convincing lip service to them – but we lack the discipline to prevent falling into bad habits. Especially when writing test code, it is easy for good intentions to turn into bad habits, which will be the focus of this article. But first, let’s get the definitions right. An anti-pattern isn’t simply the absence of any structured approach, which would amount to no preparation, no plan, no automated tests and just hacking the shortest line from brainwave to source code. This chaotic approach is more like non-pattern programming. Anti-patterns are still patterns, just unproductive ones, according to the official definition. The approach must be structured and repeatable, even when it is counter-productive. Secondly, a more effective, documented, and proven solution to the same problem must be available.

Automate Your Kubernetes Deployments With Helm

Aggregated on: 2022-11-25 16:13:51

Why We Need Automated Deployments Over the last decade, there has been a paradigm shift in the way applications are written, deployed, and managed. Businesses have adopted cloud-native as their strategy for dealing with applications. As a result, applications have shifted to a microservices architecture. The deployment platforms are now managed by clouds or Kubernetes. When applications are written in a microservices way, a single application is broken into many small applications. Each one of these small applications is fully independent. They might have their own DB, cache server, messaging queues, and any such enterprise infrastructure.

12 Essential Skills Your ReactJS Developers Should Have

Aggregated on: 2022-11-25 16:13:51

While more and more users are moving towards an omnichannel approach, it is important to have a seamless user experience. One way to achieve this is by having a fully responsive application to ensure users have the best experience across platforms. There are numerous JavaScript libraries available for creating mobile/web app front-ends, but React is the best when it comes to creating a responsive application. In addition to supporting website front-ends, this well-equipped framework allows developers to create truly responsive web applications. Furthermore, React provides reusable custom HTML code for apps.

14 Web Development Trends You Should Look For in 2023

Aggregated on: 2022-11-25 14:28:51

It was the year 1991 when the first web page was published, transforming our lives forever. Today, we spend hours on the Internet, earning money, spending money, listening to music, finding our soul mates, obtaining university degrees, watching movies, and investigating conspiracy theories, among other activities. The list is virtually never-ending. From simple web pages to complex websites, web development has evolved to a great extent, allowing users to become immersed in certain topics and experience things.

Kubernetes Autoscaling: How to Use the Kubernetes Autoscaler

Aggregated on: 2022-11-25 14:28:51

Imagine a situation where when user load increases or decreases on an application, there is no way for the application to scale out or in its resources as required. Such an application probably wouldn’t survive in today’s market thanks to Kubernetes Autoscaling, which can easily overcome such a situation with its different scaling mechanisms. Kubernetes Autoscaling mechanisms help scale in and out pods and nodes as required. There are three different methods supported by Kubernetes Autoscaling.

Utilize These Detection-as-Code Best Practices

Aggregated on: 2022-11-24 15:43:50

Today, 94% of organizations are using cloud technology, and this swift evolution to the cloud means security teams are handling more data and more alerts than ever. Additionally, threats and attacks are only increasing in frequency — it’s estimated that a cyber attack occurs every 11 seconds — and sophistication. But more often than not, security teams are overwhelmed because they don’t have the right tools and approaches to handle modern threat detection at scale. Security team leaders should have updated tools and approaches to help them protect their organization, and the best approach they can take is to adopt detection-as-code. Here's more about detection-as-code and its benefits, as well as some best practices to help you gain success as you begin to use detection-as-code in your security approach.

A Comprehensive List of Agile Methodologies and How They Work

Aggregated on: 2022-11-24 15:13:50

Agile workflows have fast become a cornerstone of software development environments globally, so much so that the methods that underpin the framework have spread into many other disciplines. Whether it’s helping a marketing team revamp their core strategy or giving customer relations a boost toward their latest service goal, Agile is here to stay.

What Is Oracle Cloud ERP?

Aggregated on: 2022-11-24 14:13:50

Foremost, the Oracle Cloud ERP, abbreviated as Oracle Cloud Enterprise Resource Planning includes all sorts of functions in an enterprise or organization, i.e, from account management, financial management, and project management to procurement. Likewise, Oracle entered the IT industry with its maximum used cloud-based technology based on the end-to-end Saas (Software as a service) platform suite that runs on Oracle cloud centers. The tool has the ability to grow the business in an efficient way with its utmost delivery structure at lower costs. Oracle Cloud ERP has a start of the art data security that makes the data flow much easier to share and secure. With this cloud tool, the components which get through the end users help the company or an organization to streamline the analytical flow, user experience, reporting and automation tools, compliance and Oracle Fusion SCM, and more.

Why SASE Is the Future of Networking and How to Actualize It

Aggregated on: 2022-11-24 14:13:50

Secure Access Service Edge (SASE) represents the dawn of a new era for enterprise security. Since its introduction by Gartner analysts Neil MacDonald and Joe Skorupa, SASE has been hailed as the future of network connectivity. Data centers were the go-to solution for WAN connectivity and security in the past. However, with the proliferation of cloud-based solutions combined with an untethered work environment, the traditional network security we were familiar with is no longer a viable option. SASE addresses these issues by providing an approach to combine network connectivity and network security functions. On the endpoint side, this aids in delivering a cloud-native solution to WAN security that eliminates the requirement for security to be run through a centralized data center. SASE has been gaining traction since its establishment in 2019; nevertheless, we saw an accelerated increase in the SASE market in the aftermath of the worldwide pandemic. On-premises solutions and networks with perimeters started to make no sense when the whole workforce was scattered worldwide. SASE solves this by bringing security to the user via the cloud, making it one of the most promising examples of edge computing. As a direct result of the pandemic, the worldwide SASE market, which was already gaining steam, experienced exponentially accelerated growth almost overnight. According to the Dell'Oro group, the SASE industry will rise to $5 billion per year by 2024.

Top 8 Major Advantages of SaaS Application Development

Aggregated on: 2022-11-24 13:28:50

In a business environment, SaaS is a compelling alternative to installing software on-premises (the traditional method), which requires the creation of the server, installation of the application, and configuration. As a result, the applications are hosted on a remote cloud network that is accessible via a web browser or API, just as if they were rented out. This software can be used by you and your organization for some time, and you will be responsible for paying for it. Learn more about the advantages of Software as a Service (SaaS). Technology and business models are both evolving quickly, so it’s important to consider long-term perspectives and effects while choosing the best software for a given situation. The key goal is to have maximum product adaptability so that business owners may stay current and remain competitive.

Learning Angular as a React Developer

Aggregated on: 2022-11-24 03:43:50

As a front-end developer, you always get asked whether you specialize in React or Angular. These two mainstream technologies have their strengths and weaknesses, and each is more appropriate depending on the project, but in this article, we will not be discussing that. I am currently working at Apiumhub, and recently I started working in Angular. This is a quick cheat sheet for getting started in Angular as a React developer. Creating a Component in Angular Let’s imagine we want to create a CustomButton component. When we create this component in React, it can look similar to this:

A Closer and Detailed Look at SOAP API

Aggregated on: 2022-11-23 21:13:50

An API, or Application Programming Interface, is software that allows two different applications to send messages to each other. For example, say you need to log into an app using your Google account because you want a check-in email to arrive in your main inbox. Here, a Web API is used. It’s an agreement between these web services about how they are going to exchange data.

5 Principles of Database Hardening for Security

Aggregated on: 2022-11-23 20:28:50

Configuring your database systems for security is also known as database hardening. Doing this can significantly limit the vulnerability of your sensitive data. Designing for security means understanding the best practices for database configuration and applying them to your database environments efficiently. In this article, specific database configuration settings are unique to each database system based on the network architecture, hardware, software, and applications used. We will examine some general principles of data security that can be implemented across various scenarios. 1. Secure the Physical Space Designing a secure database environment begins with the physical space. If you have any on-premise servers, make sure they are in a locked and monitored environment to limit unnecessary access. The location should have security protocols for physical emergencies such as fire or natural disasters. Ensure your data is safe by implementing and maintaining database backups according to your technology’s best practices.

A Poor Man’s API

Aggregated on: 2022-11-23 20:28:50

Creating a full-fledged API requires resources, both time and money. You need to think about the model, the design, the REST principles, etc., without writing a single line of code. Most of the time, you don't know whether it's worth it: you'd like to offer a Minimum Viable Product and iterate from there. I want to show how you can achieve it without writing a single line of code. The Solution The main requirement of the solution is to use the PostgreSQL database. It's a well-established Open Source SQL database.

Can Artificial Intelligence Provide Value in IoT Applications?

Aggregated on: 2022-11-23 20:28:50

If you are involved in the field of IoT technology, then it is essential to understand the importance and benefits of AI. In this section, I will discuss all the aspects related to AI so that you can get a clear picture of this topic. Today, IoT applications are in visual recognition, predicting a future event, and identifying an object.

How to Create an Ansible Playbook

Aggregated on: 2022-11-23 18:28:50

In this post, you will learn how to create an Ansible playbook. As an exercise, you will install an Apache Webserver onto two target machines and change the welcome page. 1. Introduction In the two previous Ansible posts, you learned how to setup an Ansible test environment and how to create an Ansible inventory. This post continues this series, but it is not necessary to read the first two posts. In this post, you will learn how to create an Ansible playbook. A playbook consists out of one or more plays which execute tasks. The tasks call Ansible modules. Do not worry if you do not understand this yet, this is what you will learn. It is also advised to read the introduction to playbooks in the Ansible documentation.

The Evolution of DevOps: Where It Began and What's Ahead

Aggregated on: 2022-11-23 17:13:50

The goal of DevOps is to increase the speed, quality, and reliability of software delivery. It is a set of practices that combines software development (Dev) and IT operations (Ops). DevOps is often used in the context of web ops, application development, and security.

IMAP OAuth 2.0 Authorization in Exchange Online

Aggregated on: 2022-11-23 15:58:50

Microsoft announced that starting October 2022, Basic authentication for specific protocols in Exchange Online would be considered deprecated and turned off gradually and randomly for certain tenants. As insightful details concerning this topic may be found in Resources items one and two, among these protocols, there are Exchange ActiveSync (EAS), POP, IMAP, Remote PowerShell, Exchange Web Services (EWS), Offline Address Book (OAB). Consequently, customer applications leveraging Basic authentication towards Exchange Online as part of their business use cases need to replace it with Modern authentication — OAuth 2.0 token-based authorization — which no doubt has many benefits and improvements that help mitigate the former's risks.

When AI Gets Creative: Is This Cause for Concern?

Aggregated on: 2022-11-23 15:58:50

For decades, robots have worked with humans. Within the automobile business, as an example, they've long been the foremost precise and reliable welders and painters. Sitting in one place and doing the same thing over and over again has traditionally been automation's sweet spot. But, with the explosion in computing, robots can now comprehend how to accomplish much more complicated, nuanced tasks. This includes work that needs to be done inside, like at a factory, or outside, like in the field. In agriculture, they are not only tilling fields, but they can also detect weeds and zap them with lasers. In hospitals, robots do everything from helping nurses get the needed supplies to assisting surgeons in directing their instruments more precisely. A 2020 World Economic Forum report predicted that artificial intelligence and automation would displace 85 million jobs globally within five years. However, it also made a prediction that advancements in technology would create would ninety-seven million new jobs—generally ones requiring a lot of skills and education. Today we'll take a glance at the kind of labor AI is ready to try and accomplish and what we can expect from AI in the future.

Make Your Go Code Work 1.5x Faster or Even More

Aggregated on: 2022-11-23 15:28:50

Performance is a key thing in everything; we (humans) don't like to wait and waste our time. Therefore, sometimes quick solutions, as most of the managers suppose, are better than slow ones but with good engineering and design. But today, we are not speaking about management but about code performance. We have a small text formatting library that allows us to format text using a template in a convenient on our view way. These not only formatting functions do what fmt.Sprintf does this in a more convenient way but also provides additional features. Previous versions of our module were loose in performance to fmt.Sprintf, but since 1.0.1, we are better. And today, we are going to tell you how to make any golang code work faster. Parameters and Return Values There are two options to pass either argument and/or get function result - by pointer and by value; consider the following example:

Let’s Build an End-to-End NFT Project Using Truffle Suite

Aggregated on: 2022-11-23 13:13:50

Non-fungible tokens (NFTs) are one of the biggest (and most popular) innovations to come out of Web3 so far. Although their capabilities are still being explored, both the art industry and the gaming world have benefited from the technology. This is because NFTs create an indisputable record of digital ownership. Building NFTs isn’t as complicated as you think, especially with tool suites such as Truffle Suite This article will introduce you to the Truffle Suite of tools and teach you how to use them to build an end-to-end NFT project and include the following:

Hangfire Introduction and Implementation in .NET Core 6 Web API

Aggregated on: 2022-11-23 04:43:50

Prerequisites Basic understanding of .NET Core 6 API and C#. Visual Studio 2022 SQL Server Agenda Create .NET Core API using version 6 Configure Hangfire in .NET Look into the different types of jobs that are present in Hangfire. Overview Hangfire is open-source and used to schedule the job at a particular event and time. It is also used to create, process, and manage your background jobs. We use this in background processing without user intervention. Hangfire is reliable and persistent. It will take care of all things once the job is scheduled. Hangfire dashboard is also available for us to manage all things easily. Why Hangfire is Required in Background Processing Sometimes we need to do lengthy operations like database updates and database maintenance so it’s managed periodically. Batch import from XML, JSON, and YAML files. Recurring reports on a periodic basis. Mass notification on subscription and sign up basis. So, these are things that we are able to do periodically over a certain period of time as per our requirements. There are different types of jobs that are present in Hangfire. We will look at them one by one.

Brute Force Attacks: How to Guarantee Data Protection in Companies

Aggregated on: 2022-11-23 04:13:49

Brute force attacks are an attempt by a user to gain access to an account or system by constantly entering credentials, either manually or automatically. The objective of the breach is to discover passwords to enter accounts or find hidden links and get hold of confidential data.

5 Kubernetes Lens Alternatives

Aggregated on: 2022-11-23 04:13:49

What Is Kubernetes Lens? Lens is an open-source integrated development environment (IDE) that allows users to connect and manage multiple Kubernetes clusters on Mac, Windows, and Linux platforms. It provides a feature-rich, intuitive graphical interface that allows users to deploy and manage clusters directly from the console. Kubernetes Lens provides built-in dashboards that provide key metrics and insights into resources running on a Kubernetes cluster, including deployment, configuration, networking, storage, access control, and custom resources.

Tutorial: Build DynamoDB-Compatible Apps for Any Cloud (Or On-Prem)

Aggregated on: 2022-11-23 02:58:49

Alternator is an open-source project that allows teams to build "run-anywhere" DynamoDB-compatible applications with ScyllaDB. This means you can deploy wherever you want: on-premises or on any public cloud. In this tutorial, we’ll start by introducing the project. Afterward, we’ll see a hands-on example of creating a one-node ScyllaDB cluster and performing some basic operations on it.

Graph Databases: How Do They Work?

Aggregated on: 2022-11-23 02:43:49

To understand how graph databases work, let’s define it first. A graph database is a type of NoSQL databases which stores nodes and relationships instead of tables or documents. The data is stored without restricting it to a pre-defined model giving flexibility. Abovementioned NoSQL databases are often perceived as “no SQL” at all but it’s actually “not only SQL.” In SQL databases, the data is only stored in tables. Using NoSQL, you can store four types of databases: document, key-value, wide-column, and graph. SQL database examples: MySQL, PostgreSQL, OrientDB, MariaDB, CouchDB, and more. NoSQL database examples: MongoDB, Apache Cassandra, Neo4j, Redis, DynamoDB, HyperTable, and more. In fact, there are so many databases now that it’s difficult to choose. However, there are lists of best or top databases that have descriptions of functionality and use cases.

Why I Don't Do TDD

Aggregated on: 2022-11-23 02:43:49

I recently gave a talk about debugging for the London Java Community. During the Q&A part of the talk, someone asked me about my approach to Test Driven Development. In the past, I looked at that practice in a more positive light. Writing lots of tests — how can that be bad? But as time moved on, I now see it in a different light. I see it as a very limited tool that has very specific use cases. It doesn’t fit into the type of projects I build and often hinders the fluid processes it’s supposed to promote. But let’s backtrack for a second. I really liked this post that separates the types and problems in TDD. But let’s simplify it a bit, let’s clarify that every PR should have good coverage. This isn’t TDD. It’s just good programming.

What Is Distributed Tracing?

Aggregated on: 2022-11-22 23:58:49

Cloud-native development has revolutionized application development in ways that are both positive and challenging. The adoption of microservices architectures on container-based infrastructures enables faster software development lifecycles. At the same time, problems can strike when changes are made to apps, such as adding new features. Moreover, app updates can happen multiple times a day. So how do teams track down problems when error messages pop up, or when it suddenly takes longer to load an application? Unlike the monolithic approach to application development, where a straightforward application call makes it easy to find where a problem exists, cloud-native applications and the container-based infrastructure they run on are ephemeral. This means problems are elusive. The need for distributed tracing, which tells you exactly where a problem is happening, becomes acutely important for teams needing to quickly fix their applications.

Automatically Deploy Apps to VPS With Git Triggers and Coolify

Aggregated on: 2022-11-22 20:28:49

I make a lot of silly projects, and one thing I’ve wanted for a long time is a way to automate the deployment process using Git. So in this blog post, we are going to be tackling just that. First things first, let’s settle on what the target is. What I want is a system with the following:

Why It’s Time to Shift Left Technical Debt

Aggregated on: 2022-11-22 19:58:49

Shifting left has brought massive benefits for testing, security and DevOps, but not for tech debt and issue tracking. Yet, technical debt is one of the main reasons development teams struggle or fail to deliver. Let’s face it – developers don’t want to work through an endless backlog of tech debt. It just sucks. Tech debt hampers productivity, which impacts morale. There’s nothing worse than slowly grinding to a halt as a team.

How Are These OTT (Over-the-Top) Service Challenges Solved by Artificial Intelligence (AI)?

Aggregated on: 2022-11-22 19:58:49

OTT (Over-the-top) content platforms have become a part of entertainment today. Due to the pandemic and lockdown, OTT platforms have become popular among consumers seeking home entertainment. Most of the major media companies have become a part of the OTT platform.

Big O Notation - Why? When? Where?

Aggregated on: 2022-11-22 19:13:49

What Is Big O? This is a mathematical term that originated in the early 20th century in number theory and came almost immediately to computer science as questions arose with resource optimization. Wiki-defined Big O notation is a mathematical notation that describes the limiting behaviour of a function when the argument tends towards a particular value or infinity. Let's rephrase it, and make this definition a little simpler and closer to software development. Any task is solved according to one or another approach, one or another algorithm of action. To compare the effectiveness of heterogeneous solutions written in different programming languages using different approaches, you can start analyzing its execution through the notation. Next, let's look at the most common classes of frequently encountered time complexities. O(1) - Сonstant

Salesforce Functions for Caching Expensive Queries

Aggregated on: 2022-11-22 18:28:49

Caching is a strategy that can help you conserve resources and improve performance. When you have an oft-run expensive query with a result that seldom changes, caching is an ideal solution. By caching the result of that query, you can return the cached result when necessary. The result is the same, but you save the need to run the expensive query. Everybody wins. In this article, we will walk through the use of Salesforce Functions to cache expensive queries. For instance, we want to query for some value across a large number of records, and the page requiring this query is often loaded. However, the result will not change from one query execution to the next.

Genjector: Reflection-free Run-Time Dependency Injection framework for Go 1.18+

Aggregated on: 2022-11-22 17:58:49

I haven’t been writing much lately. It’s not that I wouldn’t like to do it, but it’s hard to get time off from a regular job. And being in a position to write some new code is even less realistic. It’s no surprise that I felt like Alice in Wonderland when I hit the keyboard last weekend. I’ve been thinking for a while now about providing a dependency injection framework for Go that wouldn’t be based on reflection. Everything I tried before March of this year was a dead end. Why March? Because in March, Google released a new version of Go, which allowed us to use Generics. So, this summer, I used my time at the beach, as any good software engineer should, to think about new possibilities for dependency injection.

Engineering Productivity: What Not to Do

Aggregated on: 2022-11-22 17:28:49

Motivated by some recent M&A news and general productivity pressures in time of tight budgets, we present some anti-patterns to the use of engineering metrics and give an overview of how to use metrics for productivity insights instead. First, let us first start with what to avoid: Anti-Pattern to Engineering Metrics Lines of code written: While lines of code can be a proxy metric for how much code you have to maintain (and therefore costs to incur), it has been proven to be a terrible metric for productivity. This shouldn’t need much explanation, but: Output does not necessarily equate to the outcome. Different programming languages have different verbosity, and e.g., just including some open-source packages or pasting in some code from Stack Overflow does not make you more productive. And, of course, solving hard problems often does not require a lot of code but a lot of thinking, exploration, and collaboration.

How to Filter NSFW Images and Programmatically Blur Them

Aggregated on: 2022-11-22 13:43:49

If you are building a social app where users can upload images, then content moderation should be the most important task for your developers. If a user uploads any disturbing image on your platform, it may hurt others' sentiments, or in extreme cases, it can become a legal issue. Therefore your best bet should be to remove or blur the NSFW content that violates your content policy. In this tutorial, we are going to look at how to detect NSFW images using machine learning algorithms and then automatically blur the image.

Learning From Failure With Blameless Postmortem Culture

Aggregated on: 2022-11-22 13:43:49

This is an article from DZone's 2022 Performance and Site Reliability Trend Report.For more: Read the Report Site reliability engineering aims to keep servers and services running with zero downtime. However, outages and incidents are inevitable, especially when dealing with a complex system that constantly gets new updates. Every company has a relatively similar process to manage incidents, mitigate risks, and analyze root causes. This can be considered an opportunity to identify issues and prevent them from happening, but not every company is successful at making it a constructive process.

WordPress Core — Unauthenticated Blind SSRF

Aggregated on: 2022-11-22 13:43:49

WordPress is the world’s most popular content management system, used by over 40% of all websites. This wide adoption makes it a top target for threat actors and security researchers that get paid for reporting security issues through their public bug bounty program. Vulnerability brokers are also very interested in acquiring unpatched vulnerabilities enabling them to take over WordPress instances, sometimes offering up to $300,000 for critical ones. As such, WordPress has a heavily reviewed code base in which researchers are not expected to find low-hanging fruits anymore. My previous research on this target required extensive expertise and effort to uncover security issues.

What Is the Difference Between Data Compliance and Data Privacy?

Aggregated on: 2022-11-22 03:43:49

When deploying any application that holds customer or user data, both data compliance and data privacy are important areas to consider. Yet these two areas of data management are sometimes misunderstood. This article will shed some light on the differences between data compliance and data privacy. What Is Data Compliance? Data compliance refers to the requirement to meet certain legal obligations around the collecting, processing, and storing of data.

Getting Started With Plugin Development for JetBrains IDEs Like IntelliJ and GoLand

Aggregated on: 2022-11-22 03:13:49

No matter what your technical background as a developer looks like, you’ve probably come across or even used an IDE from JetBrains. Their product line features IDEs for various languages and technologies and they’re all built according to the same schema: Atop an extensible core (“IntelliJ platform”) sits a set of curated extensions that provides all of the “integrated” functionality of the IDE. But what if you could extend your IDE further with your own plugins? Doing your own plugin for your workflows can improve your productivity drastically. In this post we’ll explore the IntelliJ API and build a simple plugin step-by-step to get you going.

Migration from Amazon SQS and Kinesis to Apache Kafka and Flink

Aggregated on: 2022-11-22 02:43:49

Even digital natives — that started their business in the cloud without legacy applications in their own data centers — need to modernize their cloud-native enterprise architecture to improve business processes, reduce costs, and provide real-time information to their downstream applications. This blog post explores the benefits of an open and flexible data streaming platform compared to a proprietary message queue and data ingestion cloud services. A concrete example shows how DoorDash replaced cloud-native AWS SQS and Kinesis with Apache Kafka and Flink. Message Queue and ETL vs. Data Streaming With Apache Kafka A message queue like IBM MQ, RabbitMQ, or Amazon SQS enables sending and receiving of messages. This works great for point-to-point communication. However, additional tools like Apache NiFi, Amazon Kinesis Data Firehose, or other ETL tools are required for data integration and data processing.

Request Tracing in Spring Cloud Stream Data Pipelines With Kafka Binder

Aggregated on: 2022-11-21 23:43:49

What Is Data Pipeline? A data pipeline is a process to transfer data from a single source/multiple sources to a destination via batch or stream process. In this process, raw data is filtered, cleaned, transformed, enriched, and feed to data lakes or data warehouses. In an enterprise, data is scattered to multiple systems in different formats; in those cases, data pipelines help to collect data from all the sources in the same format so that it’s appropriate for business intelligence and analytics. What Is Request Tracing? In a distributed system, a request travels multiple services before completion. Services can be hosted in different network zones, different VMs, different cloud providers, or any combination of these. Triaging an issue in this environment is tedious and time-consuming to eliminate; request tracing is helpful. A unique id is minted at the origin of the request, and it will be carried forward to all the systems request travel; with this approach, using a unique id, we can trace the journey of the request.

Jakarta EE and MicroProfile at EclipseCon Community Day 2022

Aggregated on: 2022-11-21 23:13:49

Community Day at EclipseCon 2022 was held in person on Monday, October 24 in Ludwigsburg, Germany. Community Day has always been a great event for Eclipse working groups and project teams, including Jakarta EE/MicroProfile. This year was no exception. A number of great sessions were delivered from prominent folks in the community. The following are the details including session materials. The agenda can still be found here. All the materials can be found here. Jakarta EE Community State of the Union The first session of the day was a Jakarta EE community state of the union delivered by Tanja Obradovic, Ivar Grimstad, and Shabnam Mayel. The session included a quick overview of Jakarta EE releases, how to get involved in the work of producing the specifications, a recap of the important Jakarta EE 10 release, and as well as a view of what’s to come in Jakarta EE 11. The slides are embedded below and linked here.

Shift Left: Value Creation in Scrum

Aggregated on: 2022-11-21 22:43:49

As a tactical framework, Scrum is good at delivering Increments into customers’ hands. As we work in iterations, we probably do that several times per month, mitigating risk by closing feedback loops. Nevertheless, there is a potentially hazardous void in the framework that successful Scrum teams start plugging early: how to figure out what is worth building—product discovery—in the first place. As a result, value creation in Scrum is not as straightforward as you might have thought. The Context of Value Creation According to the Scrum Guide Let’s start with a look at the manual: “The Product Goal describes a future state of the product which can serve as a target for the Scrum Team to plan against. The Product Goal is in the Product Backlog. The rest of the Product Backlog emerges to define “what” will fulfill the Product Goal.” (Source.)

Data Transformation Tips for the Trenches

Aggregated on: 2022-11-21 22:28:49

Here are some quick and dirty data transformation techniques you might find useful. I'm not much into exposition, so that we can get started. Depending on the application, data transformation has different purposes. It can prevent over-fitting for regression problems. In classification, it improves model accuracy significantly.

Scaling Continuous Deployment Adoption

Aggregated on: 2022-11-21 22:28:49

Cloud Platforms and cloud-native development have become very prevalent. However, the benefits of cloud-native platforms are realized only when DevOps automation is implemented along with it. The two key processes in DevOps are Continuous Integration (CI) and Continuous Deployment (CD). Most organizations have successfully automated CI over the last decade. Continuous deployment helps in delivering software faster with high quality and low risk and improves productivity and efficiency. However, the automation for CD is a lot less, as per many surveys, due to underlying complexity. Deep knowledge of cloud-native platforms, architectures, operations, release management, verification, and observability is required for the successful adoption of CD. Multitudes of tools across disciplines, such as CI, CD, security, testing, and observability, have to be orchestrated to make such automation happen. This paper explores the key aspects organizations should consider while adopting continuous deployment and offers guidance to improve the adoption maturity.

How CI Observability Can Strengthen Your DevSecOps Pipeline?

Aggregated on: 2022-11-21 21:28:49

Recently, we see a special emphasis has been laid on software supply chain security referring to the recent report by Google – 2022 Accelerate State of DevOps Report. With security holding center stage, we see a good emergence of practices such as “SRE” and “DevSecOps.” Looking at the State of DevOps report from different companies such as Google, CircleCI, Puppet, and Dynatrace, we have got some common findings, such as:

Auto-Scaling a Spring Boot Native App With Nomad

Aggregated on: 2022-11-21 21:28:49

Horizontal scaling and in particular auto-scaling is a common challenge for many applications no matter their architectural approach (monolithic or microservices) and the deployment environment (cloud or on-premises). Today's Cloud providers provide a rich variety of options and tools to accommodate this critical requirement: to be able to scale services instances up or down, based on specified rules, in an efficient manner in order to: Avoid unavailability Ensure the quality of service Optimize resources utilization Avoid unnecessary charges Many of the out-of-the-box solutions rely on Kubernetes and offer various levels of abstraction to make this process as smooth as possible. However, there are many cases where we don't really need or cannot afford all these cloud offerings as-a-service or we need to operate our own data centre on-premises. As presented in a previous article "Look, Ma! No Pods!", HashiCorp Nomad is a simple and flexible deployment tool, able to manage both containers and non-containerized applications running on-prem, cloud or hybrid environments. Lately, Nomad provides auto-scaling functionality via the Nomad Autoscaler Agent. As described in the Nomad Autoscaler Overview, the following strategies are supported:

When To Use the @DynamicUpdate With Spring Data JPA

Aggregated on: 2022-11-21 21:28:49

The Spring Data JpaRepository interface comes with the save(S entity) method that is frequently used to update an entity representation on the database level. By default, the method generates an UPDATE statement that requests a change to the values of all the entity class columns. This includes the columns not touched by your application code. However, this behavior can be far from satisfactory for several reasons. Luckily, Hibernate supports the @DynamicUpdate annotation that, once applied to an entity class, helps the save(S entity) and similar methods generate a SQL update command with columns that are actually changed.