News Aggregator

Test Your Smart Contract Code with ChatGPT and Diligence Fuzzing

Aggregated on: 2023-06-03 23:30:45

ChatGPT is exceptional at finding bugs without requiring any meaningful additional context beyond the source code. This makes it an excellent first place to start your testing journey. One of the first things you learn as a smart contract coder is that contracts are extremely unforgiving when it comes to vulnerabilities. Contracts are immutable by default. At the same time, they’re capable of handling extraordinary amounts of money. This makes security (and smart contract testing) arguably the biggest concern for any team of web3 developers.

Microservices With Apache Camel and Quarkus (Part 2)

Aggregated on: 2023-06-03 14:30:44

In the first part of this series, we saw a simplified microservices-based money transfer application, implemented using Apache Camel and AWS SDK (Software Development Kit) as Java development tools and Quarkus as a runtime platform. As noted, there are many deployment scenarios that might be considered in order to run the production of such an application; the first and simplest one is running it locally in a standalone manner. It's the scenario that we'll be considering during this new post. Quarkus is able to run your applications in two modes: JVM (Java Virtual Machine)-based and native. The JVM-based mode is the standard classical way of running Java applications. Here, the running application isn't executed directly against the operating system, but in kind of an execution medium where Java libraries and APIs are embedded and wrapped around. These libraries and APIs might be very large, and they occupy a specific part of the memory named Resident Set Size (RSS). In order to know more about the RSS and Quarkus (as opposed to how Spring Boot handles it), see here.

Essential Architecture Framework: In the World of Overengineering, Being Essential Is the Answer

Aggregated on: 2023-06-02 23:00:44

Before telling you how we found and executed an essential architecture framework for successful software, we need to remember the context we live here and, possibly, you will identify with it. Following the history of software development, we have never had so many options for frameworks, programming languages, and project templates, and at the same time, we have never failed so much in meeting customer expectations.

How To Use Geo-Partitioning to Comply With Data Regulations and Deliver Low Latency Globally

Aggregated on: 2023-06-02 19:30:44

In today's interconnected world, application users can span multiple countries and continents. Maintaining low latency across distant geographies while dealing with data regulatory requirements can be a challenge. The geo-partitioning feature of distributed SQL databases can help solve that challenge by pinning user data to the required locations. So, let’s explore how you can deploy a geo-partitioned database cluster that complies with data regulations and delivers low latency across multiple regions using YugabyteDB Managed.

Health Check Response Format for HTTP APIs

Aggregated on: 2023-06-02 18:45:44

I'm continuing my journey of getting more familiar with HTTP APIs by reading related RFCs. This time, I read the Health Check Response Format for HTTP APIs at the suggestion of Stefano Fago. In this post, I'd like to summarize my reading. Note that it's a draft. Moreover, it has been dormant for nearly two years and, thus, has been automatically expired. However, it's the closest to a specification on health checks and thus deserves some love.

Tech Hiring: Trends, Predictions, and Strategies for Success

Aggregated on: 2023-06-02 16:45:44

The tech industry has seen a significant change in the skills, qualifications, and titles listed in job postings over the past few years. What does that mean for companies — and for the candidates themselves? On this week’s episode of Dev Interrupted, we talk to Maryam Jahanshahi, co-founder and Head of R&D at Datapeople, who breaks down the biggest hiring trends in tech, from title inflation to salary transparency and the skyrocketing costs of recruitment.

Constructing Real-Time Analytics: Fundamental Components and Architectural Framework — Part 2

Aggregated on: 2023-06-02 16:45:44

In Part 1, I discussed the growing demand for real-time analytics in today's fast-paced world, where instant results and immediate insights are crucial. It compared real-time analytics with traditional analytics, highlighting the freshness of data and the speed of deriving insights as key features. The article emphasized the need for selecting the appropriate data architecture for real-time analytics and raised considerations such as events per second, latency, dataset size, query performance, query complexity, data stream uptime, joining multiple event streams, and integrating real-time and historical data. And I teased the following Part 2 of the article, which delves into designing an appropriate architectural solution for real-time analytics. Building Blocks To effectively leverage real-time analytics, a powerful database is only part of the equation. The process begins with the capacity to connect, transport, and manage real-time data. This introduces our first foundational component: event streaming.

Step Into Serverless Computing

Aggregated on: 2023-06-02 16:45:44

Serverless computing has revolutionized the process of building and deploying modern applications. It offers a demand-driven approach to backend services, eliminating the need for users to manage supporting infrastructure. With auto-scaling capabilities, serverless services allow businesses to pay for the exact resources they utilize, leading to cost optimization. Although servers are still utilized, developers are relieved of the responsibility of server management and monitoring. Historical Context In the early days of the Internet, creating a web application required purchasing expensive and bulky hardware to run a server. The introduction of cloud computing allowed users to remotely rent fixed quantities of servers or server space. However, this often led to over-provisioning to avoid exceeding monthly limitations due to sudden increases in traffic. Cloud providers introduced auto-scaling models to address this issue, but unexpected spikes in activity like DDoS attacks remained costly.

Structured Logging

Aggregated on: 2023-06-02 16:15:44

When I was learning about writing serverless functions with AWS Lambda and Java, I came across the concept of structured logging. This made me curious about the concept of Structured Logs, so I decided to explore it further. What Is Structured Logging? Typically, any logs generated by an application would be plain text that is formatted in some way.

A React Frontend With Go/Gin/Gorm Backend in One Project

Aggregated on: 2023-06-02 15:30:44

The ReactAndGo project is used to compare a single page application frontend based on React and a Rest backend based on Go to Angular frontends and Spring Boot/Java backends. The goal of the project is to send out notifications to car drivers if the gas price falls below their target price. The gas prices are imported from a provider via MQTT messaging and stored in the database. For development, two test messages are provided that are sent to an Apache Artemis server to be processed in the project. The Apache Artemis server can be run as a Docker image, and the commands to download and run the image can be found in the 'docker-artemis.sh' file. As a database, Postgresql is used, and it can be run as a Docker image too. The commands can be found in the 'docker-postgres.sh' file.

Risk Based Testing: Comprehensive Guide With Best Practices

Aggregated on: 2023-06-02 15:00:44

Risk based testing (RBT) is a type of software testing that focuses on identifying and prioritizing high-risk areas of the software applications being tested. In simple terms, risk based testing is an approach that evaluates the features of software applications at high risk of failure based on software complexity. Even though there are other software testing types like white box testing, grey box testing, and system testing that focus on testing every feature of software applications, why do we need risk based testing?

Azure Virtual Machines

Aggregated on: 2023-06-02 14:15:44

Azure Virtual Machines (VMs) are a cornerstone of Microsoft Azure's Infrastructure as a Service (IaaS) offering. In this article, we will explore the capabilities of Azure VMs and delve into their key features and benefits. We will also discuss practical use cases and highlight the advantages of leveraging Azure VMs for your cloud computing needs. Understanding Azure Virtual Machines Azure Virtual Machines provide on-demand, scalable computing resources in the cloud. They are virtualized instances of computer systems that enable users to run applications and services without the need to manage the underlying hardware infrastructure. Azure VMs offer a wide range of options for operating systems, configurations, and sizes, allowing users to customize their virtual machine environment according to their specific requirements.

Grow Your Skills With Low-Code Automation Tools

Aggregated on: 2023-06-02 13:45:44

Some of the tools we can use to automate tests are low-code tools. One of the tools I use to automate browser testing is the low-code tool, Ghost Inspector. You can develop reliable automated tests and learn a great deal about test automation by using Ghost Inspector. What Is Ghost Inspector? Ghost Inspector can be used to automate tests without writing any code. You can build a test out of steps that do not require coding. You can identify elements in the web page that you are testing using an identifier such as an XPath or a CSS selector and then choose a test step that interacts with the element in the way that you require, for example, by clicking or by assigning a value. This provides a simple way to create automated tests. Creating tests without writing code can be valuable because it enables a tester who can not write code to start creating tests. However, if a large number of tests are created in this way, they can become can be time-consuming to maintain and hard to read. They can take a lot of time to maintain because if an element on a page that you are testing changes, you will have to find every use of that element and update it. Tests created in this way can also be hard to read as when you read the steps, see a step that clicks on a div whose class starts with a certain string and then a step that waits for an element that is a span whose class contains a certain string, and you are left trying to remember what do these steps do.

Automating Data Quality Check in Data Pipelines

Aggregated on: 2023-06-02 13:45:44

In a recent survey by Great Expectations, 91% of respondents revealed that data quality issues had some level of impact on their organization. It highlights the critical importance of data quality in data engineering pipelines. Organizations can avoid costly mistakes, make better decisions, and ultimately drive better business outcomes by ensuring that data is accurate, consistent, and reliable. However, 41% of respondents in the survey also reported that lack of tooling was a major contributing factor to data quality issues. Employing data quality management tools in data pipelines can automate various processes required to ensure that the data remains fit for purpose across analytics, data science, and machine learning use cases. They also assess existing data pipelines, identify quality bottlenecks, and automate various remediation steps.

White Box Testing Tutorial: A Comprehensive Guide With Examples and Best Practices

Aggregated on: 2023-06-02 12:30:44

White box testing is a software testing method in which the internal structure and implementation of the software being tested are known and used as the basis for designing the test cases. It involves testing the software at the code level and requires a deep understanding of the code and the design of the software. It is also known as glass box, transparent box, clear box, or structural testing. Most testers have already had some experience with this box-testing at some point in their careers. Techniques like this are battling for existence in a world that is becoming increasingly agile-driven. Adopting agile approaches does not include putting off any tasks to meet the project effectively.

How To Approach Java, Databases, and SQL [Video]

Aggregated on: 2023-06-02 04:30:44

We want to save our thumbnail data to a database so that we can render our pictures to a nice HTML gallery page and finish the proof of concept for our Google Photos clone! Which database should we use and why? Which Java database API? What database tools will make our lives easier along the way? Find out in this episode of Marco Codes!

Using Render Log Streams to Log to Papertrail

Aggregated on: 2023-06-01 23:15:44

The console.log function — the poor man’s debugger — is every JavaScript developer’s best friend. We use it to verify that a certain piece of code was executed or to check the state of the application at a given point in time. We may also use console.warn to send warning messages or console.error to explain what happened when things have gone wrong. Logging makes it easy to debug your app during local development. But what about debugging your Node.js app while it’s running in a hosted cloud environment? The logs are kept on the server, to which you may or may not have access. How do you view your logs then?

The Dark Side of DevSecOps and Why We Need Governance Engineering

Aggregated on: 2023-06-01 21:30:44

For today’s software organizations security has never been more top of mind. On one side, there is the present and growing threat of being hacked by malicious actors, set out in Crowdstrike’s recent Global threat report. And on the other, there is a wave of cybersecurity regulation from the government to mitigate such cybersecurity vulnerabilities. Software organizations feel the heat from both sides as they work to improve their security posture in ways that will also achieve audit and compliance with new rules. To meet these challenges, we hear a lot about the software supply chain and the importance of securing it with DevSecOps approaches and golden paths to production. Since the Biden administration's executive order in 2021, we’ve heard a lot about software bills of materials (SBOMs), and new products, services, and vendors have sprung up to help us with them. But do DevSecOps and supply chain approaches meet our cybersecurity challenges head-on?

Effective Java Collection Framework: Best Practices and Tips

Aggregated on: 2023-06-01 18:15:44

Java collection framework provides a variety of classes and interfaces, such as lists, sets, queues, and maps, for managing and storing collections of related objects. In this blog, we go over effective Java collection framework: best practices and tips. What Is a Collection Framework? The Java collection framework is a key element of Java programming. To effectively use the Java collection framework, consider factors like utilizing the enhanced for loop, generics, avoiding raw types, and selecting the right collection.

Angular Unit Testing With Karma and Jasmine

Aggregated on: 2023-06-01 18:15:44

Angular unit testing checks isolated pieces of code in an Angular app. It allows users to add new features without interrupting any other part of their application. Jasmine is a JavaScript testing framework, and Karma is a node-based testing tool for JavaScript codes across multiple real browsers. This blog helps you get started with Angular unit testing leveraging Karma and Jasmine. Introduction to Angular Unit Testing First things first, you must have Angular installed on your machine. That is where you need to start Angular installation. If you already have Angular installed, feel free to skip the next step.

JavaScript Temperature Anomaly Chart

Aggregated on: 2023-06-01 17:45:44

Hello! In this article, we will create a Temperature Anomaly chart.

Turbocharge Ab Initio ETL Pipelines: Simple Tweaks for Maximum Performance Boost

Aggregated on: 2023-06-01 17:30:43

Ab Initio is a powerful data integration and processing platform that enables the design and development of data integration applications. The software consists of two core components: GDE — Graphical Development Environment: The Graphical Development Environment (GDE) is the primary development tool in Ab Initio. It provides a graphical interface for designing, developing, and managing data integration applications, referred to as graphs. The GDE allows developers to construct graphs by connecting various components together, providing a visual representation of the data flow. Co>Operating System: The Co>Operating System (Co>Op) is a critical component of Ab Initio's data integration platform. It serves as the underlying operating system that facilitates the execution of Ab Initio graphs and manages the necessary resources for data processing. Designed for distributed computing environments, the Co>Op enables parallel execution of Ab Initio applications. While Ab Initio offers impressive performance by default, there are several performance tweaks that can significantly enhance the data processing applications. Performance tuning in Ab Initio is crucial to optimize the execution of your applications and ensure efficient data processing. By implementing these tweaks, you can further boost performance and improve overall efficiency.

Navigating Digital Assurance With a Scrum Master: Maximizing Quality in Agile Projects

Aggregated on: 2023-06-01 17:00:43

The scrum master simplifies the scrum process, which includes assisting the team in expanding and improving their task management. They are responsible for providing emerging Scrum team members with real leadership and coordinating them to deliver high-quality products on time. So companies that are keen on providing digital assurance services can help their digital teams look into such processes. An Increase in Scrum Masters Working in Digital Assurance During the pandemic, every sector moved into a digital environment. During this phase, the scrum master played a vital role and delivered a high-quality product on time to the customer.

Internationalization and Localization: The Challenges

Aggregated on: 2023-06-01 16:00:43

In a globalized world, our software companies serve the needs of customers that happen to have their business in multiple geographical regions. Hence we developers need to ensure that the software we build is usable in different languages and cultural contexts. In other words, our software must be designed with internationalization (i18n) in mind. This includes the employment of Unicode character sets, flexible layouts (allowing bi-directional texts), or externalizing strings that vary across languages. Localization (l10n) is a process of using i18n to adapt the software to the locale, language, and cultural requirements of a given market. Very good definitions of internationalization and localization can be found in this W3C article. Moreover, UX, Engineering, Product Management, and other relevant stakeholders must ensure that the level/depth of i18n and l10n is consistent across the product suite. For example, it can be very user-unfriendly if one part of the UI is in Spanish while another is in English. It can be even misleading if one product uses commas as a thousand separator while the other as a decimal separator. Not to mention the American date format mm/dd/yyyy vs. the European dd/mm/yyyy. This is especially problematic in embedded use cases, where the user has no way of telling where one UI ends and the other begins.

Simplify Kubernetes Resource Management With Sveltos, Carvel ytt, and Flux

Aggregated on: 2023-06-01 16:00:43

Managing Kubernetes add-ons can be a challenging task, especially when dealing with complex deployments and frequent configuration changes. In this article, we will explore how Sveltos and Carvel ytt can work together to simplify Kubernetes resource management. Sveltos is a powerful Kubernetes add-on management tool, while Carvel ytt is a templating and patching tool for YAML files. We will delve into the integration of Carvel ytt with Sveltos using the ytt controller, enabling seamless deployment and configuration management. Introducing Sveltos Sveltos is an open-source project tool that simplifies the process of managing and deploying add-ons to Kubernetes clusters. It provides a comprehensive solution for installing, configuring, and managing add-ons, making it easier to enhance the functionality and capabilities of Kubernetes.Sveltos provides support for Helm charts, Kustomize, and resource YAMLs.

How To Use an Automatic Sequence Diagram Generator

Aggregated on: 2023-06-01 16:00:43

You may have heard about or experienced the value of sequence diagrams. In this article, I’ll introduce sequence diagrams and describe what they are good for. I’ll also give you a tour of the features of sequence diagrams and explain how to use each feature to interpret the behavior of your code. Then I’ll show how AppMap makes sequence diagrams even better by (a) generating diagrams from your running code and (b) making diagrams fully interactive. What Is a Sequence Diagram? Sequence diagrams are a type of Unified Modeling Language (UML) diagram that shows the interactions between different components or objects in a system over time. They show the flow of messages and the order in which these messages are exchanged. They are great for understanding how code works because they show enough detail to communicate the key elements of code behavior while being compact enough to fit a lot of information onto a single page.

Leveraging RuntimeFabric To Deploy Mule Runtime to AWS (EKS)

Aggregated on: 2023-06-01 15:30:46

Runtime Fabric (RTF) is a particular deployment model based on container service where the Mule runtime can be deployed on the cloud (AWS/Azure/GKE/OpenShift) or on a data center (on-premise). It provides all cloud (PaaS) benefits, such as high availability, automatic failover, rolling updates, etc. This article provides the prescriptive steps to migrating Mule CloudHub to RTF on EKS Deployment Model. It also describes changes to the application, process, ASM/Monitoring, or Observability required to harness the power of RTF.

How To Integrate the Stripe Payment Gateway Into a React Native Application

Aggregated on: 2023-06-01 15:00:43

You’ve developed an incredible software application with a rich feature set! Well, your job doesn’t end here. You need to also integrate a secure payment gateway into your app to enable in-app payment transactions. The payment functionality has become a crucial component in app development these days as the facility of in-app payments is a standard expectation of modern-day app users. This feature is a must-have in finance and eCommerce applications, but most other apps, even social media apps, are now offering this feature as a USP. Integrating the payment feature into an application is not an easy task and has to be carried out flawlessly and following standard practices. This post will discuss in detail how to integrate the payment processing feature into your app while adhering to the best practices. Here, you’ll learn all about how to integrate the Stripe payment gateway into a React Native application. We have chosen Stripe as it is one of the most secure and widely used platforms globally. We considered React Native as it is one of the most preferred frameworks for developing new-age cross-platform apps. So, let’s get started.

10 Effective Strategies To Safeguard Your IoT Network From Threats

Aggregated on: 2023-06-01 14:30:43

IoT is one of the leading technologies in the world and has added a new level of comfort and convenience for people. Whether it is your home, office, or industry, IoT enables us to do work more quickly and precisely. However, when using IoT devices, various people often neglect the security aspect of IoT networks. As we all know, IoT devices are smart and designed to gather our data all the time. This means your network if not secured, is vulnerable to hacks, thefts, and unauthorized access to your private information. Therefore, you must take adequate steps to prevent any lethal attacks on your IoT network and keep it safe.

An Overview of Cloud Cryptography

Aggregated on: 2023-06-01 14:30:43

Cloud Cryptography refers to a set of techniques used to secure data stored and processed in the cloud environment and is essential for protecting sensitive information. By using encryption and secure key management techniques, Cloud Cryptography provides the following security: Data Privacy Data Confidentiality Data Integrity The three common methods used in Cloud Cryptography are as follows:

The Role of Automation in Streamlining DevOps Processes

Aggregated on: 2023-06-01 13:45:43

Automation is a fundamental aspect of DevOps that revolutionizes the software development and delivery process. It involves using tools and technologies to automate manual and repetitive tasks, resulting in streamlined processes, increased efficiency, and improved outcomes. However, organizations often face challenges when relying on manual tasks, such as human errors, delays, and scalability limitations. By implementing automation in DevOps, businesses can overcome these challenges and unlock numerous benefits. Automation enables faster and more frequent releases through continuous integration and continuous deployment (CI/CD), reducing time-to-market and enhancing agility. It simplifies infrastructure provisioning and configuration management, ensuring consistent and reliable environments across development, testing, and production stages. Automated testing and quality assurance processes enhance code quality, reduce bugs, and improve software reliability. Additionally, automation facilitates effective monitoring and logging, enabling proactive identification and resolution of issues.

From On-Prem to SaaS

Aggregated on: 2023-06-01 13:15:43

Organizations that develop software solutions are often classified according to the targeted consumers: business shops whose solutions support the business (e.g., internal consumption) and software shops whose solutions are sold (more correctly licensed) to customers (e.g., external consumption). This classification is not closed — i.e., what is a consulting firm hired to develop internal applications or a business shop whose internal application is sold – but I’m comfortable sticking with a business shop and software shop. Software companies generate revenue by selling/licensing their solutions, along with other supporting services. Historically, the customer is responsible for installing the purchased/licensed software within their corporate infrastructure: acquiring/installing the requisite infrastructure (i.e., servers, storage, networking, security), provisioning supporting tools (i.e., messaging solutions, database schemas, disk storage, security groups), and, finally, installing the application. The software company may provide consulting time for more complex solutions, but ultimately a successful deployment is in the customer’s hands.

The State of Data Streaming for Manufacturing

Aggregated on: 2023-06-01 13:15:43

This blog post explores the state of data streaming for manufacturing in 2023. The evolution of industrial IoT, manufacturing 4.0, and digitalized B2B and customer relations require modern, open, and scalable information sharing. Data streaming allows integrating and correlating data in real time at any scale. I look at trends like software-defined manufacturing and how data streaming helps modernize and innovate the entire engineering and sales lifecycle. The foci are trending enterprise architectures in the manufacturing industry and data streaming customer stories from BMW, Mercedes, Michelin, or Siemens. A complete slide deck and on-demand video recording are included. General Trends in the Manufacturing Industry Researchers, analysts, startups, and last but not least, labs and the first real-world rollouts of traditional players show a few upcoming trends in the manufacturing industry:

Power of PHP Frameworks: Boosting Web Development With PHP Frameworks

Aggregated on: 2023-06-01 13:15:43

PHP has emerged as a dominant programming language for creating dynamic and interactive websites in the expansive web development realm. Its rich ecosystem of frameworks that simplify development processes, enhance code reusability, and advocate best practices are a significant factor contributing to PHP's popularity popularity is its rich ecosystem of frameworks that simplify development processes, enhance code reusability, and advocate best practices. PHP frameworks offer a structured approach to web application development, allowing developers to concentrate on functionality and efficiency.

Knowing and Valuing Apache Kafka’s ISR (In-Sync Replicas)

Aggregated on: 2023-06-01 12:45:43

To get more clarity about ISR in Apache Kafka, we should first carefully examine the replication process in the Kafka broker. In short, replication means having multiple copies of our data spread across multiple brokers. Maintaining the same copies of data in different brokers makes possible the high availability in case one or more brokers go down or are untraceable in a multi-node Kafka cluster to server the requests. Because of this reason, it is mandatory to mention how many copies of data we want to maintain in the multi-node Kafka cluster while creating a topic. It is termed a replication factor, and that’s why it can’t be more than one while creating a topic on a single-node Kafka cluster. The number of replicas specified while creating a topic can be changed in the future based on node availability in the cluster. On a single-node Kafka cluster, however, we can have more than one partition in the broker because each topic can have one or more partitions. The Partitions are nothing but sub-divisions of the topic into multiple parts across all the brokers on the cluster, and each partition would hold the actual data(messages). Internally, each partition is a single log file upon which records are written in an append-only fashion. Based on the provided number, the topic internally split into the number of partitions at the time of creation. Thanks to partitioning, messages can be distributed in parallel among several brokers in the cluster. Kafka scales to accommodate several consumers and producers at once by employing this parallelism technique. This partitioning technique enables linear scaling for both consumers and providers. Even though more partitions in a Kafka cluster provide a higher throughput but with more partitions, there are pitfalls too. Briefly, more file handlers would be created if we increase the number of partitions as each partition maps to a directory in the file system in the broker.

Decoding eBPF Observability: How eBPF Transforms Observability as We Know It

Aggregated on: 2023-05-31 23:15:43

There has been a lot of chatter about eBPF in cloud-native communities over the last 2 years. eBPF was a mainstay at KubeCon, eBPF days and eBPF summits are rapidly growing in popularity, companies like Google and Netflix have been using eBPF for years, and new use cases are emerging all the time. Especially in observability, eBPF is expected to be a game changer. So let’s look at eBPF — what is the technology, how is it impacting observability, how does it compare with existing observability practices, and what might the future hold?

Replacing Apache Hive, Elasticsearch, and PostgreSQL With Apache Doris

Aggregated on: 2023-05-31 23:15:43

I worked as a real-time computing engineer for a due diligence platform, which is designed to allow users to search for a company's business data, financial, and legal details. It has collected information of over 300 million entities in more than 300 dimensions. The duty of my colleagues and I is to ensure real-time updates of such data so we can provide up-to-date information for our registered users. That's the customer-facing function of our data warehouse. Other than that, it needs to support our internal marketing and operation team in ad-hoc queries and user segmentation, which is a new demand that emerged with our growing business. Our old data warehouse consisted of the most popular components of the time, including Apache Hive, MySQL, Elasticsearch, and PostgreSQL. They support the data computing and data storage layers of our data warehouse:

Never Use Credentials in a CI/CD Pipeline Again

Aggregated on: 2023-05-31 23:15:43

As someone who builds and maintains cloud infrastructure, I have always been leery from a security perspective of giving third-party services, such as CI/CD platforms, access to resources. All the service vendors claim to take stringent precautions and implement foolproof processes, but still, vulnerabilities get exploited and errors happen. Therefore, my preference is to use tools that can be self-hosted. However, I may not always have a choice if the organization is already committed to an external partner, such as Bitbucket Pipelines or GitHub Actions. In that case, in order to apply some Terraform IaC or deploy to an autoscaling group, there is no choice but to furnish the external tool with an API secret key, right? Wrong! With the proliferation of OpenID Connect, it is possible to give third-party platforms token-based access that does not require secret keys. The problem with a secret is that there is always a chance of it leaking out. The risk increases the more it is shared, which happens as employees leave and new ones join. One of them may disclose it intentionally or they may be the victim of phishing or a breach. When a secret is stored in an external system, that introduces an entire new set of potential leak vectors. Mitigating the risk involves periodically changing the credentials, which is a task that adds little perceptible value.

Writing a Vector Database in a Week in Rust

Aggregated on: 2023-05-31 20:15:43

Vector databases are currently all the rage in the tech world, and it isn't just hype. Vector search has become ever more critical due to artificial intelligence advances which make use of vector embeddings. These vector embeddings are vector representations of word embeddings, sentences, or documents that provide semantic similarity for semantically close inputs by simply looking at a distance metric between the vectors. The canonical example from word2vec in which the embedding of the word "king" was very near the resulting vector from the vectors of the words "queen", "man", and "woman" when arranged in the following formula:

Microservices With Apache Camel and Quarkus

Aggregated on: 2023-05-31 18:45:43

Apache Camel is everything but a new arrival in the area of the Java enterprise stacks. Created by James Strachan in 2007, it aimed at being the implementation of the famous "EIP book" (Enterprise Integration Patterns by Gregor Hohpe and Bobby Woolf, published by Addison Wesley in October 2003). After having become one of the most popular Java integration frameworks in early 2010, Apache Camel was on the point of getting lost in the folds of history in favor of a new architecture model known as Enterprise Service Bus (ESB) and perceived as a panacea of the Service Oriented Architecture (SOA). But after the SOA fiasco, Apache Camel (which, meanwhile, has been adopted and distributed by several editors including but not limited to Progress Software and Red Hat under commercial names like Mediation Router or Fuse) is making a powerful comeback and is still here, even stronger for the next decade of integration. This comeback is also made easier by Quarkus, the new supersonic and subatomic Java platform.

Rule-Based Prompts: How To Streamline Error Handling and Boost Team Efficiency With ChatGPT

Aggregated on: 2023-05-31 17:15:43

Personally, I have achieved outstanding results by incorporating ChatGPT into our team's workflow. This has enabled us to simplify the preparation of user stories and technical documentation, reduce the need for communication between various departments, and decrease our dependence on analysts. In this article, I will provide a specific example that will illustrate just how we achieved all of these goals with the help of ChatGPT and the rule-based prompts. The Challenges of Constructing Effective Prompts When interacting with ChatGPT and other generative models, the main goal is to achieve the best possible results based on prompts. However, there are some challenges in constructing prompts that will ensure the AI follows instructions correctly. Issues often arise due to the structure of the prompt, causing the AI to either not fully follow the request or focus on unnecessary ‘noisy’ words.

What Is Envoy Proxy?

Aggregated on: 2023-05-31 17:15:43

The article will cover the following topics: Why is Envoy proxy required? Introducing Envoy proxy Envoy proxy architecture with Istio Envoy proxy features Use cases of Envoy proxy Benefits of Envoy proxy Demo video - Deploying Envoy in K8s and configuring as a load balancer Why Is Envoy Proxy Required? Challenges are plenty for organizations moving their applications from monolithic to microservices architecture. Managing and monitoring the sheer number of distributed services across Kubernetes and the public cloud often exhausts app developers, cloud teams, and SREs. Below are some of the major network-level operational hassles of microservices, which shows why Envoy proxy is required.

Building a Robust Data Engineering Pipeline in the Streaming Media Industry: An Insider’s Perspective

Aggregated on: 2023-05-31 17:15:43

In this detailed and personal account, the author shared his journey of building and evolving data pipelines in the rapidly transforming streaming media industry. Drawing from his extensive experience, the author highlights the fundamental role data engineering plays in the industry, explaining the construction and challenges of typical data pipelines and discussing the specific projects that marked significant transformations. The article delves into technical aspects such as real-time data processing, ETL processes, and cloud technologies and provides insights into the future of data engineering within the industry. The piece serves as an invaluable resource for data professionals seeking to understand the dynamic interplay of data engineering and streaming media, emphasizing the need for adaptability, continuous learning, and effective collaboration. In the last two decades, data engineering has dramatically transformed industries. With multiple years of experience as an industry leader, I've had the privilege of witnessing this change and, indeed, driving it. Nowhere has this transformation been more apparent than in the streaming media industry.

Ad-Hoc Testing: A Comprehensive Guide With Examples and Best Practices

Aggregated on: 2023-05-31 16:30:43

Ad-hoc testing is the process of creating tests based on a need to test. These tests are not created as part of a set plan or to test specific features but rather to ensure that specific areas of the product work correctly. It is also performed as part of regression testing to ensure that new patches and fixes do not cause any issues with the current version of the product. A tester must be willing to challenge any idea, no matter how good or absurd. One of the best ways to test your ideas is through Adhoc testing, which is a great way to discover new issues and risks with minimal effort. It allows you to take a more creative approach and try new things to find a problem by not formalizing the process.

Database Integration Tests With Spring Boot and Testcontainers

Aggregated on: 2023-05-31 16:30:43

With Spring Data JPA, you can easily create database queries and test them with an embedded H2 database. But sometimes, testing against a real database is much more useful, especially if we use queries tied to a specific database implementation.

Constructing Real-Time Analytics: Fundamental Components and Architectural Framework — Part 1

Aggregated on: 2023-05-31 16:00:43

The old adage "patience is a virtue" seems to have lost its relevance in the fast-paced world of today. In an era of instant gratification, nobody is inclined to wait. If Netflix buffering takes too long, users won't hesitate to switch platforms. If the closest Lyft seems too distant, users will readily opt for an alternative. The demand for instant results is pervading the realm of data analytics as well, particularly when dealing with large datasets. The capacity to provide immediate insights, make swift decisions, and respond to real-time data without any delay is becoming crucial. Companies like Netflix, Lyft, Confluent, and Target, along with thousands of others, have managed to stay at the forefront of their industries partially due to their adoption of real-time analytics and the data architectures that facilitate such instantaneous, analytics-driven operations.

Data Exploration Using Serverless SQL Pool In Azure Synapse

Aggregated on: 2023-05-31 15:30:43

Azure Synapse Analytics, formerly known as Azure SQL Data Warehouse, is a cloud-based analytics service provided by Microsoft Azure. It provides capabilities like data exploration, data analysis, data integration, advanced analytics, machine learning, etc., on the Azure Data Lake Blob Storage. In this article, we will take a deep dive into the ingestion of data files into Azure Synapse, data analysis, and transformation of the files in Azure Synapse using a Serverless SQL Pool. This article expects you to have a basic understanding of Azure fundamentals. Open your Synapse workspace and have it ready. Let's go.

Unlocking Game Development: A Review of ‘Learning C# By Developing Games With Unity'

Aggregated on: 2023-05-31 15:00:43

I've recently had the pleasure of immersing myself in Harrison Ferrone's Learning C# by Developing Games with Unity. I'd heard it was a great primer for those of us eager to get our feet wet in the gaming industry, and I can now confirm it's a game changer. This book wasn't just written for tech-savvy gamers. It was written for the curious beginner, the coding newbie, and the Unity novice. Ferrone breaks down intimidating coding concepts into bite-sized, digestible chunks, making this complex world not only accessible but actually enjoyable.

ChatGPT Will Not Replace Your Job, but the People Using ChatGPT as Their Assistant Will

Aggregated on: 2023-05-31 15:00:43

The development of generative AI has been a true game-changer in the world of technology. It has unlocked new and exciting possibilities for creating and generating original content that was previously only achievable through human effort. By leveraging the power of artificial intelligence to analyze patterns in existing data, generative AI can now create original text, images, and music that are virtually indistinguishable from those created by humans. This groundbreaking advancement has immense implications for various industries, including media, advertising, and entertainment. One industry that stands to benefit significantly from the integration of generative AI is the media sector. In today's fast-paced world, where news and articles are constantly being produced to keep the public informed, the efficiency and speed offered by generative AI can be of great value to newsrooms and publishers. By utilizing generative AI, news articles and product descriptions can be created quickly and efficiently, enabling newsrooms to keep up with the ever-increasing demand for content. However, it is important to note that while generative AI can generate content, the human touch is still crucial in producing material that resonates with readers. Human writers possess a unique ability to craft compelling stories, develop persuasive arguments, and create engaging content that genuinely touches the hearts and minds of the audience. Through the infusion of human creativity and emotion, content takes on a deeper level of connection and authenticity.

Operator Overloading in Java

Aggregated on: 2023-05-31 14:30:43

In this post, we'll delve into the fascinating world of operator overloading in Java. Although Java doesn't natively support operator overloading, we'll discover how Manifold can extend Java with that functionality. We'll explore its benefits, limitations, and use cases, particularly in scientific and mathematical code. We will also explore three powerful features provided by Manifold that enhance the default Java-type safety while enabling impressive programming techniques. We'll discuss unit expressions, type-safe reflection coding, and fixing methods like equals during compilation. Additionally, we'll touch upon a solution that Manifold offers to address some limitations of the var keyword. Let's dive in!