News Aggregator

Why Detecting Generic Credentials Is a Gamechanger

Aggregated on: 2021-11-10 19:20:54

Introduction Working for a company specialized in detecting secrets (if you don’t know what a secret is, please take a moment here and come back), we had to address the question: what would be a good way to categorize secrets? Take a look at this:

Test-Driven Development With The oclif Testing Library: Part Two

Aggregated on: 2021-11-10 18:50:54

In Part One of this series on the oclif testing library, we used a test-driven development approach to building our time-tracker CLI. We talked about the oclif framework, which helps developers dispense with the setup and boilerplate so that they can get to writing the meat of their CLI applications. We also talked about @oclif/test and @oclif/fancy-test, which take care of the repetitive setup and teardown so that developers can focus on writing their Mocha tests. Our time-tracker application is a multi-command CLI. We’ve already written tests and implemented our first command for adding a new project to our tracker. Next, we’re going to write tests and implement our “start timer” command.

Why Continuous Performance Testing for Retail Apps Matters

Aggregated on: 2021-11-10 18:05:54

The retail industry has been actively adopting digital transformation in order to provide a better user experience. According to current trends, the reliance on digital channels has been enormous, placing them at the core of all significant online retail operations. Mobile sales increased by 68 percent in 2020 and are anticipated to surpass other channels as the largest source of all sales by the end of this year. Can you identify the true reason for increased sales using retail mobile applications? The answer is "digital confidence."

Main Features and Benefits of Google Kubernetes Engine

Aggregated on: 2021-11-10 17:35:54

In the modern technology world, the technical domain is inclining towards cloud computing as it solves various problems such as accessibility and scalability. Most of the time, people use the same resources for running multiple software or programs on various operating systems, which creates inconsistencies. But this issue eradicates with Google Kubernetes Engine or GKE as it includes containers that make programs independent of OS and speeds up the app development process using solutions created with the cloud ecosystem. GKE is the simplest way for deploying, scaling, and managing apps through Google infrastructure. In this blog, you will understand Kubernetes in detail, GKE’s salient features, and the advantages you can get by implementing it.

Best Accessories and External Components for AI Computers

Aggregated on: 2021-11-10 17:35:54

What Are the Best Accessories and Components for a System Built for AI? You don’t have to look far or wide to find guides on building the best gaming rigs. Finding the best accessories for AI computers, though? That is a tougher search, although there can be some overlap. There aren’t many companies talking about the ins and outs of DIY AI computer build-essentials. It is exactly for that reason that we have compiled a list of the most important components you will need for artificial intelligence (AI) build and what we recommend.

My Favorite Quotes From Best Programming Books To Motivate Yourself

Aggregated on: 2021-11-10 16:35:53

I have always had a strong interest in knowing great people in my field i.e., programming. Knowing about them, following them, and reading them not only gives you immense knowledge but the motivation you need to excel in what you are doing. It's motivation, not the knowledge that will propel your career. Knowledge is useless without motivation because knowing is not enough, you need to apply that knowledge, and without motivation, you just can't do that. There are times when the programmers also felt tired and bored. These are the times when you spent hours constantly looking at the computer screen, clicking here and there, surfing the net without doing anything.

Kubeflow Fundamentals Part 4: External Add-ons

Aggregated on: 2021-11-10 15:50:53

Welcome to the fourth blog post in our “Kubeflow Fundamentals” series specifically designed for folks brand new to the Kubelfow project. The aim of the series is to walk you through a detailed introduction of Kubeflow, a deep-dive into the various components, add-ons, and how they all come together to deliver a complete MLOps platform. If you missed the previous installments in the “Kubeflow Fundamentals” series, you can find them here:

What is Data Lineage and How Can It Ensure Data Quality?

Aggregated on: 2021-11-10 15:20:53

Introduction Are you spending too much time tracking down bugs for your C-level dashboards? Are different teams struggling to align on what data is needed throughout the organization? Or are you struggling with getting a handle on what the impact of a potential migration could be? Data lineage could be the answer you need for data quality issues. By improving data traceability and visibility, a data lineage system can improve data quality across your whole data stack and simplify the task of communicating about the data that your organization depends on.

Transaction API Support for Codeless

Aggregated on: 2021-11-10 14:20:53

The purpose of transactions is to protect your data from partial database updates that fail to be complete. The Transaction API is now available for Codeless. Transaction API With the Transaction API, our Database can execute multiple database requests grouped into a single transaction. When any one of these grouped database operations fails, the entire transaction is rolled back – meaning any other changes within the same transaction are canceled.

Codeless OAuth API and Email API Attachments

Aggregated on: 2021-11-10 14:05:53

The OAuth API allows you to get the full benefits of third-party OAuth login but with a new Codeless block, your application can include login with the likes of Facebook, Twitter, Google, Amazon, LinkedIn, Discord, GitHub, Spotify, and more. In addition to the OAuth API, Backendless has also added new functionality for adding email attachments via API, which is also now available in Codeless. Email attachments – PDFs, images, videos, etc. – can even be added to email templates using Codeless. Read on to learn more.

Interviewing and Hiring Software Performance Test Professionals - Book Review

Aggregated on: 2021-11-10 11:35:53

A few weeks ago, I reviewed The Hitchhiking Guide To Load Testing Projects: A Fun, Step-by-Step Walk-Through Guide by Leandro Melendez. It is good to see more books on the performance testing/engineering genre recently. One of my favorite books is The Art of Application Performance Testing series by Ian Molyneaux. On this line, James Pulley's book is different, it is completely a non-technical book in the performance testing/engineering genre. In this review, I am going to share my views about the book, and please note that I have not been compensated to write a review about the book; it is my honest and unbiased review.

Want to Know How to Run Selenium Tests in Docker?

Aggregated on: 2021-11-10 06:35:53

Automation testing and Continuous Integration (CI) are integral parts of the development and test activity. Selenium test automation is one such approach that helps in the end-to-end testing of the web product. The not-so-preferred way of performing tests using the Selenium framework involves installing the required web browser and its corresponding browser drivers. In this blog, we deep dive into how to run Selenium tests in Docker in order to accelerate the Selenium test automation activity. Introduction to Docker When it comes to Selenium automation testing, it is important that a test run in one execution environment does not hinder the execution of tests run in another test environment (s). Hence, automation tests should be run in isolation, and Docker helps in realizing this ‘essential’ requirement.

How to Scan for Personal Data Across Your Systems

Aggregated on: 2021-11-10 05:50:53

Before You Start Before scanning for personal data in your systems, you need to understand the technical aspects of your system components, the types of files you need to scan, and which services you have direct control over. Aside from scanning and detecting data, you also need to be ready to interpret what you find and understand its importance.

How Bokeh Secures Its Open-Source Repositories

Aggregated on: 2021-11-10 04:50:53

Open-source is everywhere, it is one of the driving forces of software innovation from the academic to the enterprise world (75 percent of codebases audited by Synopsys in the 2021 OSSRA report rely on open-source components). Its prevalence in commercial software is reaching unprecedented levels, to the extent that the European Commission has recently identified it as a public good in a recent study assessing its impact on the region’s economy. But the interstitial nature of open-source in modern software also makes it a subject of security and compliance concerns, as it is capable of exposing organizations that use it to a host of unknown risks and vulnerabilities. Most discussions we are hearing today around security in this space are focused on the identification, fixing, and remediation of vulnerabilities — all seen from the “consumer” perspective.

iOS Crystalline Blurred Backgrounds with CSS Backdrop Filters

Aggregated on: 2021-11-10 04:50:53

iOS is full of cool crystalline glass effects. This effect has long been easy to do when you have easy access to graphic shaders since these can do most of the heavy lifting in terms of calculating what is underneath the layer, and blurring it into the foreground. However, for a long time, it hasn't been possible in CSS... until now. In more recent specifications of CSS, we have finally gotten backdrop-filter. Mostly meant for modals, it also has applications on stylized dropdowns which are common across the web.

API Security Issue 155

Aggregated on: 2021-11-10 03:50:53

This week, we have a vulnerability in the BrewDog mobile app exposing users’ PII courtesy of hard-coded bearer tokens, Cisco has announced the arrival of their APIClarity at KubeCon 2021, F5 has published a report on API attacks in Open Banking, and finally, there’s a mega-guide on API security best practices. Vulnerability: Hard-Coded API Bearer Token in Brewdog Mobile App

Data Masking: SQL Server vs Gallium Data

Aggregated on: 2021-11-10 03:50:53

Introduction In SQL Server 2016, Microsoft introduced a new feature called dynamic data masking, which allows you to mask the values of certain columns and keep that data hidden from certain users, without having to modify your applications. Let's take a look at how SQL Server does data masking, and compare it to the way Gallium Data goes about it.

3 Reasons You Should Talk About Release Schedules More Often

Aggregated on: 2021-11-10 03:50:53

Release schedules drive many of the processes for IT teams. The problem is, business teams, don’t like release schedules. Maybe it’s because they don’t understand the need for the formal process or they feel release cycles slow down the delivery of new features and fixes. Whatever the reason, if you work as a developer or in DevOps, talking about release schedules with your business stakeholders is important.

My First Thoughts as an Engineering Manager

Aggregated on: 2021-11-10 03:05:53

Recently, I've joined Nextail Labs as an Engineering Manager. This is my first experience working in a Software Startup and also as an Engineering Manager. I've been leading Engineers Teams most of my professional career with other roles including the following: Tech Lead in a small software consultant company. Solution Architect and Team Lead of consultant team of an important software vendor. Team Lead and Product Owner in an important fashion retailer company. None of these roles were focused on the people, there were always other main goals. Today, I know how different they are and how different the challenge is.

Building High-Quality Software

Aggregated on: 2021-11-10 01:35:53

start as early as possible\n\nIt’s better to add these gates as early as possible. It’s much better to build your process around quality checks than retrofit these checks into the existent process. NIST did classic research to show that catching bugs at the beginning of the development process could be more than ten times cheaper than if a bug reaches production. If you start catching bugs early, it will save you tons of time fixing them later.\n\n> Design review\n\nIt’s a very powerful tool when used in a good way. It sits at the very beginning of the process before the code is written and can save an immense amount of time down the road (of somebody spending tons of time just to get to a dead-end). It really helps to talk through the problem, the solution, alternative ideas, corner cases, and so on. I really like what one of the smartest people with whom I worked said: “A good design is a design where you can see the code”. It’s like working with the code without writing it.\n\nUnfortunately, I know multiple very senior engineers who really like to go with “fire, aim, ready” approach. Let’s put together a prototype (even before thinking about different alternatives), let’s call this prototype an alpha version, and fix bugs and limitations in it for the years to come. Saving several hours preparing and doing a design review will cost hundreds (if not thousands of hours) of fixing issues down the road.\n\n> unit tests\n\nI don’t believe that I have to say that in 2021, but I have never seen a quality product without unit tests. Period. There are so many benefits. It helps to prove that your code does what it should do, unit tests removes all simple problems. They help to get rid of a lot of flaky behavior and this list goes way beyond catching bugs. Yeah. It’s not a silver bullet, but it can easily catch a very high percentage of all your bugs.\n\n> code review\n\nAgain, nothing new here. Somebody looking at your code and saying “WTF?” is a great way to see where your code is over complex/brittle/doesn’t handle some scenarios. Important note. As with any non-automatic checks, you get as much from it as you invest (rubber stamping PR won’t add any value).\n\n> Monitoring\n\nWe (humans) are terrible at imagining all possible permutations of the system with billions and billions of possible states. All of our testing (both unit tests and integration tests cover a tiny sliver of all states). And, unfortunately, the only place where you can see everything that can happen is the production.\n\nIt’s incredible how many people entirely ignore it. You may think that you know how the system works. In the best case, you know only how the system was designed to work. Many more complex and subtle problems emerge only in production and could be caught via monitoring/alerting/analysis.\n\nThis is probably the newest addition to my list. Like everything else on this list, I had to learn it the hard way. After several outages which could have been prevented by trivial monitoring/alerting/analysis, you start treating your monitoring as a first-class citizen.\n\n> manual testing\n\nYeap. I said it. We live in a time when everybody is irked by manual testing. I tend to agree that you don’t want to spend tons of time doing only manual testing. However, it’s a must-have for most products to work well. Automation testing catches predicted problems but is almost useless for unpredicted issues.\n\nThere were so many times when one of the best QA person who I worked with came to me saying something like: “I don’t know. It works, but there is something funky in there”. This sentence is not a binary result of tests, and if it was reported by some automated tool, people would easily ignore it as a false positive. However, as soon as I hear it from this QA person, it raises a huge red flag.\n\n> root cause\n\nYou don’t need to analyze each tiny bug. However, as soon as you have some severe bugs escaping, you need to figure out whether you need to beef up one of the games (which should have caught it) or whether you need to introduce additional gates to detect such types of bugs.\n\nNICE TO HAVE\n\n> the static code analysis tool (and similar tools)\n\nTheir efficiency depends a lot on the language and a tool. The beauty of it is that it’s completely automatic and, as a result, very cheap. There are some languages (like C++) where this should be on the must-have list. Other languages may be harder to handle with such tools.\n\n> end-to-end (integration) test\n\nSome level of integration tests is helpful to see that your system works as a whole. However, It’s useful as a seasoning for unit tests and not as a main dish.\n\nIt’s great to have maybe one or two end-to-end tests for some major features. However, it’s not a unit test. You can’t cover everything, and more importantly, supporting it will cost you, so you don’t even want to try to cover everything.\n\nANTI-PATTERNS\n\n> Excessive manual regression testing\n\nOn the one hand, I understand where it’s coming from. As the company’s customer base grows, the impact of bugs becomes more significant. As a result, there is a desire to catch all regression bugs. However, usually excessive regression testing shows a lack of other gates that catches the problems. As a result, there is an overemphasis on the last regression verification.\n\n> End-to-ends tests as a replacement for unit tests\n\nAs a counter-reaction to manual regression testing, which takes more and more time, companies will try to replace it with excessive automated end-to-end tests. Unfortunately, this especially often happens for a code with poor quality and low unit test coverage. It almost always ends up a costly endeavor (even more expensive than regression testing), resulting in many very fragile tests that are failing left and right.\n\nI saw a company that tried to retrofit quality like that and created a set of 8000 copy/paste end-to-end tests. Last time I heard, about 80% pass and 20% fail on each run. This 20% is somewhat ignored because trying to analyze 1600 failed tests is pretty much impossible. They are rerun(in the best case) and thus defeating the whole purpose of this exercise while spending tons and tons of time/money/energy on this).\n\n> Manage quality purely via metrics\n\nMaking high-quality products requires a lot of attention to detail (understanding where the problems are, the best way to catch them, where are the strong places and so on). Metrics abstract you away from all details. You can gauge metrics fast, but you can’t (read shouldn’t) make a decision purely based on them.\n\nTo be honest, this concentration on metrics boggles my mind. I saw a company spending a nontrivial amount of time gathering all these statistics, asking people to constantly fill out gazillion JIRA fields, google spreadsheets, and so on, just to say at the end, “This component is in good shape, and this one is in bad.” The funny thing is that any SRE working in a company for more than a year could have provided this info in 10 minutes without wasting the time of half of the engineering.\n\nBTW. A side note. As soon as some process (like gathering metrics) becomes a goal (vs. being a tool), you will see more of these time-wasting activities with little or no output.\n\nSummarizing. As you can see, nothing is magical, and very little is unconventional here. However, as I mentioned initially, the thing I see missing in a lot of these discussions is this systematic analysis: defense-in-depth, choosing the proper gates, being retrospective and detail-oriented. And what is even more sobering is that many companies have very few people who have a clear mental model for building high-quality software.\n\nP.S. The list above is obviously not exhaustive, and it’s more of high-level items which could be easily plugged into the development process and can be easily applied to the whole team. There are tons and tons of different practices which can improve quality on a personal level (e.g., TDD, thinking through edge cases, code conciseness, and so on).\n\n\n***\nThis story was originally published at https://medium.com/nerd-for-tech/building-high-quality-software-1efa45c719bc on July 16, 2021.\n\nPlease follow me on Medium, subscribe via email and share this article."}">I have interviewed many engineers and managers lately, and one of the standard questions I ask is how to build high-quality software. Of course, I provide more context and explanations, but the gist is the same. start as early as possible\n\nIt’s better to add these gates as early as possible. It’s much better to build your process around quality checks than retrofit these checks into the existent process. NIST did classic research to show that catching bugs at the beginning of the development process could be more than ten times cheaper than if a bug reaches production. If you start catching bugs early, it will save you tons of time fixing them later.\n\n> Design review\n\nIt’s a very powerful tool when used in a good way. It sits at the very beginning of the process before the code is written and can save an immense amount of time down the road (of somebody spending tons of time just to get to a dead-end). It really helps to talk through the problem, the solution, alternative ideas, corner cases, and so on. I really like what one of the smartest people with whom I worked said: “A good design is a design where you can see the code”. It’s like working with the code without writing it.\n\nUnfortunately, I know multiple very senior engineers who really like to go with “fire, aim, ready” approach. Let’s put together a prototype (even before thinking about different alternatives), let’s call this prototype an alpha version, and fix bugs and limitations in it for the years to come. Saving several hours preparing and doing a design review will cost hundreds (if not thousands of hours) of fixing issues down the road.\n\n> unit tests\n\nI don’t believe that I have to say that in 2021, but I have never seen a quality product without unit tests. Period. There are so many benefits. It helps to prove that your code does what it should do, unit tests removes all simple problems. They help to get rid of a lot of flaky behavior and this list goes way beyond catching bugs. Yeah. It’s not a silver bullet, but it can easily catch a very high percentage of all your bugs.\n\n> code review\n\nAgain, nothing new here. Somebody looking at your code and saying “WTF?” is a great way to see where your code is over complex/brittle/doesn’t handle some scenarios. Important note. As with any non-automatic checks, you get as much from it as you invest (rubber stamping PR won’t add any value).\n\n> Monitoring\n\nWe (humans) are terrible at imagining all possible permutations of the system with billions and billions of possible states. All of our testing (both unit tests and integration tests cover a tiny sliver of all states). And, unfortunately, the only place where you can see everything that can happen is the production.\n\nIt’s incredible how many people entirely ignore it. You may think that you know how the system works. In the best case, you know only how the system was designed to work. Many more complex and subtle problems emerge only in production and could be caught via monitoring/alerting/analysis.\n\nThis is probably the newest addition to my list. Like everything else on this list, I had to learn it the hard way. After several outages which could have been prevented by trivial monitoring/alerting/analysis, you start treating your monitoring as a first-class citizen.\n\n> manual testing\n\nYeap. I said it. We live in a time when everybody is irked by manual testing. I tend to agree that you don’t want to spend tons of time doing only manual testing. However, it’s a must-have for most products to work well. Automation testing catches predicted problems but is almost useless for unpredicted issues.\n\nThere were so many times when one of the best QA person who I worked with came to me saying something like: “I don’t know. It works, but there is something funky in there”. This sentence is not a binary result of tests, and if it was reported by some automated tool, people would easily ignore it as a false positive. However, as soon as I hear it from this QA person, it raises a huge red flag.\n\n> root cause\n\nYou don’t need to analyze each tiny bug. However, as soon as you have some severe bugs escaping, you need to figure out whether you need to beef up one of the games (which should have caught it) or whether you need to introduce additional gates to detect such types of bugs.\n\nNICE TO HAVE\n\n> the static code analysis tool (and similar tools)\n\nTheir efficiency depends a lot on the language and a tool. The beauty of it is that it’s completely automatic and, as a result, very cheap. There are some languages (like C++) where this should be on the must-have list. Other languages may be harder to handle with such tools.\n\n> end-to-end (integration) test\n\nSome level of integration tests is helpful to see that your system works as a whole. However, It’s useful as a seasoning for unit tests and not as a main dish.\n\nIt’s great to have maybe one or two end-to-end tests for some major features. However, it’s not a unit test. You can’t cover everything, and more importantly, supporting it will cost you, so you don’t even want to try to cover everything.\n\nANTI-PATTERNS\n\n> Excessive manual regression testing\n\nOn the one hand, I understand where it’s coming from. As the company’s customer base grows, the impact of bugs becomes more significant. As a result, there is a desire to catch all regression bugs. However, usually excessive regression testing shows a lack of other gates that catches the problems. As a result, there is an overemphasis on the last regression verification.\n\n> End-to-ends tests as a replacement for unit tests\n\nAs a counter-reaction to manual regression testing, which takes more and more time, companies will try to replace it with excessive automated end-to-end tests. Unfortunately, this especially often happens for a code with poor quality and low unit test coverage. It almost always ends up a costly endeavor (even more expensive than regression testing), resulting in many very fragile tests that are failing left and right.\n\nI saw a company that tried to retrofit quality like that and created a set of 8000 copy/paste end-to-end tests. Last time I heard, about 80% pass and 20% fail on each run. This 20% is somewhat ignored because trying to analyze 1600 failed tests is pretty much impossible. They are rerun(in the best case) and thus defeating the whole purpose of this exercise while spending tons and tons of time/money/energy on this).\n\n> Manage quality purely via metrics\n\nMaking high-quality products requires a lot of attention to detail (understanding where the problems are, the best way to catch them, where are the strong places and so on). Metrics abstract you away from all details. You can gauge metrics fast, but you can’t (read shouldn’t) make a decision purely based on them.\n\nTo be honest, this concentration on metrics boggles my mind. I saw a company spending a nontrivial amount of time gathering all these statistics, asking people to constantly fill out gazillion JIRA fields, google spreadsheets, and so on, just to say at the end, “This component is in good shape, and this one is in bad.” The funny thing is that any SRE working in a company for more than a year could have provided this info in 10 minutes without wasting the time of half of the engineering.\n\nBTW. A side note. As soon as some process (like gathering metrics) becomes a goal (vs. being a tool), you will see more of these time-wasting activities with little or no output.\n\nSummarizing. As you can see, nothing is magical, and very little is unconventional here. However, as I mentioned initially, the thing I see missing in a lot of these discussions is this systematic analysis: defense-in-depth, choosing the proper gates, being retrospective and detail-oriented. And what is even more sobering is that many companies have very few people who have a clear mental model for building high-quality software.\n\nP.S. The list above is obviously not exhaustive, and it’s more of high-level items which could be easily plugged into the development process and can be easily applied to the whole team. There are tons and tons of different practices which can improve quality on a personal level (e.g., TDD, thinking through edge cases, code conciseness, and so on).\n\n\n***\nThis story was originally published at https://medium.com/nerd-for-tech/building-high-quality-software-1efa45c719bc on July 16, 2021.\n\nPlease follow me on Medium, subscribe via email and share this article."}">I heard all kinds of answers. However, I was puzzled that almost none were systematic, and people immediately went into a specific pet peeve. As part of this exercise, I felt that I had to crystalize my answer to this question and write it down. start as early as possible\n\nIt’s better to add these gates as early as possible. It’s much better to build your process around quality checks than retrofit these checks into the existent process. NIST did classic research to show that catching bugs at the beginning of the development process could be more than ten times cheaper than if a bug reaches production. If you start catching bugs early, it will save you tons of time fixing them later.\n\n> Design review\n\nIt’s a very powerful tool when used in a good way. It sits at the very beginning of the process before the code is written and can save an immense amount of time down the road (of somebody spending tons of time just to get to a dead-end). It really helps to talk through the problem, the solution, alternative ideas, corner cases, and so on. I really like what one of the smartest people with whom I worked said: “A good design is a design where you can see the code”. It’s like working with the code without writing it.\n\nUnfortunately, I know multiple very senior engineers who really like to go with “fire, aim, ready” approach. Let’s put together a prototype (even before thinking about different alternatives), let’s call this prototype an alpha version, and fix bugs and limitations in it for the years to come. Saving several hours preparing and doing a design review will cost hundreds (if not thousands of hours) of fixing issues down the road.\n\n> unit tests\n\nI don’t believe that I have to say that in 2021, but I have never seen a quality product without unit tests. Period. There are so many benefits. It helps to prove that your code does what it should do, unit tests removes all simple problems. They help to get rid of a lot of flaky behavior and this list goes way beyond catching bugs. Yeah. It’s not a silver bullet, but it can easily catch a very high percentage of all your bugs.\n\n> code review\n\nAgain, nothing new here. Somebody looking at your code and saying “WTF?” is a great way to see where your code is over complex/brittle/doesn’t handle some scenarios. Important note. As with any non-automatic checks, you get as much from it as you invest (rubber stamping PR won’t add any value).\n\n> Monitoring\n\nWe (humans) are terrible at imagining all possible permutations of the system with billions and billions of possible states. All of our testing (both unit tests and integration tests cover a tiny sliver of all states). And, unfortunately, the only place where you can see everything that can happen is the production.\n\nIt’s incredible how many people entirely ignore it. You may think that you know how the system works. In the best case, you know only how the system was designed to work. Many more complex and subtle problems emerge only in production and could be caught via monitoring/alerting/analysis.\n\nThis is probably the newest addition to my list. Like everything else on this list, I had to learn it the hard way. After several outages which could have been prevented by trivial monitoring/alerting/analysis, you start treating your monitoring as a first-class citizen.\n\n> manual testing\n\nYeap. I said it. We live in a time when everybody is irked by manual testing. I tend to agree that you don’t want to spend tons of time doing only manual testing. However, it’s a must-have for most products to work well. Automation testing catches predicted problems but is almost useless for unpredicted issues.\n\nThere were so many times when one of the best QA person who I worked with came to me saying something like: “I don’t know. It works, but there is something funky in there”. This sentence is not a binary result of tests, and if it was reported by some automated tool, people would easily ignore it as a false positive. However, as soon as I hear it from this QA person, it raises a huge red flag.\n\n> root cause\n\nYou don’t need to analyze each tiny bug. However, as soon as you have some severe bugs escaping, you need to figure out whether you need to beef up one of the games (which should have caught it) or whether you need to introduce additional gates to detect such types of bugs.\n\nNICE TO HAVE\n\n> the static code analysis tool (and similar tools)\n\nTheir efficiency depends a lot on the language and a tool. The beauty of it is that it’s completely automatic and, as a result, very cheap. There are some languages (like C++) where this should be on the must-have list. Other languages may be harder to handle with such tools.\n\n> end-to-end (integration) test\n\nSome level of integration tests is helpful to see that your system works as a whole. However, It’s useful as a seasoning for unit tests and not as a main dish.\n\nIt’s great to have maybe one or two end-to-end tests for some major features. However, it’s not a unit test. You can’t cover everything, and more importantly, supporting it will cost you, so you don’t even want to try to cover everything.\n\nANTI-PATTERNS\n\n> Excessive manual regression testing\n\nOn the one hand, I understand where it’s coming from. As the company’s customer base grows, the impact of bugs becomes more significant. As a result, there is a desire to catch all regression bugs. However, usually excessive regression testing shows a lack of other gates that catches the problems. As a result, there is an overemphasis on the last regression verification.\n\n> End-to-ends tests as a replacement for unit tests\n\nAs a counter-reaction to manual regression testing, which takes more and more time, companies will try to replace it with excessive automated end-to-end tests. Unfortunately, this especially often happens for a code with poor quality and low unit test coverage. It almost always ends up a costly endeavor (even more expensive than regression testing), resulting in many very fragile tests that are failing left and right.\n\nI saw a company that tried to retrofit quality like that and created a set of 8000 copy/paste end-to-end tests. Last time I heard, about 80% pass and 20% fail on each run. This 20% is somewhat ignored because trying to analyze 1600 failed tests is pretty much impossible. They are rerun(in the best case) and thus defeating the whole purpose of this exercise while spending tons and tons of time/money/energy on this).\n\n> Manage quality purely via metrics\n\nMaking high-quality products requires a lot of attention to detail (understanding where the problems are, the best way to catch them, where are the strong places and so on). Metrics abstract you away from all details. You can gauge metrics fast, but you can’t (read shouldn’t) make a decision purely based on them.\n\nTo be honest, this concentration on metrics boggles my mind. I saw a company spending a nontrivial amount of time gathering all these statistics, asking people to constantly fill out gazillion JIRA fields, google spreadsheets, and so on, just to say at the end, “This component is in good shape, and this one is in bad.” The funny thing is that any SRE working in a company for more than a year could have provided this info in 10 minutes without wasting the time of half of the engineering.\n\nBTW. A side note. As soon as some process (like gathering metrics) becomes a goal (vs. being a tool), you will see more of these time-wasting activities with little or no output.\n\nSummarizing. As you can see, nothing is magical, and very little is unconventional here. However, as I mentioned initially, the thing I see missing in a lot of these discussions is this systematic analysis: defense-in-depth, choosing the proper gates, being retrospective and detail-oriented. And what is even more sobering is that many companies have very few people who have a clear mental model for building high-quality software.\n\nP.S. The list above is obviously not exhaustive, and it’s more of high-level items which could be easily plugged into the development process and can be easily applied to the whole team. There are tons and tons of different practices which can improve quality on a personal level (e.g., TDD, thinking through edge cases, code conciseness, and so on).\n\n\n***\nThis story was originally published at https://medium.com/nerd-for-tech/building-high-quality-software-1efa45c719bc on July 16, 2021.\n\nPlease follow me on Medium, subscribe via email and share this article."}">Let me start with high-level thoughts (specifically to make it systematic). First of all, I want to concentrate on software code quality (vs. larger topics, including problem definition, documentation, UX, design, etc.). High-quality software is software that has fewer bugs (and a shorter tail of fixing remaining issues). There are a bunch of other things like code readability, maintainability, debugability, and so on which can easily be swept under the quality umbrella. Let’s concentrate on the core that the product operates as expected.

Node.js Vs. Python: Pros, Cons, and Use Cases

Aggregated on: 2021-11-10 00:05:54

When choosing a programming language for the backend development, your choice determines how the product will operate, scale, and fulfill user demands. One of the most common is the dilemma of Node.js vs. Python. The two options are hugely popular and have their pros and cons. We work with both and are here to compare their advantages and disadvantages and help you to decide which one is better for your project.

How to Create High Order Components With Angular

Aggregated on: 2021-11-10 00:05:54

I was working with React Projects for a long time. I have evolved a mindset that allows you to abuse HOC to make code more reusable and somewhat clean. But when I got into Angular, it felt weird at first that there is no such concept. The Spark At some point, I got in touch with Modals and there it was the gong that awoke the ninja in me. First, I dove again into angular.io docs and indeed found a way to pass components through @Input() and render them. But that was unsatisfying because of the directive. That could be because decorators are something new after React. Or because it splits such a simple concept as HOC into too many files.

Logic Behind Software — allMatch On Empty Stream

Aggregated on: 2021-11-09 23:35:53

Basically, it works this way because of the logic concept called Vacuous truth. It is a simple and short answer but because we do not like such answers, we will dive deeper. But why I even decided to write about such a niche topic. For some time, I have been interested in how mathematics affects the software engineering world directly, and I have even written an article about calculating scalability. I believe such niche topics can be quite an interesting thing for you. We will start by shortly presenting the main problem described in today's text.

How to Build HTML Forms Right: Security

Aggregated on: 2021-11-09 22:35:53

While many guides to creating forms for the web are mainly focused on the frontend, security goes beyond that. We have to consider the current user, other users, and our own security. As such, we will look at the whole application architecture from frontend to backend and beyond. Encrypt Traffic (SSL) Before we get too far, I will be using the term “SSL” to refer to a technology used to encrypt traffic on the internet. Technically, I mean Transport Layer Security (TLS), but “SSL” is commonly used and understood to mean the same thing. It’s what gives websites the little green lock in the URL bar and why they start with “https” instead of “http” (no “s”).

PyTorch Lightning Tutorial: Using TorchMetrics and Lightning Flash

Aggregated on: 2021-11-09 22:35:53

TorchMetrics unsurprisingly provides a modular approach to define and track useful metrics across batches and devices, while Lightning Flash offers a suite of functionality facilitating more efficient transfer learning and data handling, and a recipe book of state-of-the-art approaches to typical deep learning problems. We’ll start by adding a few useful classification metrics to the MNIST example we started with earlier. We’ll also swap out the PyTorch Lightning Trainer object with a Flash Trainer object, which will make it easier to perform transfer learning on a new classification problem. We’ll then train our classifier on a new dataset, CIFAR10, which we’ll use as the basis for a transfer learning example to CIFAR100.

Kubernetes Logging in Production

Aggregated on: 2021-11-09 22:05:53

Historically, in monolithic architectures, logs were stored directly on bare metal or virtual machines. They never left the machine disk and the operations team would check each one for logs as needed. This worked on long-lived machines, but machines in the cloud are ephemeral. As more companies run their services on containers and orchestrate deployments with Kubernetes, logs can no longer be stored on machines, and implementing a log management strategy is of the utmost importance.

Connecting to and Using Google’s Cloud SQL With Ballerina

Aggregated on: 2021-11-09 22:05:53

Introduction Cloud SQL Cloud SQL is a cloud-based alternative to local MySQL, PostgreSQL, and SQL server databases that is a part of the Google Cloud Platform Suite. You can follow this guide to create your own Cloud SQL instance in 5 minutes. Ballerina Ballerina is an open-source programming language for the cloud that makes it easier to use, combine, and create network services. To learn more about Ballerina, visit the official website.

How to Automate Restful APIs Using Jayway Library

Aggregated on: 2021-11-09 22:05:53

Software APIs have been one of the most trending technologies. While most functional testing requires interaction with the user interface, API testing involves by-passing the user interface and communicating directly with an application server by making calls to its API. Successful web applications and mobile apps are all the results of the greatest API at the backend. In agile methodology, most of the QA’s time is spent automating against the GUI using Selenium or any other framework. But in this article, we would look up the smart way to automate our APIs so that UI automation can be reduced a little and automation testing can be done more reliably.

Taming the Moose: Classing up Perl Attributes

Aggregated on: 2021-11-09 19:35:53

At my work, we extensively use the Moose object system to take care of what would ordinarily be very tedious boilerplate object-oriented Perl code. In one part of the codebase, we have a family of classes that, among other things, map Perl methods to the names of various calls in a third-party API within our larger organization. Those private Perl methods are in turn called from public methods provided by roles consumed by these classes so that other areas aren't concerned with said API's details. Without going into too many specifics, I had a bunch of classes all with sections that looked like this:

Test-driven development with Quarkus

Aggregated on: 2021-11-09 18:50:53

Many development teams today have adopted test-driven development (TDD). Continuous testing support in Quarkus enables developers to take advantage of this practice. When running Quarkus Dev Mode, you can enable continuous testing with the press of a key, empowering Quarkus to automatically rerun tests affected by a code change in the background. Quarkus understands which tests are affected by classes and methods within the application. As you make code changes, you get immediate feedback if the change passes your existing test suite. This capability is integrated directly into Quarkus—no IDE or special tooling is required. The future of developer productivity and joy is now!

Tutorial: How to Define SQL Functions With Presto Across All Connectors

Aggregated on: 2021-11-09 18:50:53

Presto is the open-source SQL query engine for data lakes. It supports many native functions which are usually sufficient for most use cases. However, there is maybe a corner case where you need to implement your own function. To simplify this, Presto allows users to define expressions as SQL functions. These are dynamic functions separated from the Presto source code, managed by a functions namespace manager that you can set up with a MySQL database. In fact, this is one of the most widely used features of Presto at Facebook, with over 1000s of functions defined. Function Namespace Manager A function namespace is a special catalog.schema that stores functions in the format like mysql.test. Each catalog.schema can be a function namespace. A function namespace manager is a plugin that manages a set of these function catalog schemas. Catalog can be mapped to connectors in Presto (a connector for functions, no tables or view) and allows the Presto engine to perform actions such as creating, altering, and deleting functions.

What To Include in Your CV

Aggregated on: 2021-11-09 16:05:53

There are hundreds of articles on how to properly draw up a CV. In this article, we will look at what you can do to insert the information about it later into your CV. Participation in all these activities develops both your hard skills and soft skills, and recently they are especially relevant. 1) Open Source Participating in open source projects will give you the skill of reading code and experience working with large projects, and most importantly, it will show your employer that you can put your skills into practice. Reading code well is very important. At first, you will spend most of your time at the company reading the code and understanding the project. Therefore, the skill to understand the code on your own will help you a lot.

Interview - Thought-provoking Conversation With AI Expert, Joanna Bryson

Aggregated on: 2021-11-09 16:05:53

It's easy to spend hours doom-scrolling bad news about algorithms breaking bad. But instead of jumping down a dystopian rabbit hole, let's take some 'woosah' time, and rewind the tape on AI visionary Joanna Bryson (@j2bryson), as she drops some serious knowledge on: Rooting out bias in AI. Regulating AI without taking the magic out of AI innovation. Prioritizing due diligence in software development. By the way, Bryson is Professor of Ethics and Technology at the Hertie School of Governance in Berlin, where she educates future technologists and policymakers on AI governance, ethics, and collaborative cognition. In 2020, Bryson was among nine experts nominated by Germany to the Global Partnership for Artificial Intelligence. On top of that, she was also recognized as a top digital influencer to watch in 2021 by the European Digital Development Alliance.

Step-By-Step Guide To Enable JSON Logging on OpenShift

Aggregated on: 2021-11-09 16:05:53

OpenShift Logging’s API enables you to parse JSON logs into a structured object and forward them to either OpenShift Logging-managed Elasticsearch or any other third-party system. Prerequisites You will need to install the following technologies before beginning this exercise:

Smart Dependency Injection With Spring: Overview (Part 1 of 3)

Aggregated on: 2021-11-09 15:35:53

Preface Spring Framework is a very powerful framework and provides first-class support for dependency injection (DI). It contains a lot of features or ways to implement DI. Therefore, I decided to share my experience with it in this series. This series contains three articles: Basic usage of DI (this article) DI with assignability DI with generics In This Article, You Will Learn: What is dependency injection? How to implement DI with Spring Framework Which configuration types Spring Framework supports Which variants of injection Spring Framework supports What the injection rules are in Spring Framework What bean types can be injected with Spring Framework Several hints and gotchas for DI with Spring Framework What Is Dependency Injection? Dependency Injection (DI) is a well-known design pattern for a separation of concerns. It's been used for some time. Wikipedia provides us with this definition: Dependency injection separates the creation of a client's dependencies from the client's behavior, which promotes loosely coupled programs and the dependency inversion and single responsibility principles. Fundamentally, dependency injection is based on passing parameters to a method. DI is an example of the more general concept known as Inversion of Control (IoC). The Hollywood principle is also often referred to as a synonym to DI. Used Classes Before we start with an explanation of DI, we should first define the used classes and their dependencies. We use EntityManager, UserRepository and UserService here. The EntityManager is defined by JPA, and the rest is defined by us. We can see their relationship depicted below.

Code Review: A Comprehensive Checklist

Aggregated on: 2021-11-09 15:05:53

A code review is a helpful tool for teams to improve code quality besides many other benefits to reviewing code. Not to mention the reduced development cost when catching bugs early in the development lifecycle, sharing knowledge, and improving the team's estimation skills. In the last section of this article, you can find a code review checklist to use when implementing a code review process in your developer workflow.

SQL JOINs Tutorial With Examples

Aggregated on: 2021-11-09 13:35:53

Introduction to SQL JOINs As we all know, table data is the core of any SQL database. Naturally, SQL Server databases store tons and tons of table data. To use the table data effectively, database administrators need to extract records from several tables based on certain conditions regularly. And that’s exactly what SQL JOINs are for. JOIN is an SQL clause used to retrieve data from two or more tables based on logical relationships between the tables. Joins indicate how SQL Server should use data from one table to select the rows in another table.

11 Best Selenium Alternatives for Testers

Aggregated on: 2021-11-09 12:50:53

Selenium is a free and open-source tool for automated testing. It can test web applications for functional, regression, and load across a variety of browsers and systems. Selenium is an excellent tool, however, it does have certain disadvantages. The following is a selected list of Selenium alternatives that have been thoroughly tested.

Smart Dependency Injection With Spring - Generics (Part 3/3)

Aggregated on: 2021-11-09 12:05:53

Preface The Spring framework is a powerful framework that provides first-class support for dependency injection (DI). This article is the last one in my mini-series dedicated to dependency injection with Spring Framework. This series is split into three articles:

Top Reasons for Hiring QA Analysts

Aggregated on: 2021-11-09 11:50:53

We know software testing services play an integral part in every business to be successful. Quality assurance (QA) testers are essential for any company, they avoid untidy coding and offer excellent user interaction. If you think it’s time to improve your software quality, you need to hire a quality assurance team. There may be times when the development team becomes painfully apparent and get nervous about a new feature, or the customer support gets flooded. It means it's time you hire a software tester to make your system function well. Let’s understand the streamlined process with understanding the ways to hire QA analysts.

Who Makes a Better DevOps Engineer? | Developer Vs. Sysadmin

Aggregated on: 2021-11-09 11:05:53

Who makes a better DevOps engineer? What’s the outcome of developer vs. sysadmin? This article will provide you with the answers to these questions and even more. What’s the Difference? Before getting into details, let’s see what is the difference between a software developer and a system administrator?

Smart Leaders are Good Listeners Who act Wisely

Aggregated on: 2021-11-09 11:05:53

Leadership might be one of the interesting topics in the IT industry where having smart leaders can make a lot of difference. Leader’s decisions in the IT industry not only impact on the employees but also this would impact on the products are being produced and therefore the end-users will be impacted as well. This is how leadership in this area is quite critical and as the software industry is moving forward with advanced technologies, leadership is developing to get adapted to the new generations of engineers and new ways of working. Leader Story One day I got a message from one of the developers that something in design for the migration is not right. I did not know anything about the context that’s why I scheduled a quick call for the day after to understand it. The day after when I understood the context of the issue, I also noticed that developer has a lot of experience with the context and he is also passionate about implementing the new solution for that issue. I was listening to his complaints and he was emphasizing that he has been shouting at people that they are not right about the approach they take for this issue. This situation immediately reminded me many years ago when I was complaining all the time without doing anything. Even though I was right, no one was listening or understanding what I was saying.

Upgrading to .Net 6

Aggregated on: 2021-11-09 10:35:53

.Net 6 SDK was just released by Microsoft, and apparently it's a really kick ass upgrade. It significantly optimises Stream, reduces memory footprint, and contains tons of new cool features. Probably the most important code improvement change for me personally, is the mutable DOM classes for System.Text.Json, which over time will allow me to completely replace Newtonsoft.JSON with the builtin libraries, reducing dependencies in my core. Dependencies is a big thing for me in Magic, which I'm trying to reduce to a minimum. In fact, dependency reduction is arguably my single most important axiom for improving code quality and modularity, something I guess the fact that Magic contains 30+ projects is a testimonial towards. My experience with updating Magic to use version 6 was almost 100% painless. Basically, I did a search and replace through my .csproj file replacing "net5.0" with "net6.0", and everything (almost) immediately worked. 1,000+ unit tests are green, 125+ integration tests are green, and while manually quality assuring features in Magic after updating, I was not able to find any breaking changes at all for me personally. Of course, I am trying as much as possible to use .Net Standard as much as I can in most of these projects, but still there were 20+ .csproj files that were updated during this process.

From TDD to PBT via Kotest

Aggregated on: 2021-11-09 06:05:54

In this post, we see how to integrate PBT into your Kotlin tests. Introduction I’ve been a big fan of Property Based Testing for a number of years, based on my experiences with ScalaCheck. It’s always been an annoyance that Kotlin did not support this testing style, at least to the same extent. There was some functionality in Kotest (formerly KotlinTest), but it paled in comparison to what was available in Scala, F# and Python.

Using Prism for API Mocking and Contract Testing

Aggregated on: 2021-11-09 06:05:54

What is API mocking? In short, it’s about creating a simplified version of an API that works at a mechanical level, even though it’s not implementing the logic of the API. Such an “API mock” then can be used by consumers of an API to start developing applications that use the API. This means that it’s possible to get feedback from consumers earlier in the development cycle, making it faster and more effective to gather early feedback and improve the API design. Prism is an open software tool maintained and used by Stoplight for their API tooling that’s providing API mocking. It’s specifically for REST/HTTP APIs and is using the OpenAPI specification as its description language for APIs.

Reducing Costs and Improving Efficiencies of Your Log Management Investment

Aggregated on: 2021-11-09 03:35:55

People’s involvement has been inevitable with log management despite advancements in ITOps. Log management at a high level collects and indexes all your application and system log files so that you can search through them quickly. It also lets you define rules based on log patterns so that you can get alerts when an anomaly occurs. Log management analytics solution leveraging Robotic Data Automation(RDA) has been able to detect anomalies and aid predictive models over a machine learning layer. This has demonstrated improved efficiencies and a direct reduction in costs.

Installing and Debugging an Apereo Cas Application

Aggregated on: 2021-11-09 03:35:54

The Central Authentication Service (CAS) is a single sign-on software application protocol. The duty of the CAS is to permit a user to access multiple applications while providing their credentials only once to a centralized CAS Server. It also allows web applications to authenticate users without gaining access to a user's security credentials, such as a password. Apereo CAS Apereo CAS is an open-source implementation of above mentioned CAS protocol by the Yale University laboratory. Apero CAS is not only a simply authentication software but also it has got some versatile features such as:

Boost throughput with RESTEasy Reactive in Quarkus 2.2

Aggregated on: 2021-11-09 03:35:54

Quarkus has, from its beginning, provided core features for coding Java programs in both imperative and reactive styles. With the new 2.2 release, Quarkus continues to improve in terms of network-related features, reactive programming, and integration with the Eclipse Vert.x event bus. For example, RESTEasy Reactive in Quarkus is a new JAX-RS implementation based on the Vert.x layer that achieves much higher throughput by handling reactive events on the non-blocking I/O thread. The Red Hat build of Quarkus 2.2 provides production support for the RestEasy Reactive extensions, with additional features:

How to Use SingleStore With Spark ML for Fraud Detection

Aggregated on: 2021-11-09 02:50:53

Abstract SingleStore is a database technology that can easily integrate with a wide range of big data tools and services. One such tool is Apache Spark™. In this 3-part series of articles, we’ll see how easy it is to use the SingleStore Spark Connector and the performance benefits it provides. We’ll also discuss a Credit Card Fraud Detection case study using actual data that we save into SingleStore, and then develop a Machine Learning model using Spark to determine if a Credit Card transaction is fraudulent or not. The notebook files used in this article series are available on GitHub in DBC, HTML, and iPython formats

Advice to My Younger Self as a Software Engineer

Aggregated on: 2021-11-09 02:50:53

A piece of advice to my younger self — “Set all kinds of goals for what you want to achieve in life. No goal is too big with the right plan and vision.” In the past 25 years, I had my fair share of success and failure. I find this journey memorable as it made me understand the importance of failure and success. Every success is close to my heart that constantly provides me a better understanding of team works and a sense of achievement but every failure provided me a new and innovative way to find extraordinary solutions.