Leverage the fast and elegant framework to extract web content

Laptop and desktop computer on desk
Laptop and desktop computer on desk
Photo by Domenico Loia on Unsplash.

Web scraping is a handy tool to have in your arsenal. It can be useful in a variety of situations, like when a website does not provide an API or you need to parse and extract web content programmatically. TK (https://www.devdungeon.com/content/web-scraping-go)

This tutorial walks through using the standard library to perform a variety of tasks like making requests, changing headers, setting cookies, using regular expressions, and parsing URLs. It also covers the basics of the package to scrape information from an HTML web page on the internet. TK

There are many web scraping frameworks on Go. I have chosen Colly

“It works on my machine.” This line has become an excuse often overused by developers to escape the burden of having to investigate problems experienced by end-users that they are unable to replicate on their development machines. While this may excuse developers — after all, they didn’t experience the problem during development and therefore couldn’t have anticipated it happening to some users — it doesn’t solve the problem for the end-users.

Here we’re going to take a look at two of the most popular tools — Docker and Vagrant — to see how they try to solve these problems and…

Modern applications rely on microservices to remain scalable and efficient. Kubernetes provides the perfect environment for microservices to exist and work together with the tools and features it offers. As each part of the application gets placed in a container, the system as a whole becomes more scalable.

The use of microservices and containers also lends itself to today’s CI/CD workflow. There is no need to bring the entire system down for an update because each microservice (container) can be updated separately. This, in turn, makes the lifecycle of containers shorter.

During the lifetime of an app and it’s microservices…

Kubernetes has already solved the container orchestration problem. The remaining challenge of the cloud-native ecosystem is to make microservice delivery more efficient and resilient. This can be accomplished with service mesh technology.

Open-source service mesh projects like Istio, Envoy and Linkerd are gaining popularity in recent years. This post explores the basics of service meshes, the challenges of plain vanilla Kubernetes, and introduces several Kubernetes service mesh products.

What Is a Service Mesh?

A service mesh is a network infrastructure layer that controls and visualizes the communication between different parts of an application. Modern applications often work in this way. The network is separated into…

Amazon EC2 Container Service (ECS) is Amazon’s solution for running and orchestrating Docker containers. It provides an interface for defining and deploying Docker containers to run on clusters of EC2 instances.

The initial setup and configuration of an ECS cluster is not exactly trivial, but once configured it works well and makes running and scaling container-based applications relatively smooth. ECS also has support for blue-green deployments built in, but first we’ll cover some basics about getting set up with ECS and about what is Blue/Green Deployments.

Why Blue/Green Deployments

With hardware becoming more and more cost-effective, Blue/Green deployments are becoming common. Blue/Green deployments…

For the past few weeks, I have been digging a lot into Chef. Although the documentation is good, there have been a lot of times when I was stuck with no clue whatsoever. So I will be giving a quick walk through on how can you setup your own infra using chef by just following the below steps. If you haven’t heard about Chef at all don’t worry I will explain all of it.

What is Chef and why?

Chef is a powerful automation platform that transforms infrastructure into code. …

I’ll start this article by saying I never wrote much blogs as I thought myself always as a person who lacked the essence of using words to show my feelings or knowledge. But it all changed in the last year when I started reading Blogs on Medium. Medium was the first Blog Aggregator Website which was gaining a lot of traction, the sole reason for this success in Initial Phase goes to the fluid UI Design they used which made reading so much easier and less hassle due to the No AD Policy. Then I started writing small blogs in…

Kubernetes is a powerful container orchestrator and has been establishing itself as IT architects’ container orchestrator of choice. But Kubernetes’ power comes at a price; jumping into the cockpit of a state-of-the-art jet puts a lot of power under you, but knowing how to actually fly it is not so simple. That complexity can overwhelm a lot of people approaching the system for the first time. Google has been using Kubernetes for quite some time and it was their endeavour to make the Kubernetes Project Open Source.

Part 1: Getting Started

Just Starting the Journey

With a machine as powerful as Kubernetes, I like to identify the absolute…

Running Docker in production has quickly become the norm. Cloud hosting providers like AWS, Azure and Google Cloud realized that this is what Organizations are in need for. Services like EKS(Elastic Kubernetes Service) and ECS(Elastic Container Service) from AWS offer a completely managed environment for your Docker containers to run on. Through this article, we’ll take a closer look to one of them, Amazon ECS, which is Amazon Elastic Container Service. We are going to describe what AWS ECS is, its functions, and its importance in the current market.

“AWS ECS is a fully-managed, scalable and production-ready platform for running…

The process is like a child playing with shape sorter, although sometimes you might need to do modifications either to the shape or to the hole that fits that shape. So, as a DevOps professional, it’s important to do a personal evaluation of any technology. Without further ado, here are my thoughts on serverless.


I am a professional Backend and DevOps Developer I mainly work on NodeJS and Python based Backend. On Cloud I have my forte on AWS and Azure.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store