Go with multiple workers - Goroutines

What I wanted is give a plan with golang as a job workers . So build very small project initially as a prototype.

Requirements :

  1. Language should be golang
  2. Wanted to plan without multiple servers(service)
  3. Queue management already with rabbitmq
  4. Wanted to run concurrently multiple jobs(any no of jobs define)
  5. To run a job takes some times (minutes)

Let’s plan how we achieve this.

  1. We need to add multiple subscribers for rabbitmq queue.
  2. We need to implement rabbitmq acknowledgement (that will help to system pick same job if job not running. Until its acknowledged info will not delete from the rabbitmq queue). once Job is complete we need to ack

To run concurrently in go we need to use Goroutines (link : https://tour.golang.org/concurrency/1)

Assume following things

  • you have basic idea about golang.
  • rabbitmq connection already available.
  • rabbitmq queue already available.

I will provide another post basic rabbitmq with golang later (how to connect and how to create queue.)

will write a function for generate multiple goroutines . will call that function as “multipleWorkers”

func multipleWorkers() {

for i := 1; i <= 10; i++ {
go worker(i)
}
}

inside that function will create 10 goroutines for run 10 jobs.

Now inside worker function we need to read rabbitmq queue and get one job and ack once job is done.

“failOnError” function is very straight forward that will handle errors for now just show the error.

keep in mind auto-ack is false so we need to send ack status.

var job *Job is job struct

json.Unmarshal(d.Body, &job) will help to json to struct

status := job.Process() run the job and give the bool status

d.Ack(!status) according to the above status, will ack for rabbitmq.

following lines to help to run these forever

forever := make(chan bool)
//and
<-forever

THATS IT…..!

Next post will talk sync.WaitGroup, limited concurrent for many http requests.

--

--

--

VP of Engineering

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

I Am Dev Enough

End of service for imToken versions 1.5 and lower

[TryHackme]- HaskHell Walkthrough

Pair Programming For The Win

Reversing Different Objects in Different Ways in Python

How to use Postman to test ASP NET Core Web API secured with JWT Token authentication

What to do when you’re ready to publish your first website

Using A Class And Instantiating Objects With Inheritance In Python

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Saman Weerasinghe

Saman Weerasinghe

VP of Engineering

More from Medium

Use pprof to view go program stack traces

GoReleaser v1.6 — the boring release

Golang developer — Part 2 — Intermediate of Golang

How to Check nil Interface in Golang?