Posts in "Python" category

Featured Image

Running Multiple Celery Beat Instances in One Python Project

 Feb. 1, 2021    0 comments 

In Python world Celery is a popular tool for running background tasks. It includes Celery beat that allows to run periodic tasks on some schedule. At my current work I participated in developing a system for running multiple periodic tasks. Since our technology stack is based on Python, Celery beat was a natural choice. One of the requirements to the system was granular control over individual tasks, that is, the ability to start/stop/restart each task individually. The straightforward solution was to run multiple Celery beat/worker pairs for each task, but after some googling it turned out that running multiple Celery beat instances seemed to be impossible. At least everybody said so, e.g. check this Stack Overflow discussion. If you try to do so you'll get duplicate tasks in your workers because each beat sends tasks to each worker. (...)

Read post

 CeleryPython

Featured Image

Formsets In Django Class-based Views

 Dec. 14, 2020    0 comments 

Django offers several out-of-the-box class-based views, including CreateView and UpdateView, that allow you to quckly implement basic CRUD functionality in your web-application without much of boilerplate code. Under the hood CreateView and UpdateView classes create, validate and save a Django ModelForm instance tied to your model class. But sometimes you need to create/edit related model instances along with the main model on the same page. (...)

Read post

 DjangoPython

Featured Image

Extracting Text from a PDF Using Python

 Jan. 6, 2019    0 comments 

Recently I needed to extract text from a PDF file using Python. Quick googling led me to PyPDF2 package, however I wasn't able to extract any text from my test PDF with it. The test PDF was created with Google Docs (a very common scenario) and did not have any fancy formatting, so PyPDF2 was disqualified for my purposes. After further googling I found pdfminer package and its Python 3 compatible version — pdfminer.six. (...)

Read post

 Python

Featured Image

Using Docker with Travis Continuous Integration

 Sept. 16, 2018    0 comments 

Note 2: This article was written before GitHub Actions became publicly available so now it has more of a historic value.

Note: This is a re-worked version of my previous article. I it has been updated based on practical experience and to reflect recent changes in Travis CI.

GitHub supports several cloud-based continuous integration services. One of them is Travis CI that allows to run automated testing and deployment tasks on Linux Ubuntu and macOS. Unfortunately, their Ubuntu environment is quite dated — currently they use Ubuntu 14.04 LTS (Trusty Tahr) or Ubuntu 16.04 (Xenial Xerus) while the current LTS release (at the moment of this writing) is Ubuntu 18.04 LTS (Bionic Beaver), so some of the recent language and tooling versions may not be available in Travis CI build environment. (...)

Read post

 Continuous IntegrationDockerGitHubPython

Featured Image

Scrapy Tutorial - Part 2

 Aug. 7, 2018    0 comments 

In the 1st part of my tutorial I demonstrated how to create a simple Scrapy spider that collects data from a website. Now I will show you how to save scraped data in JSON and XLSX formats. (...)

Read post

 PythonScrapy