Posts in "Python" category

In Python world Celery is a popular tool for running background tasks. It includes Celery beat that allows to run periodic tasks on some schedule. At my current work I participated in developing a system for running multiple periodic tasks. Since our technology stack is based on Python, Celery beat was a natural choice. One of the requirements to the system was granular control over individual tasks, that is, the ability to start/stop/restart each task individually. The straightforward solution was to run multiple Celery beat/worker pairs for each task, but after some googling it turned out that running multiple Celery beat instances seemed to be impossible. At least everybody said so, e.g. check this Stack Overflow discussion. If you try to do so you'll get duplicate tasks in your workers because each beat sends tasks to each worker. (...)

Django offers several out-of-the-box class-based views, including CreateView and UpdateView, that allow you to quckly implement basic CRUD functionality in your web-application without much of boilerplate code. Under the hood CreateView and UpdateView classes create, validate and save a Django ModelForm instance tied to your model class. But sometimes you need to create/edit related model instances along with the main model on the same page. (...)

Recently I needed to extract text from a PDF file using Python. Quick googling led me to PyPDF2 package, however I wasn't able to extract any text from my test PDF with it. The test PDF was created with Google Docs (a very common scenario) and did not have any fancy formatting, so PyPDF2 was disqualified for my purposes. After further googling I found pdfminer package and its Python 3 compatible version — pdfminer.six. (...)

Note 2: This article was written before GitHub Actions became publicly available so now it has more of a historic value.

Note: This is a re-worked version of my previous article. I it has been updated based on practical experience and to reflect recent changes in Travis CI.

GitHub supports several cloud-based continuous integration services. One of them is Travis CI that allows to run automated testing and deployment tasks on Linux Ubuntu and macOS. Unfortunately, their Ubuntu environment is quite dated — currently they use Ubuntu 14.04 LTS (Trusty Tahr) or Ubuntu 16.04 (Xenial Xerus) while the current LTS release (at the moment of this writing) is Ubuntu 18.04 LTS (Bionic Beaver), so some of the recent language and tooling versions may not be available in Travis CI build environment. (...)

In the 1st part of my tutorial I demonstrated how to create a simple Scrapy spider that collects data from a website. Now I will show you how to save scraped data in JSON and XLSX formats. (...)