Categories
Posts

Adding type information to exported Scrapy items

By default, Scrapy won’t include any type information when using feed exports to serialize scraped items. It follows that, when exporting multiple types of items at once, we later on can’t easily discern between the different concepts represented by the items. Consider the following items.py module: In the above example, the application apparently needs to […]

Categories
Posts

Running background tasks in Django

For modern web applications, running asynchronous tasks in the background is often a must. Whether you need to parallelize something not-so-time-critical (say, thumbnail generation) or access that miraculous-but-really-slow machine learning API in the background, there is a plethora of other use cases that require the developer to isolate time-consuming operations from Django’s default synchronous request-response […]

Categories
Posts

Hosting a Django application with Apache’s mod_wsgi

Install mod_wsgi: Obviously, this step depends on your package manager (which is usually determined by your distribution). On Arch Linux it goes something like this: sudo pacman -Sy mod_wsgi Adapt Apache’s configuration: In order to tell the Apache web server to interface with your Django WSGI app, a few directives need to be added to […]

Categories
Posts

A minimal logging config for Django