Blog
Articles about computational science and data science, neuroscience, and open source solutions. Personal stories are filed under Weekend Stories. Browse all topics here. All posts are CC BY-NC-SA licensed unless otherwise stated. Feel free to share, remix, and adapt the content as long as you give appropriate credit and distribute your contributions under the same license.
tags · RSS · Mastodon · simple view
Van der Pol oscillator
In this post, we will apply phase plane analysis to the Van der Pol oscillator. The Van der Pol oscillator is a non-conservative oscillator with nonlinear damping, which was first described by the Dutch electrical engineer Balthasar van der Pol in 1920. We will explore how phase plane analysis can be used to gain insights into the behavior of this system and how it can be used to predict its long-term behavior.
Nullclines and fixed points of the Rössler attractor
After introducing phase plane analysis in the previous post, we will now apply this method to the Rössler attractor presented earlier. We will investigate the system’s nullclines and fixed points, and analyze the attractor’s dynamics in the phase space.
Using phase plane analysis to understand dynamical systems
When it comes to understanding the behavior of dynamical systems, it can quickly become too complex to analyze the system’s behavior directly from its differential equations. In such cases, phase plane analysis can be a powerful tool to gain insights into the system’s behavior. This method allows us to visualize the system’s dynamics in phase portraits, providing a clear and intuitive representation of the system’s behavior. Here, we explore how we can use this method and exemplarily apply it to the simple pendulum.
PyTorch on Apple Silicon
Already some time ago, PyTorch became fully available for Apple Silicon. It’s no longer necessary to install the nightly builds to run PyTorch on the GPU of your Apple Silicon machine as I described in one of my earlier posts.
Rössler attractor
Unlike the Lorenz attractor which emerges from the dynamics of convection rolls, the Rössler attractor does not describe a physical system found in nature. Instead, it is a mathematical construction designed to illustrate and study the behavior of chaotic systems in a simpler, more accessible manner. In this post, we explore how we can quickly simulate this strange attractor using simple Python code.
Understanding Hebbian learning in Hopfield networks
Hopfield networks, a form of recurrent neural network (RNN), serve as a fundamental model for understanding associative memory and pattern recognition in computational neuroscience. Central to the operation of Hopfield networks is the Hebbian learning rule, an idea encapsulated by the maxim ‘neurons that fire together, wire together’. In this post, we explore the mathematical underpinnings of Hebbian learning within Hopfield networks, emphasizing its role in pattern recognition.
Building a neural network from scratch using NumPy
Ever thought about building you own neural network from scratch by simply using NumPy? In this post, we will do exactly that. We will build, from scratch, a simple feedforward neural network and train it on the MNIST dataset.
Python’s version logos
Have you ever noticed that Python has introduced individual version logos starting with version 3.10? I couldn’t find any official announcement, but luckily, the Python community on Mastodon was able to help out.
Conditional GANs
I was wondering whether it would be possible to let GANs generate samples conditioned on a specific input type. I wanted the GAN to generate samples of a specific digit, resembling a personal poor man’s mini DALL•E. And indeed, I found a GAN architecture, that allows what I was looking for: Conditional GANs.
Eliminating the middleman: Direct Wasserstein distance computation in WGANs without discriminator
We explore an alternative approach to implementing WGANs. Contrasting from the standard implementation that requires both a generator and discriminator, the method discussed here employs the optimal transport to compute the Wasserstein distance directly between the real and generated data distributions, eliminating the need for a discriminator.
Wasserstein GANs
We apply the Wasserstein distance to Generative Adversarial Networks (GANs) to train them more effectively. We compare a default GAN with a Wasserstein GAN (WGAN) trained on the MNIST dataset and discuss the advantages and disadvantages of both approaches.
Probability distance metrics in machine learning
Probabilistic distance metrics play a crucial role in a broad range of machine learning tasks, including clustering, classification, and information retrieval. The choice of metric is often determined by the specific requirements of the task at hand, with each having unique strengths and characteristics. In this post, we discuss five commonly used metrics: the Wasserstein Distance, the Kullback-Leibler Divergence (KL Divergence), the Jensen-Shannon Divergence (JS Divergence), the Total Variation Distance (TV Distance), and the Bhattacharyya Distance.
Comparing Wasserstein distance, sliced Wasserstein distance, and L2 norm
In machine learning, especially when dealing with probability distributions or deep generative models, different metrics are used to quantify the ‘distance’ between two distributions. Among these, the Wasserstein distance (EMD), sliced Wasserstein distance (SWD), and the L2 norm, play an important role. Here, we compare these metrics and discuss their advantages and disadvantages.
Approximating the Wasserstein distance with cumulative distribution functions
In the previous two posts, we’ve discussed the mathematical details of the Wasserstein distance, exploring its formal definition, its computation through linear programming and the Sinkhorn algorithm. In this post, we take a different approach by approximating the Wasserstein distance with cumulative distribution functions (CDF), providing a more intuitive understanding of the metric.
Wasserstein distance via entropy regularization (Sinkhorn algorithm)
Calculating the Wasserstein distance can be computational costly when using linear programming. The Sinkhorn algorithm provides a computationally efficient method for approximating the Wasserstein distance, making it a practical choice for many applications, especially for large datasets.
Wasserstein distance and optimal transport
The Wasserstein distance, also known as the Earth Mover’s Distance (EMD), provides a robust and insightful approach for comparing probability distributions and finds application in various fields such as machine learning, data science, image processing, and information theory. In this post, we take a look at the optimal transport problem, required to calculate the Wasserstein distance, and how to calculate the distance metric in Python.
Visualizing Occam’s Razor through machine learning
Here, we illustrate the concept of Occam’s Razor, a principle advocating for simplicity, by examining its manifestation in the domain of machine learning using Python.
Mamba vs. Conda: Unleashing lightning-fast Python package installations
If you’ve ever experienced the frustration of waiting for ages while installing Python packages with conda, there’s a game-changer I wish I’d heard about earlier: Mamba. This lightning-fast package manager surprised me with its incredible speed, making package installations a breeze. Here is my personal experience and why Mamba is the speed demon you may have been looking for.
The Integrate and Fire Model: A simple neuronal model
In this post we explore the Integrate-and-Fire model, a simplified representation of a neuron. We also run some simulations in Python to understand the model dynamics.
Assessing animal behavior with machine learning
High-throughput and multi-modal behavior experiments, coupled with machine learning analysis, unlock valuable insights into complex systems by capturing diverse behavioral responses and deciphering hidden structures within high-dimensional datasets. I just completed a short introductory lecture on this topic, which is now available in the Teachings section.
Bioimage analysis with Napari
I’ve added new teaching material on using the free and open-source software (FOSS) Napari for bioimage analysis. Feel free to use and share it.
Using random forests for pixel classification
Beyond traditional classification problems, random forests have proven their effectiveness in pixel classification. In this post, we will delve into this domain and explore how random forests can be effectively utilized to tackle the task of pixel classification.
Decision Trees vs. Random Forests for classification and regression: A comparison
Decision trees and random forests are popular machine learning algorithms that are widely used for both classification and regression tasks. In this blog post, we elucidate their theoretical foundations and discuss the differences as well as their advantages and drawbacks.
Image denoising techniques: A comparison of PCA, kernel PCA, autoencoder, and CNN
In this post, we explore the performance of PCA, Kernel PCA, denoising autoencoder, and CNN for image denoising.
Using Autoencoders to reveal hidden structures in high-dimensional data
In this Python tutorial, we explore the application of Autoencoders for dimensionality reduction, demonstrating how this powerful technique can help us uncover and interpret hidden patterns within our data.
Unlocking hidden patterns with Factor Analysis
In this Python tutorial, we dive into Factor Analysis, a powerful statistical method used to uncover hidden, or ‘latent,’ variables within high-dimensional datasets. Like PCA, grasping this technique will allow us to simplify complex data structures, thereby aiding in more effective data interpretation and decision-making.
Untangling complexity: harnessing PCA for data dimensionality reduction
This tutorial explores the use of Principal Component Analysis (PCA), a powerful tool for reducing the complexity of high-dimensional data. By delving into both the theoretical underpinnings and practical Python applications, we illuminate how PCA can reveal hidden structures within data and make it more manageable for analysis.
t-SNE and PCA: Two powerful tools for data exploration
Dimensionality reduction techniques play a vital role in both data exploration and visualization. Among these techniques, t-SNE and PCA are widely used and offer valuable insights into complex datasets. In this blog post, we explore te mathematical background of both methods, compare their methodologies, and discuss their advantages and disadvantages. Additionally, we take a look at their practical implementation in Python and compare the results on different sample datasets.
Bridging ideas on the go: WikiLinks come to DEVONthink To Go
The WikiLinks feature has finally arrived on DEVONthink to go, DEVONthink’s mobile app, which unleashes new possibilities to work with your Personal Knowledge Management (PKM) system on the go.
Understanding L1 and L2 regularization in machine learning
Regularization techniques play a vital role in preventing overfitting and enhancing the generalization capability of machine learning models. Among these techniques, L1 and L2 regularization are widely employed for their effectiveness in controlling model complexity. In this blog post, we explore the concepts of L1 and L2 regularization and provide a practical demonstration in Python.
Understanding gradient descent in machine learning
Gradient descent is a fundamental optimization algorithm widely used in machine learning for finding the optimal parameters of a model. It is a powerful technique that enables models to learn from data by iteratively adjusting their parameters to minimize a cost or loss function. In this blog post, we explore the mathematical background of this method and showcase its implementation in Python.
Loading and saving files in Google Colab
Enable I/O support in your notebooks running in Google Colab with just a few additional commands.
Mutual information and its relationship to information entropy
Mutual information is an essential measure in information theory that quantifies the statistical dependence between two random variables. Given its broad applicability, it has become an invaluable tool in diverse fields like machine learning, neuroscience, signal processing, and more. This post explores the mathematical foundations of mutual information and its relationship to information entropy. We will also demonstrate its implementation in some Python examples.
Information entropy
A fundamental concept that plays a pivotal role in quantifying the uncertainty or randomness of a set of data is the information entropy. Information entropy provides a measure of the average amount of information or surprise contained in a random variable. In this blog post, we explore its mathematical foundations and demonstrate its implementation in some Python examples.
Understanding entropy
In physics, entropy is a fundamental concept that plays a crucial role in understanding the behavior of physical systems. It provides a measure of the disorder or randomness within a system, and its study has far-reaching applications across various branches of physics. This blog post aims to provide a brief overview of entropy in order to gain a better understanding of it.
Zen and natural sciences
In this post, I broaden the scope and explore the intersections of Zen and natural sciences more generally.
The Zen of Python
The connection between Zen and programming is not a subjective one at all. For instance, Python has built it directly into its core programming, known as The Zen of Python.
How to get an RSS feed of your Mastodon bookmarks
The third-party service Mastodon Bookmark RSS allows you to subscribe to your Mastodon bookmarks via RSS, so you don’t forget to make use out of them. You can even integrate the feed into your favorite Zettelkasten apps such as DEVONthink and Obsidian.
Problems with large vaults in Obsidian
In the past few days I played a bit with Obsidian. Turns out that its iOS app has some serious problems with large vaults.
DEVONthink and privacy
One thing I really love about DEVONthink, is its high security and privacy measures regarding the synchronization of my notes across different devices. No other app that I have so far used offered such high standards.
Bio-image registration with Python
Which method works best for which registration problem? In this tutorial we compare different methods for the registration of bio-images using Python.
Using VS Code as LaTeX editor
It doesn’t take much to convert Visual Studio Code into a powerful LaTeX editor. Here are the necessary steps that enable full LaTeX support.
Moving a Mastodon account to another server
I recently moved my Mastodon account to a new server, including all my followers. I was surprised, how easy and seamless it worked. Here is a how-to, summarizing the migration steps.
I’m on Mastodon
Mastodon is not just a Twitter alternative. It’s a free and open-source social media platform of its own kind. Here is my story how I got there.
How to run PyTorch on the M1 Mac GPU
As for TensorFlow, it takes only a few steps to enable a Mac with M1 chip (Apple silicon) for machine learning tasks in Python with PyTorch.
How to run TensorFlow on the M1 Mac GPU
In just a few steps you can enable a Mac with M1 chip (Apple silicon) for machine learning tasks in Python with TensorFlow.
Is there a difference between miniconda and miniforge?
Simply said: not really. Miniconda is the company driven minimal conda installer, while miniforge is its community driven variant. In the end, you’ll get the same minimal conda installation on your machine – with a minor difference.
Hacks and extensions to improve your coding with Visual Studio Code
This curated list contains useful hacks and extensions to improve the overall coding performance with Visual Studio Code (VS Code).
Setting up Visual Studio Code for Python
In just a few steps you can turn Visual Studio Code (VS Code) into a powerful Python editor for both pure Python code and Jupyter Notebooks.
Laying off thousands of employees: Not okay! How to delete a Twitter account
Putting thousands of people on the street is anything else than cool. Here is how to fix it.
Enable interactive plots and other plot modes in Jupyter notebooks
Learn how to enable interactive, static and stand-alone window plots in Jupyter notebooks with the magic command
%matplotlib
.
Enable code folding in JupyterLab
Learn how to enable code folding in JupyterLab for both, Jupyter Notebooks and pure Python scripts.
How to create and apply a requirements.txt file in Python
Learn how to install Python packages with a requirements.txt file and how to create one yourself.
Virtual environments with venv
In addition to conda’s
create
command, Python’s built-in venv
command offers another way for creating virtual environments.
Using pip to install Python packages
pip is another package installer for Python. Learn how to use it for installing and managing Python packages in your projects.
How to install and run Python code from GitHub
Learn how to install code from GitHub, that is, e.g., not (yet) available via conda or pip.
A minimal Python installation with miniconda
Learn how to install miniconda to have a quick and minimal Python installation on any operating system. Also learn how to use conda to create and manage virtual environments, install packages, run Python scripts and run Jupyter Notebooks and JupyterLab.
Stable installation of Napari on a M1 Mac
In case you’re having problems installing Napari on your M1 Mac, try to install it from conda instead of pip.
Using Zarr for images – The OME-ZARR standard
As for any other NumPy array, we can use the Zarr file format to store image files. In this post we additionally explore the NGFF (next-generation file format) OME-ZARR standard.
Zarr – or: How to save NumPy arrays
What is Zarr and why is it probably the most suitable file format for saving NumPy arrays?
How to read patch clamp recordings in WaveMetrics IGOR binary files (ibw) in Python
This is a mini tutorial on how to read patch clamp recordings in WaveMetrics IGOR binary files (*.ibw) in Python using the neo and igor packages.
How to add statistical annotations to matplotlib plots
This mini tutorial shows, how to add statistical annotations to matplotlib plots with just a few commands.
The Feynman method as an effective learning tool
The Feynman method can help you not only to remember new knowledge, but also to really and deeply understand it.
Variable Explorer in Jupyter Notebooks
Extend your Jupyter environment with Notebook Extensions and enable, e.g., the option to explore your currently defined variables in a running Jupyter session.
How DEVONthink’s auto-WikiLink feature changed my Zettelkasten workflow
DEVONthink’s automatic WikiLinks function is a powerful tool, both for discovering connections between notes – expected and unexpected ones – and for the automatized linking of these notes. In this post I briefly explain, how this feature has impacted my Zettelkasten workflow.
DEVONthink Markdown Table-of-Contents generator
I wrote a custom AppleScript for DEVONthink Markdown files, that bypasses the problem of broken links in the auto-generated Table-of-Contents (TOC) of MultiMarkdown (MMD).
Floating Back-to-top button for Markdown documents
You can quickly add a floating Back-to-top button to your Markdown documents in just two steps.
Use your Zettelkasten as a research, thinking and learning tool – Personal knowledge management as a system
In the last part of the series about personal knowledge management, we dive deeper into the Zettelkasten method and demonstrate, how to integrate all parts as an overall system into our research workflow.
Take smart notes with the Zettelkasten method
With the Zettelkasten method by Niklas Luhmann, we give the previously presented personal knowledge network a concrete shape and practical implementation. This is the second of three parts of the series about personal knowledge management.
Don’t take isolated notes, connect them! Vannevar Bush on building a self-organizing network of knowledge
In 1945, Vannevar Bush presented his concept of a self-organizing personal knowledge network by linking informational units with each other. This concept, that would later be known as the Hypertext concept or Hypertext theory, provides the theoretical base of the personal knowledge management system presented in the short series on that topic. This is the first of three parts of that series.
Boost your research with a smart personal knowledge management system
My next posts will be a short series about personal knowledge management and how it can be integrated as a holistic system into our overall research workflow. The system is based on the Hypertext Theory and the Zettelkasten method, and its core element is the personal note-taking process. We go step by step through all parts and see, how we can practically implement them into our daily research work.
Clean Thesis: A simple and elegant LaTeX thesis template
If you’re looking for some inspiration for your thesis, I just came across Clean Thesis by Ricardo Langner, a simple and elegant LaTeX template for thesis documents.
Using Markdown for note-taking
It might be a bit difficult to learn at the beginning, but there are several benefits of taking personal notes in Markdown. Here is why I switched.
Opening a Jupyter notebook from GitHub in Binder: A step-by-step guide
Opening a Jupyter notebook from GitHub in Binder simplifies access to shared code and facilitates seamless collaboration. With just a few steps, you can launch and interact with Jupyter notebooks directly in your browser, without the need for complex setup procedures.
The quickest way to find help for Python commands: The help() command
Python’s built-in help system is probably the fastest way, to quickly look up Python commands and their syntax. It works without leaving your Python environment and is fully offline available.
On teaching
I strongly believe that teaching is not a unidirectional thing, but both sides, the participants and the teacher benefit from it. This is a personal comment on teaching.
My website is now completely cookie-free
I made several changes to my website to further increase the privacy protection. As a result, it runs now completely without cookies.
New Teaching Material: Python Cheat Sheets
I’ve started a collection of various Python cheat sheets that contain some useful and commonly used commands and usage examples.
New Teaching Material: Statistical data analysis and basic time series analysis with Python
I’ve added two new tutorials in the teaching section on statistical data analysis and basic time series analysis with Python.
New Teaching Material: Analyzing IGOR binary files of patch clamp recordings
I’ve added a new tutorial in the teaching section on how to read and process IGOR binary files (ibw) of patch clamp recordings.
Create fancy text styles with Unicode
I found an online font generator to create fancy text styles, simply by using Unicode letters.
New Teaching Material: Fiji short course
There is a new tutorial in the Teaching Material. It’s a short Fiji tutorial on analyzing biomedical image data.
On website subscriptions via RSS and Atom feeds
Personal opinion on how to create and maintain personal news feeds beyond the dependence on big social media and tech companies.
Dealing with future posts in Jekyll
While drafting blog posts in Jekyll, you may want to keep some posts hidden from the public eye until they’re ready to be published. In the world of blogging with Jekyll, there are several effective methods to draft such posts without immediately publishing them. Here are three practical approaches.
Running and testing your Jekyll site locally with custom options
Developing with Jekyll often requires running your site locally to test changes before deploying them live. Here is a handy yet useful one-line command that I usually use to run my Jekyll site locally with custom options.
strftime Cheat Sheet
Cheat Sheet on formatted date and time strings used, e.g., in Python, C/C++ or even on Jekyll websites by using Liquid tags.
Liquid Cheat Sheet
This Cheat Sheet gives an overview of Liquid syntax commands one might encounter while developing a Jekyll website.
Minimal Mistakes Cheat Sheet
A quick overview of available commands for creating content with the Minimal Mistakes Jekyll theme.
New Teaching Material: LaTeX Guide
I’ve added a LaTeX guide to the General Teaching Materials in the teaching section. It serves as a Getting started with LaTeX guide and as a LaTeX glossary.
The Weierstrass function and the beauty of fractals
Fractals are captivating mathematical objects that exhibit intricate patterns and self-similarity at various scales. In this post, we explore the elegance and significance of the Weierstrass function, its relation to fractals and fractal geometry, and discuss other notable fractals. Through this journey, we will discover the fascinating world of fractal geometry and its beautiful and profound impact.
Running a personal website with Jekyll
I have redesigned my website and moved it to a new host as well: I’m running it as personal Jekyll website hosted on GitHub now.
The Lotka-Volterra equations: Modeling predator-prey dynamics
The Lotka-Volterra system, also known as the predator-prey equations, is a mathematical model that describes the interaction between two species: predators and their prey. The system captures the dynamic relationship between the population sizes of predators and prey over time, highlighting the intricate balance between them. In this post we explore this system and calculate its numerical solution using numerical integration Python.
Interactive COVID-19 data exploration with Jupyter notebooks
Amidst the ongoing challenges of the COVID-19 pandemic, I have written a Jupyter notebook that facilitates interactive exploration of COVID-19 data. You can select specific countries and visualize key aspects such as confirmed cases, deaths, and vaccinations. The notebook is openly available on GitHub. Feel free to use and share it.
The SIR model: A mathematical approach to epidemic dynamics
In the wake of the COVID-19 pandemic, epidemiological models have garnered significant attention for their ability to provide insights into the spread and control of infectious diseases. One such model is the SIR model, forming the foundation for studying the dynamics of epidemics. In this blog post, we delve into the details of the SIR model, providing a mathematical description, and showcasing its application through a Python simulation.
The two-body problem
The two-body system is a classical problem in physics. It describes the motion of two massive objects that are influenced by their mutual gravitational attraction. The two-body problem is a special case of the n-body problem, which describes the motion of two objects that are influenced by their mutual gravitational attraction. In this post, we make use of Runge-Kutta methods to solve the according equations of motion and simulate the trajectories of artificial satellites around the Earth.
Solving the Lorenz system using Runge-Kutta methods
In my previous post, I introduced the Runge-Kutta methods for numerically solving ordinary differential equations (ODEs), that are challenging to solve analytically. In this post, we apply the Runge-Kutta methods to solve the Lorenz system. The Lorenz system is a set of differential equations known for its chaotic behavior and non-linear dynamics. By utilizing the Runge-Kutta methods, we can effectively simulate and analyze the intricate dynamics of this system.
Runge-Kutta methods for solving ODEs
In physics and computational mathematics, numerical methods for solving ordinary differential equations (ODEs) are of central importance. Among these, the family of Runge-Kutta methods stands out due to its versatility and robustness. In this post we compare the first four orders of the Runge-Kutta methods, namely RK1 (Euler’s method), RK2, RK3, and RK4.
Earth’s dipolar magnetic field
In physics and computational mathematics, numerical methods for solving ordinary differential equations (ODEs) are of central importance. Among these, the family of Runge-Kutta methods stands out due to its versatility and robustness. In this post we compare the first four orders of the Runge-Kutta methods, namely RK1 (Euler’s method), RK2, RK3, and RK4.
Restarting my website
In the wake of the COVID-19 pandemic, I have made the decision to relaunch my website. While I have previously utilized my website for smaller personal projects and showcasing my photographs, I now intend to broaden its scope. I will be posting on a range of topics including physics, neuroscience, data science, machine learning, artificial intelligence, open-source projects, and more. As a result, I will be revamping the website in the upcoming months. Stay tuned for the updates.
Posts from 2013 to 2020 moved to the archive
I just cleaned up my website and put a lot of old stuff from 2013 to 2020 into the archive.