Fractals and Brains (Part 2): Our fractal Perceptions

This is the second part of my article about fractals and brains. The focus of this part now is again our brain but this time not so much its physical structure and its cytoarchitecture but rather how our brains exploit fractal processes that generate our experiences.

Please note: Due to the positive feedback to the first part of this article (and hopefully to this second one here as well) I have decided to publish one more part (Part 3). The Part 3 will deal with the interesting question of how smart machines can become fractal generators that reproduce and improve themselves and how this might help to solve the problem of how to crack the neural code !

I hope you will enjoy this overview series about Fractals and Brains. Let me know any feedback you may have in the comments !

Fractals in Time

In the first part of this article I have discussed fractal spatial structures in nature, biology, our bodies, our brains and in science in general. However, fractals don't only occur as objects in space like the Mandelbrot set or the tissues and cells of our brains, they also exist in time as fractal processes ! This is especially important and relevant when we want to understand better and describe how we experience something (which happens in time) and how our brains and minds work as dynamic and complex systems in time.

It was again Benoit Mandelbrot who first investigated and studied seriously the fractal properties and characteristics of dynamic systems and the resulting time-series - especially time-series as we find them in the financial world and financial markets like stock- or commodity prices and their fluctuations over time. In such time-series usually the fractals are not as easy to recognise as in physical fractal structures of concrete objects in space. Nevertheless, fractal features and properties like the fractal dimension of financial time-series for example can equally well be detected and measured and used for the analysis of the underlying series as well as for forecasting purposes or for example for the analysis of optimised financial portfolios.

Besides calculating the fractal dimension of various financial time-series Mandelbrot also studied mechanisms and algorithms of how to construct fractals that can approximate the fluctuations of given time-series.

To see how this is possible, consider the following approach: first, the time series graph gets roughly approximated by simple straight line segments (see first line of the chart below). Then these line segments themselves get replaced in iterative steps with simple fractal segments (generators) which then get replaced again with copies of themselves on smaller scales as shown in the two parallel charts below.

It is important to notice, that to make this approximation process work, the fractal generators can usually not be chosen to be identical. That would often be too simple for most real time-series. The fractal segments may need to be squeezed or stretched according to the relative length of the line segment they need to approximate.

Also, notice that the fractal parts usually need to flip between an "up" and "down" section to reflect the frequent up and down swings of the real financial data. Finally, the fractal parts usually also need to be swapped upside down to reflect reversing segment trends. The generators often also cannot be 100% deterministic but may need to include some kind of noise or random factors during the iteration process.

Time-series can not only be approximated by fractals but they can also easily be constructed from scratch by re-iterating simple self-similar line segments. See the picture below how for example a typical standard technical chart analysis formation like a "W" or "head-and-shoulders" formation can be created by a simple fractal process after a few iterations.

In classical financial technical chart analysis the detection of such fractal formations in the time-series can sometimes be used as an indicator for buy or sell signals. When a recurring fractal is detected at a certain scale of the time-series its mid point is often used as the centre of the fractal from where the probable direction/trend (or direction change) of the time series is projected. However, this approach has its limitations for practical use as time-series often show many such fractal sub-segments with potentially resulting contradictory trend indicators.

Fractal Body Signals

We do not want to go more into depth about fractals to analyse financial time-series. This would be worth a separate series of articles following the work of Mandelbrot and many others since. We rather will focus here on fractal time-series representing and generating bio signals emanating from our bodies and especially our brains.

It is well known today, that many of the vital signs of our bodies can be represented and measured as graphs of time-series that are often self-similar and oscillatory on certain scales with more or less frequent and important recurring fractal patterns (see pic below). As a matter of fact, a strong discrepancy from the normal bio signal structures (say for example compared to the average fractal dimension of different people) or even the complete absence of such repetitive fractal patterns in crucial bio signals usually indicates trouble and potential or actual severe medical problems for a person or patient (see below some typical vital signs and their self-similar structure over time).

In the strongly expanding and very interesting brain-computer interface technology (BCI) sector the analysis of bio signals is crucial for the success and the working of the BCI devices. Therefore many bio and body signals are measured with (new) wearable sensors and scientists are trying to enhance the communication between humans and computers by collecting and interpreting such multiple body signals possibly in real time (see pic below).

EEG Signals - and what they can tell us (and what not)

The most important and relevant of the body generated bio signals related to our brains are the measurable electrical (and magnetic) waves emanating inside our brains from the electrical communication of our nerve cells. There are many methods available to measure our brain activities and electro-magnetic brain waves while we are alive and awake and even while we are sleeping or are anaesthetised.

Various invasive methods can be used that would however require surgery and the opening of the scull. This is for example the way Elon Musk's firm Neuralink works. However, there are also many safer non-invasive methods to measure our brain activities that do not require any surgery like: EEG (electro-encephalography) or fMRI (functional magnetic resonance imaging) as well as MEG (magneto-encephalography), fNIRS (functional near infrared spectrography) and several more (see pic below).

For this article, we will only discuss non invasive measurement methods like the classical EEG which is - even though developed already over 100 years ago - still by far the most commonly used method today for non invasively studying and monitoring brain activities in living humans. Another major advantage of EEG measurements is that they can now easily be done with light, safe and relatively low cost wearable and even wireless devices and head caps (see below). This is why EEG is a key component in most low cost brain-computer interface devices these days (although other systems like fNIRS and systems with combined methods like EEG, MEG and fNIRS are catching up quickly).

Reading and interpreting an EEG signal reliably however is difficult and requires months if not years of medical, clinical training. This becomes evident by the fact, that even an experienced doctor can usually not "read" the signal if he/she does not know the exact time window of the measurement and the exact location of the electrodes on the skull from which the signal has been measured (see a typical EEG chart below for several EEG channels/locations and their sensors as horizontal signal lines). To automatically read and correctly interpret EEG signals by computers is therefore even more difficult. Key problems are the often complex and dynamically varying fractal structure of the EEG signals, their often very bad signal-to-noise ratio, the low spatial resolution of the EEG sensors and many other issues.

Each of these lines above could represent just a few seconds of recording, a few minutes, a few hours or even a few days. There is no way to tell the measured time scale by just looking at the EEG signal itself. This is a clear indicator of the fractal character of these signals. They are self-similar on practically all relevant time scales.

Nevertheless, all these EEG signals can be de-composed and classified into combinations of several different, standardised frequency bands that can be recorded and filtered out from the signals from patients dependent on their mental states (alert, asleep, or performing active conscious activities etc). The typical and most commonly measured and used frequency bands are shown below:

Now here is where it really gets even more complicated. The electrodes on the scalp cannot measure clear, single neuron signals. The sensors are placed on the scalp in a regular mesh and can have a varying number. Simple EEG systems just use a few sensors and better ones use 16, 32, 64 or 128 sensors and very high level ones may use some 256 or 512 sensors (see scheme below).

Once the mesh of electrodes is put on the scalp a so called "montage" has to be defined. The EEG measures not the absolute value of voltage at each electrode but the voltage difference of the detected voltage levels of one electrode compared with a second one. In the early days the measured difference was calculated fix between the two nearest neighbours of electrodes. However, in modern EEG systems the connection of the electrodes and measurement mesh can be defined logically and is not restricted to the hardware structure and the physical alignment of the electrodes anymore. This has some major advantages for practical purposes as one can now easily measure directly the activity differences between any two brain regions underneath 2 different electrodes.

The scheme below shows some of the many possible logical arrangements of the electrodes given the same physical arrangement of the electrodes on the scalp. The best montage depends on the problems to be studied and issues to be diagnosed or treated.

Given a concrete (logical or physical) montage of the electrodes one can then record the changing time varying activities and dominant frequencies of the various brain regions over time and study how these activities change and develop across the brain. This can be crucial and important for detecting brain diseases and malfunctions as well as for the analysis of cognitive processes and where they mostly happen in the brain. When colour coding the intensity and frequency ranges of the signals one can hence obtain interesting dynamic maps of living brains as shown below:

There are still many further problems related to EEG measurements of the brain activities and the detection and especially the localisation of the origin of the electrical brain waves. For example, each electrode sensor detects and records signals from many neurons simultaneously. That is a general and major problem for any EEG measurement (and correlates with the chosen montage and number of electrodes). The effect is that the spatial resolution of the measured signals is in general very low (contrary to the time resolution as the EEG signals can be recorded and filtered in a few milliseconds and hence in near real time).

The EEG signal is hence measurable much closer to real time than say an fMRI signal because the fMRI signals do not actually measure the electric action or field potential of the neurons but rather just the oxygen level changes of the bloodflow (BOLD signal) in the vicinity of the targeted neurons. The fMRI signals hence have a substantial time delay that can easily be in the range of seconds and they need complex calculations before they can be visualised, however, the fMRI usually has a much better spatial resolution than the EEG).

The electrical wave signals measured are usually the combined signal of a few neurons but from usually at least tens of thousands of neurones or even millions of neurones. These signals therefore don't just add up in the EEG signal, rather the opposite. The individual signals from the neurons or smaller sub-groups of neurons can have different frequencies, amplitudes and phases and hence they can cancel each other out or they can at least strongly interfere with each other.

This is comparable to a situation where you throw a single stone in a quiet pond of water. This will generate a nice circular outward moving wave on the surface of the water. However, if you move in 2 stones at different places of the pond then the resulting 2 waves will already interfere and if you throw in a hundred stones at several different places of the pond there will not be any detectable circular outward moving waveforms anymore. The surface will look chaotic like the surface of the ocean on a normal day. The waves will be coming from all sides and potentially with all kinds of amplitudes.

The same happens with the measured EEG signals on the top of the scull. The effect of the massive interference generated by the many neurons not only distorts the resulting wave forms but also makes it nearly impossible to reconstruct and find out where the signals were actually generated inside the brain because only the combined signals can be measured (see the scheme below that visualises the complex effect of just 2 neurons simultaneously signalling in the brain (red and green signals) and the resulting combined and strongly distorted signals (blue) measured at different locations of the scalp).

How complex these interference effects can get is visualised in the animation below which just shows two very simple sine curves interfering with each other (red and green waves). The resulting signal (blue wave) is a complex non linear combination of the two signals generating a clear fractal oscillation.

The complex interference effect shown above can however also cause a so called coupling effect, i.e. the signal components align over time and combine with each other. Such couplings of signals and their components can be very important and are not necessarily a negative effect. As a matter of fact, such couplings of say the amplitude of one signal with the phase of another signal can be crucial for transferring signals within the brain from one region and task to another (which we will discuss further down below). Various complex (fractal) coupling effects of signal components are shown here:

How to find the base frequencies or components of such a complex combined signal is a difficult area of science/math in itself. A standard way for filtering and decomposing such complex signals is to use the so called FTT algorithm (Fast Fourier Transformation) that decomposes a complex signal in its underlying frequency spectrum and self-similar frequency bands (see pictures below).

Here a simple recursive pseudo code for such a FFT:

We won't go further into this here as the underlying required math is a bit tricky.

It is well worth noticing here that the applicability of Fourier or Fast Fourier Transformations to a (bio) signal actually proves and means that all such signals are in principle fractal in character because they can be de-composed into a series of sine and cosine waves using the Fourier or FF Transformation into self-similar sine/cosine waves on different scales ! That again shows how important, fundamental and ubiquitous fractals are in our world.

Looking at a Fractal causes fractal Brain Waves

Now, it is a very interesting and complicated question to discuss how the outside world and our inner brain processes (as they can be measured with an EEG or other methods) correlate with each other. Does the outside world cause and generate the inner brain waves when we experience something ? Or is it rather the other way around that our inner brain processes exist first and the sensor organs just map and project outside stimuli onto already existing complex, fractal brain wave patterns and neural oscillations ? Or is it a combination of both ? Let's see if we can answer this "chicken-or-egg" type of question at least in principle.

The intuitive view and opinion held still today by most brain researchers and neuroscientists is that the physical outside world stimulates our senses and thereby causes and activates the resulting internal brain waves that we can then measure with EEG or other methods. The process responsible for this is the so called "transduction" which converts external signals into the internal electro-chemical cascades of the neurons in our brains.

Every single one of our senses has specialised receptor cells that perform this transduction process (see simplified chart of these receptors below):

The transduction process for our auditory hearing sense for example can be schematically described and summarised in the following diagram

The actual concrete auditory transduction is - like all transduction processes - quite complex in itself and involves very intricate bio-physical processes that convert the received sound waves into the electric action and local field potential waves in neurons and their immediate environments as shown below:

A recent study (however not with sound but with using our visual sense and fractal images and animations instead of sounds) seems to confirm the causation of our internal brain waves through external stimuli. This was quite a spectacular finding.

The experimental set-up used people who were shown different pictures and animations of fractals with known fractal dimensions similar to the ones we have seen in the first part of this article. While doing so the EEG of these individuals were measured. The results were more than astonishing (assuming no substantial methodological errors were made). The measured EEG brainwaves caused by looking at the fractal animations were highly correlated with the shown fractal features and even with the fractal dimensions of the presented fractal images ! The gist of the experiment is outlined in the chart below:

This seems to decide the "chicken-or-egg" question raised above. With this experiment it seems obvious that the outside world and the stimuli of our senses cause and generate our internal brain waves and their fractal features.

Cerebral Organoids

Not so fast ! Over the last 10 years or so a fascinating and powerful new cell culture and tissue technology has been developed. Scientists are now able to grow and nurture (human and/or animal) brain tissue outside our bodies in a Petri dish of a bio lab. This is done in ethical ways by not using embryonic human stem cells but rather by re-programming normal skin cells of adult humans and transform them back into pluripotent stem cells which can then be differentiated into various nerve cell types. The resulting tissues are called "cerebral organoids" or sometimes more sloppy just "mini brains" (although this is quite mis-leading as these tissues are still very far away from being anything like "brains").

The growth of nerve tissue outside the body has actually been possible and done for decades already, so far however only as flat layers of brain tissue. Nowadays neuroscientists and cell biologists can also nurture and grow compact 3-D spheres of brain tissue (so far with diameters of just a few millimetres containing somewhere between 100,000 to around 2 - 3 million nerve cells). These cerebral spheric organoids can then be used to analyse their resulting cytoarchitecture and growth patterns and provide high quality tissue for testing medicines etc.

The 3-D shape of these tiny organoids is actually very important as the nerve tissue organises itself while it grows like in an embryo in a 3-D sphere (as we have shown in Part 1). The tissue needs the 3-D shape cytoarchitecture to be able to grow the desired tissue that resembles more human brains than a 2-D layer of brain tissue could generate.

See below some stages of such 3-D spheres of cerebral organoids as compared to the development of embryonic brain tissue. In the second picture below see a photo of real organoids.

Now, some neuroscientists like Prof Alysson Muotri of the University of California in San Diego, who's team studies cerebral organoids to find out, among other things, the causes of Autism, have measured the EEG signals of such cerebral organoids and something very astonishing happened. After just a few weeks of maturation in the Petri dish the organoids suddenly start generating electrical signals and waves measurable by EEG that are very similar to EEG signals as they can be measured in early stage human embryo brains ! The chart below shows the set-up of this work on the EEG of the cerebral organoids. In the middle row the increased complexity of the measured EEG signals of the organoids over time can be seen:

The EEG signals suddenly start to be detectable after a few weeks and months of maturation of the organoids and show a simple synchronised pattern all over the cellular sphere. During the next few months however, these signals get more and more complex and diverse while the cell spheres mature. After about 6 to 8 months the organoids show a similar structure and complexity of the EEG signal as they are generated by human embryonic brains. It also seems that the complexity of the EEG signals increases with the percentage of mature Glial cells generated over time inside the organoidal spheres. This is an interesting future research field.

The emergence of complex EEG signals in the organoids with maturity is indeed very surprising and a major discovery as the organoids - in contrast to the human embryos - have no senses or sense organs, not even immature ones. The organoids have no eyes, no ears, no tongue, no nose etc. They are far away from the developmental complexity and stage where human embryos develop rudimentary sensory organs that could trigger and cause our brainwaves. The brain waves in the organoids just suddenly ignite at a certain stage of their development like the heart in an embryo starts suddenly beating after a few weeks gestation and never stops anymore until we die !

So, the spontaneous generation of clearly measurable EEG waves already in such simple organoids with only a few hundred thousands of nerve cells (compared to the many billions that grow in an embryonic brain during gestation) falsifies the theory that the sensory stimulation of the outside world generates our internal brainwaves !

The chicken-or-egg question is resolved at least in the sense that it has become a "chicken-AND-egg" solution where the brain waves come first, the sensory stimulation and their effects come second but both interact at some stage of maturity (when the sense organs have developed) and are needed for our general perception.

It seems that the fractal brain waves that occur before any experience of outside stimuli can happen in our brains indicate that our experiences are the result of the maturation of the cells in combination with a complex self-organisation processes and the reaching of some critical complexity threshold which triggers a kind of phase transition in the behaviour of the organoidal cells. The criticality threshold may be linked to the percentage and function of the maturing glial cells in the brain, not just of the neurons. Once this threshold is exceeded the maturation of the brain cells cause more and more complex fractal oscillations and detectable brain waves.

The Role of Fractals in the Perception of Sound, Music and Language

Let's see if we can now better understand how the outside stimulation of our senses and the pre-existing inner fractal brain waves work together to produce our (conscious) experiences. Due to space limitations we will focus here just on our hearing and auditory experiences. Little spoiler alert: we will find again a major role and function of various fractal processes in these perceptions on many levels.

There is indeed an obvious close connection between the physical sound waves and patterns we receive through our ears and the fractal brain processes present in the brain when we have an auditory experiences. As a matter of fact a so called "entrainment" process happens that causes our brain waves to somehow mirror the frequencies of external sound waves. We can see this especially well when we consider two important and closely link sound phenomena: music and language. This entrainment effect is mainly responsible for your rhythmic tapping of your feet or fingers or your shaking of your head when you listen to music that you like !

Music as Fractals

After all we have discussed about fractals so far it is not a surprise that music itself often also has fractal features or components. Every pop song for example usually contains at least one simple self-similar fractal in time, a so called "refrain" as the core and climax of the song that repeated over and over and is often the best remembered part of a song and therefore easy to sing along. More complex songs contain more fractal self-similar pieces and more variations of melodic fractals.

As a matter of fact, it is easy to generate music that is a pure fractal composed fully and only out of repetitive self-similar or identical pieces on different scales (in terms of frequency, rhythm and pitch). A nice example of such a pure fractal music composition that has a similar structure and composition process as the famous fractal Koch curve (that we saw already in the first part of this article). Its iterative and decreasingly smaller self-similar fractals are demonstrated in the animation below with the resulting sound and melody. Watch several iterations of the fractal music process in the video below:

Johann Sebastian Bach was an absolute genius and master in generating complex musical pieces that were based on various melodic fractals. Surely, his compositions were much, much more complex than the example fractal music given above. Bach was able to compose and construct like nobody else his famous fugues as a sequence of highly intertwined self-similar fractal variations of a few basic musical themes by modifying and morphing the themes in all kind of ways. He for example turned a base tonal sequence literally upside down, reversed it or stretched them and then re-combined them all again in beautiful harmonious chords by playing them all together at the same time.

The resulting intricate melodies of this fractal composition technique produced some of the best and most famous classical compositions any human has ever created. This can be seen for example in the impressive video visualisation of his famous organ piece :"Toccata and Fugue in D minor" below.

Since this fugue (and many others he has composed) has a complicated internal fractal structure it is very helpful to look at the visualisation of the video while listening to the fugue. It is much easier to detect the patterns and to follow the many shorter and longer fractal melody pieces visually than to try to detecting the fractal pieces just by listening to the melody he composed as many of the fractals are played concurrently in identical form or slightly varied. Probably only experienced musicians would be able to identify and detect all the many short fractal segments just by listening to this great composition. Enjoy:

Music, Language and our Perceptions as fractal Processes

We humans are different from all higher primates and any other animal species by having developed a highly sophisticated language. No other species has developed anything like our complex modern languages over the course of their evolution. Everybody knows this.

What is much less known and appreciated is that humans are also the only species that has developed music and a sophisticated musical culture. Even the most primitive human tribes have developed music and their own instruments tens of thousand years ago already. We humans sing, dance and enjoy musical performances like no other species - not to mention the highly intellectual achievements of classical and modern musical compositions like Beethoven's 5th Symphony for example.

As a matter of fact the development of language and music are closely linked. It is unclear what has come first, our ability to speak or our ability to make music and sing. It seems that both skill-sets are tightly linked and have co-evolved with each other and with the development of the vocal tract and parts of our neocortex that allowed for the fine controlled movements needed for both skills (and probably with a re-assignment of functional circuits in our cerebellum).

How closely our musical perception and our brain waves are linked is shown below. The two charts show two different music scores, their respective power spectrum (red line) and the smoothened EEG measured brain waves resulting from listening to the music scores (blue line). The matching of the lines is not perfect but close. The variations of the curves can be explained mostly by noise (and for example the attention level of the listener etc).

The match of the brain wave characteristics and the sound wave features reaching our ears becomes even more obvious and clear if one looks at just an evoked potential response of a simple tone with a varying pitch or frequency and the resulting onset of the corresponding brain wave and its oscillation, there is a near complete similarity and match.

When one measures a spoken language sound wave a similar and remarkable effect happens. The extracted envelope of the sound wave (dark line around the grey scale sound wave) correlates strongly with the measured EEG signal (red line) as shown below:

Therefore, the entrainment of brain waves from sound waves is well established. Our senses act like band filters on the incoming wide spectrum of signals. Due to their limited sensitivity they can only let through a certain range of signals. For our ears these are sound waves with a frequency band between around 20 Hz and 20,000 Hz. Everything below or above this range we simply cannot hear. The same applies to all other senses. Our eyes for example can only see light with a wavelength in a range between 380 - 750 nanometers for an average healthy individual. The frequencies of visible light is therefore in the range of around 400 to 800 Terra Hz !

This example of the frequencies of visible light shows that the transduction process is key and essential here. Our nerve cells could never act in that THz range (1 Hz is 1 oscillation per second). Our neurons can only process frequencies in the range form 0 to around 1,000 Hz ! So, compared to light and many other outside processes our brain is very slow. The transduction processes take care of this and map the external signal features that stimulate our senses into a range that our neurones can handle and process (they slow them down or speed them up or they dim them down or enhance them).

So, this gives us the missing link and idea to solving the problem how our brains construct and process our experiences.

The brain comes with its own range and set of self-similar fractal brain waves and oscillations prior to any sensual experiences. These are usually relatively slow waves (on average somewhere in the measurable range between 0 and around 100 Hz - in exceptional cases neuronal assembly frequencies can be measured up to 1000 Hz).

These basic and usually slow brain waves can only carry very little information, however. If they need to transfer more complex information they need to couple with higher frequency waves and add them up together so the resulting wave can encode more information. A slow brainwave with no coupled fast waves on top of them is like a big container ship that carries no containers - a waste of resources !

The slow brain waves act like big container ships. They are slow but they can take on "payloads", i.e. they can take on containers to carry. The slow waves can take on other waves of higher frequencies like a container ship can be loaded with smaller containers. Each container itself can then contain further even smaller containers etc. In this way the slow waves get their self-similar fractal properties which are used to communicate efficiently complex, highly informative frequency patterns across the brain over relatively long distances. Put in another visual paradigm: the slow waves are like big Russian dolls that contain other smaller Russian dolls which themselves again contain even smaller Russian dolls and so on (i.e. slow frequency waves carry higher frequency waves which can themselves also carry even more higher frequency waves) !

The process responsible for this is the coupling effect which we have already discussed earlier. As said, coupling is often an undesired side effect of interfering oscillations but here in our brains they make our perception possible and much more efficient.

The charts below show wave couplings that combine the phase of wave with the amplitude of another wave or the frequency of waves with higher frequencies etc:

To wrap up this discussion and this Part 2 of the article here a final chart that extends this fractal view of our sensory perception to groups of neurons rather than single neuron couplings which is what actually happens in the brain. We discussed in the first part of this article that the vertical cortical columns are the real major processing units of the neocortex. Hence we always need to consider large groups of neurons or better groups of vertical columns of neurons when describing our brain processes.

The final chart below summarises the situation in the brain well. The boxes A - J show various distributed neuron assemblies and coupling steps that generate the associated distributed, connected and interwoven, self-similar neuronal assembly networks that facilitate our perception processes.

E.Schoneburg

Berlin, July 20201