How to do dynamic matplotlib plotting with a fixed pandas dataframe?

Question

I have a dataframe called benchmark_returns and strategy_returns. Both have the same timespan. I want to find a way to plot the datapoints in a nice animation style so that it shows all the points loading in gradually. I am aware that there is a matplotlib.animation.FuncAnimation(), however this typically is only used for a real-time updating of csv files etc but in my case I know all the data I want to use.

I have also tried using the crude plt.pause(0.01) method, however this drastically slows down as the number of points get plotted.

Here is my code so far

x = benchmark_returns.index
y = benchmark_returns['Crypto 30'] 
y2 = benchmark_returns['Dow Jones 30']
y3 = benchmark_returns['NASDAQ'] 
y4 = benchmark_returns['S&P 500']


fig, ax = plt.subplots()
line, = ax.plot(x, y, color='k')
line2, = ax.plot(x, y2, color = 'b')
line3, = ax.plot(x, y3, color = 'r')
line4, = ax.plot(x, y4, color = 'g')

def update(num, x, y, y2, y3, y4, line): 
    line.set_data(x[:num], y[:num])
    line2.set_data(x[:num], y2[:num])
    line3.set_data(x[:num], y3[:num])
    line4.set_data(x[:num], y4[:num])

    return line, line2, line3, line4,

ani = animation.FuncAnimation(fig, update, fargs=[x, y, y2, y3, y4, line], 
                              interval = 1, blit = True)
plt.show()

William Miller · Accepted Answer · 2020-03-10 17:40:47Z

1

You could try matplotlib.animation.ArtistAnimation. It operates similar to FuncAnimation in that you can specify the frame interval, looping behavior, etc, but all the plotting is done at once, before the animation step. Here is an example

import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
from matplotlib.animation import ArtistAnimation

n = 150
x = np.linspace(0, np.pi*4, n)
df = pd.DataFrame({'cos(x)' : np.cos(x), 
                   'sin(x)' : np.sin(x),
                   'tan(x)' : np.tan(x),
                   'sin(cos(x))' : np.sin(np.cos(x))})

fig, axs = plt.subplots(nrows=2, ncols=2, figsize=(10,10))
lines = []
artists = [[]]
for ax, col in zip(axs.flatten(), df.columns.values):
    lines.append(ax.plot(df[col])[0])
    artists.append(lines.copy())

anim = ArtistAnimation(fig, artists, interval=500, repeat_delay=1000)

The drawback here is that each artist is either drawn or not, i.e. you can't draw only part of a Line2D object without doing clipping. If this is not compatible with your use case then you can try using FuncAnimation with blit=True and chunking the data to be plotted each time as well as using set_data() instead of clearing and redrawing on every iteration. An example of this using the same data from above:

import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
from matplotlib.animation import FuncAnimation

n = 500
nf = 100
x = np.linspace(0, np.pi*4, n)
df = pd.DataFrame({'cos(x)' : np.cos(x), 
                   'sin(x)' : np.sin(x),
                   'tan(x)' : np.tan(x),
                   'sin(cos(x))' : np.sin(np.cos(x))})

fig, axs = plt.subplots(2, 2, figsize=(5,5), dpi=50)
lines = []
for ax, col in zip(axs.flatten(), df.columns):
    lines.append(ax.plot([], lw=0.5)[0])
    ax.set_xlim(x[0] - x[-1]*0.05, x[-1]*1.05)
    ax.set_ylim([min(df[col].values)*1.05, max(df[col].values)*1.05])
    ax.tick_params(labelbottom=False, bottom=False, left=False, labelleft=False)
plt.subplots_adjust(hspace=0, wspace=0, left=0.02, right=0.98, bottom=0.02, top=0.98)
plt.margins(1, 1)
c = int(n / nf)
def animate(i):
    if (i != nf - 1):
        for line, col in zip(lines, df.columns):
            line.set_data(x[:(i+1)*c], df[col].values[:(i+1)*c])
    else:
        for line, col in zip(lines, df.columns):
            line.set_data(x, df[col].values)        
    return lines

anim = FuncAnimation(fig, animate, interval=2000/nf, frames=nf, blit=True)

Edit

In response to the comments, here is the implementation of a chunking scheme using the updated code in the question:

x = benchmark_returns.index
y = benchmark_returns['Crypto 30'] 
y2 = benchmark_returns['Dow Jones 30']
y3 = benchmark_returns['NASDAQ'] 
y4 = benchmark_returns['S&P 500']

line, = ax.plot(x, y, color='k')
line2, = ax.plot(x, y2, color = 'b')
line3, = ax.plot(x, y3, color = 'r')
line4, = ax.plot(x, y4, color = 'g')

n = len(x)  # Total number of rows
c = 50      # Chunk size
def update(num):
    end = num * c if num * c < n else n - 1
    line.set_data(x[:end], y[:end])
    line2.set_data(x[:end], y2[:end])
    line3.set_data(x[:end], y3[:end])
    line4.set_data(x[:end], y4[:end])

    return line, line2, line3, line4,

ani = animation.FuncAnimation(fig, update, interval = c, blit = True)
plt.show()

or, more succinctly

cols = benchmark_returns.columns.values
# or, for only a subset of the columns
# cols = ['Crypto 30', 'Dow Jones 30', 'NASDAQ', 'S&P 500']
colors = ['k', 'b', 'r', 'g']
lines = []
for c, col in zip(cols, colors):
    lines.append(ax.plot(benchmark_returns.index, benchmark_returns[col].values, c=c)[0])

n = len(benchmark_returns.index)
c = 50  # Chunk size
def update(num):
    end = num * c if num * c < n else n - 1
    for line, col in zip(lines, cols):
        line.set_data(benchmark_returns.index, benchmark_returns[col].values[:end])

    return lines

anim = animation.FuncAnimation(fig, update, interval = c, blit=True)
plt.show()

and if you need it to stop updating after a certain time simply set the frames argument and repeat=False in FuncAnimation().

edited Mar 10, 2020 at 17:40

answered Mar 10, 2020 at 2:54

William Miller

10.4k4 gold badges31 silver badges50 bronze badges

Sign up to request clarification or add additional context in comments.

18 Comments

Hamish Gibson Over a year ago

I have tried this however the runtime isn't the best. It appears that as the data scales the fps drops catastrophically.

William Miller Over a year ago

@HamishGibson That is to some extent inevitable, of course there are ways to mitigate it (using blit, using set_data() instead of clearing and replotting, etc) but still at a certain scale it becomes an issue of hardware regardless. The only real way to guarantee an FPS which is unaffected by data size is to save the animation instead of displaying it live.

Hamish Gibson Over a year ago

I have managed to get a solution working with set_data(). How would I use clear in this case however?

William Miller Over a year ago

@HamishGibson I'm not sure I understand your question - when I said 'clearing and replotting' I was referring to the use of ax.clear and plt.plot instead of set_data..... is that what you're asking about?

William Miller Over a year ago

@HamishGibson Glad to help, don't forget to mark the answer as accepted so the question is marked as such for future users.

|

Sameeresque · Accepted Answer · 2020-03-10 01:44:11Z

1

You can just update the data into the line element like so:

fig = plt.figure()
ax = fig.add_subplot(111)
liner, = ax.plot()
plt.ion()
plt.show()
for i in range(len(benchmark_returns.values)):
    liner.set_ydata(benchmark_returns['Crypto 30'][:i])
    liner.set_xdata(benchmark_returns.index[:i])
    plt.pause(0.01)

answered Mar 10, 2020 at 1:44

Sameeresque

2,6022 gold badges14 silver badges24 bronze badges

4 Comments

Hamish Gibson Over a year ago

I got a ``` File "results.py", line 91, in <module> liner, = ax.plot() ValueError: not enough values to unpack (expected 1, got 0)``` error instead

Hamish Gibson Over a year ago

I managed to fix the error but the runtime is horrendous, any tips on how to fix it?

Sameeresque Over a year ago

If you can use NumPy arrays as inputs rather than pandas series, you could see potential improvement in the runtime.

Sameeresque Over a year ago

You might find this SO helpful. There are other ways that offer faster times than matplotlib.

Collectives™ on Stack Overflow

How to do dynamic matplotlib plotting with a fixed pandas dataframe?

2 Answers 2

Edit

18 Comments

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Edit

18 Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related