Top k motif profiles are not distinct

[GChims]

Hi Sean,

I am trying to follow your tutorial here: https://stumpy.readthedocs.io/en/latest/Tutorial_STUMPY_Basics.html

I generate the top 10 pairs of subsequent neighbours below (pair _motifs), and try to plot them. However, I don't get distinct motifs and most of the generated motif plots are essentially the same. Is this problem related to the trivial match? How can I fix this so that I dont get redundant motifs (please the attached screenshots).

#Find a Motif Using STUMP
mp = stumpy.stump(df['value'], m)

motif_idx = np.argsort(mp[:, 0])[0]
top_k_idx = np.argsort(mp[:, 0])[:10]
motifs_k = list(top_k_idx)

#extract pairs of subsequent neighbours
def motif_pairs(motifs_k):
    it = iter(motifs_k)
    return list(zip(it,it))

pair_motifs = motif_pairs(motifs_k)

print(f"The motif is located at index {motif_idx}")
print('distance between the motifs: ', mp[motif_idx, 0])
nearest_neighbor_idx = mp[motif_idx, 1]
print(f"The nearest neighbor is located at index {nearest_neighbor_idx}")

fig, axs = plt.subplots(2, figsize = (20,6), sharex=True, gridspec_kw={'hspace': 0})
plt.suptitle('Motif (Pattern) Discovery', fontsize='15')

axs[0].plot(df['value'].values)
axs[0].set_ylabel('Time series', fontsize='15') 
rect = Rectangle((motif_idx, 0), m, 40, facecolor='lightgrey')
axs[0].add_patch(rect)
rect = Rectangle((nearest_neighbor_idx, 0), m, 40, facecolor='lightgrey')
axs[0].add_patch(rect)
axs[1].set_xlabel('Time', fontsize ='15')
axs[1].set_ylabel('Matrix Profile', fontsize='15')
axs[1].axvline(x=motif_idx, linestyle="dashed")
axs[1].axvline(x=nearest_neighbor_idx, linestyle="dashed")
axs[1].plot(mp[:, 0])
plt.show()


#Plotting the top k motif for each pair of subsequent neighbours
for i, j in pair_motifs:
    fig, axs = plt.subplots(2, figsize = (20,6))
    
    plt.suptitle('Motif - time series', fontsize='15') 
    axs[0].set_ylabel("Time series", fontsize='15') 
    axs[0].plot(df['value'], alpha=0.5, linewidth=1)
    axs[0].plot(df['value'].iloc[i:i+m])
    axs[0].plot(df['value'].iloc[j:j+m])
    rect = Rectangle((i, 0), m, 40, facecolor='lightgrey')
    axs[0].add_patch(rect)
    rect = Rectangle((j, 0), m, 40, facecolor='lightgrey')
    axs[0].add_patch(rect)
    axs[1].set_xlabel("Time", fontsize='15')
    axs[1].set_ylabel("Time series", fontsize='15')
    axs[1].plot(df['value'].values[i:i+m], color='C1')
    axs[1].plot(df['value'].values[j:j+m], color='C2')
plt.show()

[seanlaw]

@GChims Note that top_k_idx = np.argsort(mp[:, 0])[:10] doesn't actually give you the top-K motifs. I recommend taking a look at the stumpy.motifs function, which might help in some cases to find the top-K motifs

https://stumpy.readthedocs.io/en/latest/api.html#stumpy.motifs

Please note that this is a helper function and makes certain (basic/default) assumptions for motif discovery that may not be compatible with your data. It is essentially a nice wrapper around the stumpy.match function, which will give you more control for finding matches for a given pattern:

https://stumpy.readthedocs.io/en/latest/api.html#stumpy.match

I will try to add a note to the tutorial.

[GChims]

Thanks Sean,

I read the Find Top-K Motifs details here: https://stumpy.readthedocs.io/en/latest/Tutorial_STUMPY_Basics.html and yes upon reading again, I understand that top_10_motifs_idx = np.argsort(mp[:, 0])[10] doesn’t actually give you the motifs. I will look further at the stumpy.motifs and stumpy.match as suggested.