In recent years, deep learning has redefined algorithms for detecting cardiac abnormalities. However, many state of the art algorithms still rely on calculating handcrafted features from a given heart signal that are then fed into shallow 1D convolutional networks or transformer architectures. We propose ACQuA (Anomaly Classification with Quasi Attractors), a task agnostic algorithm that can be used in a wide variety of cardiac settings, from classifying cardiac arrhythmias from ECG signals to detecting heart murmurs from PCG signals. Using theorems from dynamical analysis and topological data analysis, we create informative attractor images that 1) are human distinguishable and 2) can be used to train small, off the shelf deep neural networks for anomaly classification. We receive an official challenge score of 0.433 (263/305) for murmur classification and a score of 12616 (208/305) for outcome classification. In addition to competing in the George B. Moody Challenge 2022, we evaluate our model on the CinC 2017 Challenge data that tasks practitioners to classify cardiac arrhythmias from ECG signals. On the CinC 2017 Challenge data, we improve upon the winning F1 scores by approximately 14\% on the hidden validation data.