I already have 24GB of RAM...
24? Not 16 or 32? You have three sticks of 8GB?
I'm a little out of the loop on current gen stuff, but your RAM needs to be in identical pairs or quads (e.g. 2x8, 2x16, 4x8, etc.) to maximize throughput. With odd-numbered configurations, your RAM will run in single channel mode, meaning diminished performance. In identical pairs, it runs in dual channel mode, meaning better performance.
I would take out a stick or add one to your build, or buy a 2x16GB kit.
To save money, you could swap out the Samsung SSD for a Crucial P5, but wait until Christmas when those parts typically go on sale.