Time and patience. You're on the right track by changing your diet, that's the only thing that has helped me.

Some thoughts:

Eat more fats. Actually, eat more in general. You're not getting anywhere near your calories according to your diary. Try to get closer so you get the proper amount of nutrients. Try getting more fat calories in.

Eat more fiberous veggies. Fiber helps with constipation. So it might help to try to increase the amount of fiber you get.

Try some flax. Flax is a natural stool softener, which might help with your constipation (though I've seen mention that flax might irritate some IBS patients, so start small and listen to your body, or try it after you've tried other things).

Stay hydrated. Shouldn't need any explanation here, but drink lots of water.

Eat fermented foods/drink fermented drinks. Kombucha, saurkraut, kefir, and other fermented goods help feed and balance gut flora, improving gut health all around.

Be patient. More complex issues take longer to see relief, so it may be a little while before you start seeing major changes. However, keep in mind that if this is the first day you've needed meds, and you were taking them more frequently than once every 4 days before going Paleo, that you're already seeing some improvement. It might not seem like it in the middle of an attack, but try to keep that in mind.