go with the rectangle. normally the octogons have to be large gallon wise becasue they are normally used for showtanks and are generally tall. so get the 29
Neither are big enough long term in my opinion. If two of the Convicts are a breeding pair you're going to need to get rid of everything in the tank; if they aren't, you're going to either have to "sell" a Convict or your Firemouth.
Like mentioned before, the 29G will have more surface area than the 30G, and surface area is more important than volume. A 50G Tall(36x18x17) has the same surface area as a 40G breeder, but 25% more volume than the 40G bredder. The extra 10G does not make a big difference unless you dealing with taller fish like angels and discus.
hex tanks are hard to get the most out of. 29G tanks aren't too good in my book, either...i'd rather have a 20 long. i know you can get cheap 29G kits, which is probably why that is your main choice besides the 30
I think the 30 octo is tall, less surface area. Go with the 29 (or a regular 30 if you can, even better). I usually stay away from the octos and bowfronts, they just cost more and for the price I'd rather have a bigger regular tank.