I am really not a latex expert, but I have been reading about latex and using latex for awhile.
While I would say latex generally does not break down like memory foam - that's for sure - I would also say that latex DOES get somewhat softer with use. Not a lot but SOME. Maybe this is an arguable point but to me it seems that it does get a bit softer, and it stands to reason that it would because it is still FOAM and FOAM does break-in break-down over time.
That said, in this case it may be that you just "feel" it differently, that it's more likely just a perception.
If it continues to feel too firm I would try a softer Talalay topper - maybe only 1" or no more than 2" on top. SleepLikeABear has Talatech latex with something like 14ILD that I am dying to try; you might consider that.