The next text St. Augustine used, and one which you will hear widely quoted as a text defending the idea of original sin as inherited guilt is Psalm 50:7 (LXX) or 51:5 (Hebrew Masoretic numbering). I’ll start by providing an English translation of each, beginning with the Septuagint rendering (OSB).
For behold, I was conceived in transgressions, And in sins my mother bore me.
And here’s the Hebrew Masoretic translation (NKJV).
Behold, I was brought forth in iniquity, And in sin my mother conceived me.
It is difficult to discuss this verse divorced from the context of the entire Psalm and the way it has been used for centuries by the Church. Even today in the Orthodox Church, the entire Psalm is used in the services of Orthros, the Third Hour, and Compline. It is also recited in every Divine Liturgy by the priest as he censes before the Great Entrance. This is the great penitential Psalm and has been so used by the Church for as long as we have any records. I think it is best approached as a whole text in that light and with that attitude.
However, the natural reading of the single verse above does not communicate to me any idea that David is saying that he was conceived and born with some sort of inherited guilt. Rather he is lamenting the damaged and broken state of humanity into which we are all born. Those who conceive and bear us do so within the brokenness of their own transgressions and sins — their own mortality. Who among us would deny that reality? It’s also possible that in his song of repentance, David is engaging in a bit of hyperbole, but it’s not necessary to assume that’s the case. It’s perhaps a bit clearer in the translation from the LXX, but David is obviously not referring to his own sin in that verse. He’s referring to that of those surrounding him from the moment of his conception. He is making reference to the brokenness which forms and shapes us all.
The verse just doesn’t say what St. Augustine and others who try to use it the same way want it to say. It doesn’t communicate that idea when you lift it out of its context and that is certainly not the message of Psalm 50 (or 51 in the Hebrew numbering) when taken as a whole.