How Do I know the taxable amount to enter on line 5b form 1040?
You need to review the US-Canada tax treaty to determine how your pension is taxed. (I'm assuming this is a private pension, not a government one, like US Social Security.)
this only applies if you made contributions toward your pension. Start with the number in Box 2 of Form 1099-R. If it's blank, or if it seems wrong, and the "taxable amount not determined" box is checked, you need to do a calculation to determine how much you can recover tax-free every year.
This IRS page https://www.irs.gov/taxtopics/tc411 has information about the "simplified method" and links to the worksheet to do the calculation. You'll need the annuity starting date, your age at the start of the annuity (both spouses if a joint & survivor annuity), and the amount of your contributions. Basically what it boils down to is dividing your contributions over your life expectancy to get a dollar amount you can exclude every year until it's all recovered.
Thank you for that!
My pension is Canadian, so it does add another layer to my question.
"Poster JMSSGV is referencing IRS Publication 721, which is the tax guide for those who have retired from a civil service position with the U.S. Government, such as postal workers, civil engineers, revenue agents, etc. That guide explains how U.S. Civil Servants can compute the non-taxable portion of their retirement benefits."
Please note that I did NOT reference Pub 721. It is one of the links on the Pensions-Tax Topic page I linked to. The information on that page is not limited to government pensions.