| Schedule of Derivative Instruments in Statement of Financial Position, Fair Value (Details) - USD ($)$ in Thousands
 | Mar. 31, 2016 | Dec. 31, 2015 | Mar. 31, 2015 | Dec. 31, 2014 | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | $ 26,390,596 | $ 21,998,381 | $ 33,709,043 | $ 48,115,523 | 
| Inverse Interest-Only Securities [Member] |  |  |  |  | 
| Derivative, Fair Value, Net [Abstract] |  |  |  |  | 
| Derivative, Fair Value, Net | 158,786 | 159,582 |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 882,726 | 932,037 | 1,106,210 | 1,168,226 | 
| Interest Rate Swap [Member] |  |  |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 15,425,513 | 14,268,806 | 19,929,000 | 18,584,000 | 
| Credit Default Swap [Member] |  |  |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 125,000 | 125,000 | 125,000 | 125,000 | 
| Interest Rate Swaption [Member] |  |  |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 5,200,000 | 5,200,000 | 12,960,000 | 12,410,000 | 
| Forward Contracts [Member] |  |  |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 1,637,000 | 297,000 | 2,496,000 | 1,325,000 | 
| Options Held [Member] |  |  |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 2,000,000 | 0 | 2,500,000 | 2,000,000 | 
| Swap [Member] |  |  |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount |  |  | 3,000,000 | 14,000,000 | 
| Total Return Swap [Member] |  |  |  |  | 
| Derivative, Fair Value, Net [Abstract] |  |  |  |  | 
| Derivative, Fair Value, Net | (7,120) | 1,645 |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 868,145 | 889,418 | 877,529 | 598,459 | 
| Loan Purchase Commitments [Member] |  |  |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 252,212 | 286,120 | $ 707,304 | $ 554,838 | 
| Derivative Financial Instruments, Assets [Member] |  |  |  |  | 
| Derivative, Fair Value, Net [Abstract] |  |  |  |  | 
| Derivative, Fair Value, Net | 197,847 | 271,509 |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 17,869,498 | 21,735,997 |  |  | 
| Derivative Financial Instruments, Assets [Member] | Inverse Interest-Only Securities [Member] |  |  |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 882,726 | 932,037 |  |  | 
| Derivative Financial Instruments, Assets [Member] | Interest Rate Swap [Member] |  |  |  |  | 
| Derivative, Fair Value, Net [Abstract] |  |  |  |  | 
| Derivative, Fair Value, Net | 18,162 | 91,757 |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 8,865,513 | 14,268,806 |  |  | 
| Derivative Financial Instruments, Assets [Member] | Credit Default Swap [Member] |  |  |  |  | 
| Derivative, Fair Value, Net [Abstract] |  |  |  |  | 
| Derivative, Fair Value, Net | 0 | 0 |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 0 | 0 |  |  | 
| Derivative Financial Instruments, Assets [Member] | Interest Rate Swaption [Member] |  |  |  |  | 
| Derivative, Fair Value, Net [Abstract] |  |  |  |  | 
| Derivative, Fair Value, Net | 10,482 | 17,374 |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 3,700,000 | 4,700,000 |  |  | 
| Derivative Financial Instruments, Assets [Member] | Forward Contracts [Member] |  |  |  |  | 
| Derivative, Fair Value, Net [Abstract] |  |  |  |  | 
| Derivative, Fair Value, Net | 9,557 | 1,074 |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 2,187,000 | 847,000 |  |  | 
| Derivative Financial Instruments, Assets [Member] | Options Held [Member] |  |  |  |  | 
| Derivative, Fair Value, Net [Abstract] |  |  |  |  | 
| Derivative, Fair Value, Net | 366 |  |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 2,000,000 |  |  |  | 
| Derivative Financial Instruments, Assets [Member] | Total Return Swap [Member] |  |  |  |  | 
| Derivative, Fair Value, Net [Abstract] |  |  |  |  | 
| Derivative, Fair Value, Net | 0 | 1,645 |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 0 | 889,418 |  |  | 
| Derivative Financial Instruments, Assets [Member] | Loan Purchase Commitments [Member] |  |  |  |  | 
| Derivative, Fair Value, Net [Abstract] |  |  |  |  | 
| Derivative, Fair Value, Net | 494 | 77 |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 234,259 | 98,736 |  |  | 
| Derivative Financial Instruments, Liabilities [Member] |  |  |  |  | 
| Derivative, Fair Value, Net [Abstract] |  |  |  |  | 
| Derivative, Fair Value, Net | (77,038) | (7,285) |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 9,621,098 | 1,362,384 |  |  | 
| Derivative Financial Instruments, Liabilities [Member] | Inverse Interest-Only Securities [Member] |  |  |  |  | 
| Derivative, Fair Value, Net [Abstract] |  |  |  |  | 
| Derivative, Fair Value, Net | 0 | 0 |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 0 | 0 |  |  | 
| Derivative Financial Instruments, Liabilities [Member] | Interest Rate Swap [Member] |  |  |  |  | 
| Derivative, Fair Value, Net [Abstract] |  |  |  |  | 
| Derivative, Fair Value, Net | (65,715) | 0 |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 6,560,000 | 0 |  |  | 
| Derivative Financial Instruments, Liabilities [Member] | Credit Default Swap [Member] |  |  |  |  | 
| Derivative, Fair Value, Net [Abstract] |  |  |  |  | 
| Derivative, Fair Value, Net | (390) | (703) |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 125,000 | 125,000 |  |  | 
| Derivative Financial Instruments, Liabilities [Member] | Interest Rate Swaption [Member] |  |  |  |  | 
| Derivative, Fair Value, Net [Abstract] |  |  |  |  | 
| Derivative, Fair Value, Net | (473) | (4,831) |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 1,500,000 | 500,000 |  |  | 
| Derivative Financial Instruments, Liabilities [Member] | Forward Contracts [Member] |  |  |  |  | 
| Derivative, Fair Value, Net [Abstract] |  |  |  |  | 
| Derivative, Fair Value, Net | (3,328) | (1,324) |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 550,000 | 550,000 |  |  | 
| Derivative Financial Instruments, Liabilities [Member] | Options Held [Member] |  |  |  |  | 
| Derivative, Fair Value, Net [Abstract] |  |  |  |  | 
| Derivative, Fair Value, Net | 0 |  |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 0 |  |  |  | 
| Derivative Financial Instruments, Liabilities [Member] | Total Return Swap [Member] |  |  |  |  | 
| Derivative, Fair Value, Net [Abstract] |  |  |  |  | 
| Derivative, Fair Value, Net | (7,120) | 0 |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | 868,145 | 0 |  |  | 
| Derivative Financial Instruments, Liabilities [Member] | Loan Purchase Commitments [Member] |  |  |  |  | 
| Derivative, Fair Value, Net [Abstract] |  |  |  |  | 
| Derivative, Fair Value, Net | (12) | (427) |  |  | 
| Notional Disclosures [Abstract] |  |  |  |  | 
| Derivative, Notional Amount | $ 17,953 | $ 187,384 |  |  |