Bug 1066812 - e10s: child process continues running if parent process crashes

Sounds like I need to learn how to do multi-threading with Gecko!

NO FEAR. (Ok, actually full of fear. But let's try to be brave here)

From bsmedberg:

"So I expect that this is peculiar to the case of the content process ilooping:

In ContentChild::ActorDestroy we QuickExit when the parent goes away, but that method is only called on the main thread on an event. http://mxr.mozilla.org/mozilla-central/source/dom/ipc/ContentChild.cpp#1557

So if the main thread is ilooping or deadlocked, we won't get there. Currently the MessageListener API only has callbacks on the main thread, but there is MessageChannel::mAbortOnError and SetAbortOnError which the plugin code uses to hard-abort. The content code could probably use something similar."

Sounds like I can use the plugin code as inspiration.

First off, some weirdness:

MessageChannel::AbortOnError takes a bool argument, but doesn't actually do anything with it: https://hg.mozilla.org/mozilla-central/file/0c2f7434c325/ipc/glue/MessageChannel.h#l89

Huh.

Well, maybe I can clean that up while I'm at it.

Wow. I put it in the initialization function of ContentChild, and it just kinda worked. Could it be that simple? I'm skeptical. It's never that simple.

I've asked billm for review.

Bill thinks this is probably fine - I just need to test b2g. I'll push to try tonight.

Try build

Looking good. Review request up.

Review in! Let's land!

GRAWWARR - backed out.

WHAT THE FUCK

FUCK YOU, TESTS.

That's B2G ICS Emulator opt Mochitest (5)

12:30:51 INFO - [Child 2022] ###!!! ABORT: Aborting on channel error.: file ../../../gecko/ipc/glue/MessageChannel.cpp, line 1604

8777 12:30:51 INFO - 668 INFO TEST-UNEXPECTED-ERROR | dom/ipc/tests/test_NuwaProcessDeadlock.html | This test left crash dumps behind, but we weren't expecting it to!

8780 12:30:52 INFO - 670 INFO TEST-UNEXPECTED-FAIL | dom/ipc/tests/test_NuwaProcessDeadlock.html | undefined assertion name - Result logged after SimpleTest.finish()

8793 12:30:56 INFO - 681 INFO TEST-UNEXPECTED-ERROR | dom/ipc/tests/test_child_docshell.html | This test left crash dumps behind, but we weren't expecting it to!

8796 12:30:56 INFO - 683 INFO TEST-UNEXPECTED-FAIL | dom/ipc/tests/test_child_docshell.html | undefined assertion name - Result logged after SimpleTest.finish()

9135 12:32:05 WARNING - PROCESS-CRASH | dom/manifest/test/test_IconsProcessor_type.html | application crashed [@ mozalloc_abort]

41643 12:34:47 INFO - 02-09 20:30:51.518 I/Gecko ( 2022): [Child 2022] ###!!! ABORT: Aborting on channel error.: file ../../../gecko/ipc/glue/MessageChannel.cpp, line 1604

41644 12:34:47 INFO - 02-09 20:30:51.518 E/Gecko ( 2022): mozalloc_abort: [Child 2022] ###!!! ABORT: Aborting on channel error.: file ../../../gecko/ipc/glue/MessageChannel.cpp, line 1604

42179 12:34:47 ERROR - Return code: 1

Windows XP Debug - Mochitest bc1

11:17:21 INFO - 993 INFO TEST-START | browser/base/content/test/social/browser_social_status.js

11:17:39 INFO - 996 INFO TEST-OK | browser/base/content/test/social/browser_social_window.js | took 14264ms

11:17:39 INFO - ++DOCSHELL 14E58400 == 50 [pid = 2644] [id = 211]

11:17:39 INFO - ++DOMWINDOW == 154 (14E5C000) [pid = 2644] [serial = 604] [outer = 00000000]

11:17:39 INFO - ++DOMWINDOW == 155 (15004000) [pid = 2644] [serial = 605] [outer = 14E5C000]

11:17:39 INFO - [Parent 2644] WARNING: NS_ENSURE_TRUE(mMutable) failed: file c:\builds\moz2_slave\fx-team-w32-d-0000000000000000\build\src\netwerk\base\nsSimpleURI.cpp, line 265

11:17:39 INFO - [Parent 2644] WARNING: Subdocument container has no frame: file c:\builds\moz2_slave\fx-team-w32-d-0000000000000000\build\src\layout\base\nsDocumentViewer.cpp, line 2504

11:17:39 INFO - ++DOMWINDOW == 156 (14E66000) [pid = 2644] [serial = 606] [outer = 0F33A400]

11:17:39 INFO - [Parent 2644] WARNING: Subdocument container has no frame: file c:\builds\moz2_slave\fx-team-w32-d-0000000000000000\build\src\layout\base\nsDocumentViewer.cpp, line 2504

11:17:39 INFO - ++DOMWINDOW == 157 (10C1D000) [pid = 2644] [serial = 607] [outer = 0F346C00]

11:17:40 INFO - --DOCSHELL 1100AC00 == 49 [pid = 2644] [id = 177]

11:17:40 INFO - --DOCSHELL 17FBE800 == 48 [pid = 2644] [id = 173]

11:17:40 INFO - --DOCSHELL 10420800 == 47 [pid = 2644] [id = 130]

11:17:40 INFO - --DOCSHELL 13226C00 == 46 [pid = 2644] [id = 142]

11:17:40 INFO - --DOCSHELL 13709400 == 45 [pid = 2644] [id = 145]

11:17:40 INFO - --DOCSHELL 181F3400 == 44 [pid = 2644] [id = 175]

11:17:40 INFO - --DOCSHELL 13704C00 == 43 [pid = 2644] [id = 8]

11:17:40 INFO - --DOCSHELL 1100C400 == 42 [pid = 2644] [id = 178]

11:17:40 INFO - --DOCSHELL 17FC0000 == 41 [pid = 2644] [id = 174]

11:17:40 INFO - --DOCSHELL 16E0FC00 == 40 [pid = 2644] [id = 161]

11:17:40 INFO - --DOCSHELL 0F550000 == 39 [pid = 2644] [id = 179]

11:17:40 INFO - --DOCSHELL 11004800 == 38 [pid = 2644] [id = 176]

11:17:40 INFO - --DOCSHELL 17DA0400 == 37 [pid = 2644] [id = 172]

11:17:40 INFO - --DOCSHELL 1500B800 == 36 [pid = 2644] [id = 198]

11:17:40 INFO - --DOCSHELL 0F338000 == 35 [pid = 2644] [id = 192]

11:17:40 INFO - --DOCSHELL 163A9C00 == 34 [pid = 2644] [id = 188]

11:17:40 INFO - --DOCSHELL 122B9400 == 33 [pid = 2644] [id = 184]

11:17:40 INFO - --DOCSHELL 0FBF0400 == 32 [pid = 2644] [id = 181]

11:17:40 INFO - --DOCSHELL 0F6A4400 == 31 [pid = 2644] [id = 180]

11:17:41 INFO - --DOCSHELL 19561C00 == 30 [pid = 2644] [id = 208]

11:17:41 INFO - --DOCSHELL 19563800 == 29 [pid = 2644] [id = 209]

11:17:41 INFO - --DOCSHELL 17B15800 == 28 [pid = 2644] [id = 204]

11:17:41 INFO - --DOCSHELL 17B2A800 == 27 [pid = 2644] [id = 205]

11:17:41 INFO - --DOCSHELL 1639C000 == 26 [pid = 2644] [id = 200]

11:17:41 INFO - --DOCSHELL 1639E000 == 25 [pid = 2644] [id = 201]

11:17:41 INFO - --DOCSHELL 0E27E800 == 24 [pid = 2644] [id = 196]

11:17:41 INFO - --DOCSHELL 0F54C800 == 23 [pid = 2644] [id = 190]

11:17:41 INFO - --DOCSHELL 0FABB800 == 22 [pid = 2644] [id = 191]

11:17:41 INFO - --DOCSHELL 0D73F400 == 21 [pid = 2644] [id = 186]

11:17:41 INFO - --DOCSHELL 11B2EC00 == 20 [pid = 2644] [id = 187]

11:17:41 INFO - --DOCSHELL 1150E800 == 19 [pid = 2644] [id = 182]

11:17:41 INFO - --DOCSHELL 11572400 == 18 [pid = 2644] [id = 183]

11:17:41 INFO - --DOCSHELL 1C66F400 == 17 [pid = 2644] [id = 210]

11:17:41 INFO - --DOCSHELL 17D50C00 == 16 [pid = 2644] [id = 206]

11:17:41 INFO - --DOCSHELL 17B16C00 == 15 [pid = 2644] [id = 203]

11:17:41 INFO - --DOCSHELL 0C958400 == 14 [pid = 2644] [id = 189]

11:17:41 INFO - --DOCSHELL 16B45000 == 13 [pid = 2644] [id = 202]

11:17:41 INFO - --DOCSHELL 13442000 == 12 [pid = 2644] [id = 195]

11:17:41 INFO - --DOCSHELL 1327A400 == 11 [pid = 2644] [id = 193]

11:17:41 INFO - --DOCSHELL 181EF800 == 10 [pid = 2644] [id = 207]

11:17:41 INFO - --DOCSHELL 16591400 == 9 [pid = 2644] [id = 185]

11:17:41 INFO - --DOCSHELL 11DA0000 == 8 [pid = 2644] [id = 197]

11:17:41 INFO - --DOCSHELL 16316C00 == 7 [pid = 2644] [id = 199]

11:17:41 INFO - --DOCSHELL 132B8C00 == 6 [pid = 2644] [id = 194]

11:17:41 INFO - --DOMWINDOW == 156 (13282000) [pid = 2644] [serial = 379] [outer = 00000000] [url = chrome://global/content/mozilla.xhtml]

11:17:41 INFO - --DOMWINDOW == 155 (0FBEF000) [pid = 2644] [serial = 400] [outer = 00000000] [url = chrome://global/content/mozilla.xhtml]

11:17:41 INFO - --DOMWINDOW == 154 (10420C00) [pid = 2644] [serial = 401] [outer = 00000000] [url = chrome://global/content/mozilla.xhtml]

11:17:41 INFO - --DOMWINDOW == 153 (15D57000) [pid = 2644] [serial = 448] [outer = 00000000] [url = https://example.com/browser/browser/base/content/test/social/social_sidebar.html]

11:17:41 INFO - --DOMWINDOW == 152 (14E5E800) [pid = 2644] [serial = 423] [outer = 00000000] [url = http://example.com/]

11:17:41 INFO - --DOMWINDOW == 151 (16707400) [pid = 2644] [serial = 439] [outer = 00000000] [url = http://example.com/]

11:17:41 INFO - --DOMWINDOW == 150 (15645000) [pid = 2644] [serial = 444] [outer = 00000000] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 149 (16E15000) [pid = 2644] [serial = 451] [outer = 00000000] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 148 (16B4A000) [pid = 2644] [serial = 456] [outer = 00000000] [url = https://test2.example.com/browser/browser/base/content/test/social/social_activate.html]

11:17:41 INFO - --DOMWINDOW == 147 (0D73A400) [pid = 2644] [serial = 459] [outer = 00000000] [url = https://test1.example.com/browser/browser/base/content/test/social/social_activate.html]

11:17:41 INFO - --DOMWINDOW == 146 (1630F800) [pid = 2644] [serial = 513] [outer = 00000000] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 145 (15D5C400) [pid = 2644] [serial = 486] [outer = 00000000] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 144 (16442000) [pid = 2644] [serial = 438] [outer = 00000000] [url = about:privatebrowsing]

11:17:41 INFO - --DOMWINDOW == 143 (16B41000) [pid = 2644] [serial = 442] [outer = 00000000] [url = about:privatebrowsing]

11:17:41 INFO - --DOMWINDOW == 142 (1539C800) [pid = 2644] [serial = 466] [outer = 00000000] [url = about:socialerror?mode=tryAgainOnly&url=https%3A%2F%2Ftest1.example.com%2Fbrowser%2Fbrowser%2Fbase%2Fcontent%2Ftest%2Fsocial%2Fsocial_panel.html&origin=https%3A%2F%2Ftest1.example.com]

11:17:41 INFO - --DOMWINDOW == 141 (1769D800) [pid = 2644] [serial = 464] [outer = 00000000] [url = https://test1.example.com/browser/browser/base/content/test/social/social_panel.html]

11:17:41 INFO - --DOMWINDOW == 140 (17B37C00) [pid = 2644] [serial = 491] [outer = 00000000] [url = chrome://global/content/mozilla.xhtml]

11:17:41 INFO - --DOMWINDOW == 139 (181F5800) [pid = 2644] [serial = 516] [outer = 00000000] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 138 (17654800) [pid = 2644] [serial = 498] [outer = 00000000] [url = https://example.com/browser/browser/base/content/test/social/social_sidebar.html]

11:17:41 INFO - --DOMWINDOW == 137 (137E2400) [pid = 2644] [serial = 507] [outer = 00000000] [url = https://example.com/browser/browser/base/content/test/social/social_sidebar.html]

11:17:41 INFO - --DOMWINDOW == 136 (17B28800) [pid = 2644] [serial = 477] [outer = 00000000] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 135 (1718BC00) [pid = 2644] [serial = 479] [outer = 00000000] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 134 (17B0C000) [pid = 2644] [serial = 502] [outer = 00000000] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 133 (1704FC00) [pid = 2644] [serial = 470] [outer = 00000000] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 132 (15398C00) [pid = 2644] [serial = 528] [outer = 11518400] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 131 (0EEF2400) [pid = 2644] [serial = 539] [outer = 11B2E400] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 130 (0FAB7400) [pid = 2644] [serial = 549] [outer = 0F552400] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 129 (11512400) [pid = 2644] [serial = 550] [outer = 0FAC1400] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 128 (16E97000) [pid = 2644] [serial = 573] [outer = 1639D800] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 127 (17B12400) [pid = 2644] [serial = 577] [outer = 1639E400] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 126 (17656C00) [pid = 2644] [serial = 499] [outer = 17FBF800] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 125 (17B31000) [pid = 2644] [serial = 510] [outer = 1100BC00] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 124 (17DA9C00) [pid = 2644] [serial = 585] [outer = 17B18C00] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 123 (1D314800) [pid = 2644] [serial = 601] [outer = 17B2B400] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 122 (1D286400) [pid = 2644] [serial = 594] [outer = 19563000] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 121 (0E42E000) [pid = 2644] [serial = 519] [outer = 0F346C00] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 120 (15629400) [pid = 2644] [serial = 562] [outer = 11D9F800] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 119 (11D9F800) [pid = 2644] [serial = 558] [outer = 00000000] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 118 (19563000) [pid = 2644] [serial = 590] [outer = 00000000] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 117 (17B18C00) [pid = 2644] [serial = 580] [outer = 00000000] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 116 (1100BC00) [pid = 2644] [serial = 505] [outer = 00000000] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 115 (17FBF800) [pid = 2644] [serial = 494] [outer = 00000000] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 114 (1639D800) [pid = 2644] [serial = 568] [outer = 00000000] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 113 (0FAC1400) [pid = 2644] [serial = 546] [outer = 00000000] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 112 (0F552400) [pid = 2644] [serial = 545] [outer = 00000000] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 111 (11B2E400) [pid = 2644] [serial = 535] [outer = 00000000] [url = about:blank]

11:17:41 INFO - --DOMWINDOW == 110 (11518400) [pid = 2644] [serial = 524] [outer = 00000000] [url = about:blank]

11:17:42 INFO - [Parent 2644] WARNING: Subdocument container has no frame: file c:\builds\moz2_slave\fx-team-w32-d-0000000000000000\build\src\layout\base\nsDocumentViewer.cpp, line 2504

11:17:42 INFO - ++DOMWINDOW == 111 (0C6B2400) [pid = 2644] [serial = 608] [outer = 0F33A400]

11:17:42 INFO - [Parent 2644] WARNING: Subdocument container has no frame: file c:\builds\moz2_slave\fx-team-w32-d-0000000000000000\build\src\layout\base\nsDocumentViewer.cpp, line 2504

11:17:42 INFO - ++DOMWINDOW == 112 (0C92B000) [pid = 2644] [serial = 609] [outer = 0F346C00]

11:17:42 INFO - [Parent 2644] WARNING: NS_ENSURE_TRUE(currentInner) failed: file c:/builds/moz2_slave/fx-team-w32-d-0000000000000000/build/src/dom/base/nsGlobalWindow.cpp, line 8886

11:17:42 INFO - --DOMWINDOW == 111 (1955C000) [pid = 2644] [serial = 589] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 110 (15626800) [pid = 2644] [serial = 425] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 109 (17B17800) [pid = 2644] [serial = 579] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 108 (16704400) [pid = 2644] [serial = 534] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 107 (12173800) [pid = 2644] [serial = 418] [outer = 00000000] [url = chrome://global/content/mozilla.xhtml]

11:17:42 INFO - --DOMWINDOW == 106 (122B9800) [pid = 2644] [serial = 460] [outer = 00000000] [url = chrome://global/content/mozilla.xhtml]

11:17:42 INFO - --DOMWINDOW == 105 (1769F800) [pid = 2644] [serial = 487] [outer = 00000000] [url = chrome://global/content/mozilla.xhtml]

11:17:42 INFO - --DOMWINDOW == 104 (0F550400) [pid = 2644] [serial = 508] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 103 (17FC0400) [pid = 2644] [serial = 495] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 102 (181F3800) [pid = 2644] [serial = 496] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 101 (1704CC00) [pid = 2644] [serial = 452] [outer = 00000000] [url = chrome://global/content/mozilla.xhtml]

11:17:42 INFO - --DOMWINDOW == 100 (132C0400) [pid = 2644] [serial = 445] [outer = 00000000] [url = chrome://global/content/mozilla.xhtml]

11:17:42 INFO - --DOMWINDOW == 99 (0F6A4800) [pid = 2644] [serial = 520] [outer = 00000000] [url = chrome://global/content/mozilla.xhtml]

11:17:42 INFO - --DOMWINDOW == 98 (13280C00) [pid = 2644] [serial = 552] [outer = 00000000] [url = chrome://global/content/mozilla.xhtml]

11:17:42 INFO - --DOMWINDOW == 97 (132C3C00) [pid = 2644] [serial = 554] [outer = 00000000] [url = chrome://global/content/mozilla.xhtml]

11:17:42 INFO - --DOMWINDOW == 96 (132BE400) [pid = 2644] [serial = 420] [outer = 00000000] [url = chrome://global/content/mozilla.xhtml]

11:17:42 INFO - --DOMWINDOW == 95 (10F6C000) [pid = 2644] [serial = 461] [outer = 00000000] [url = chrome://global/content/mozilla.xhtml]

11:17:42 INFO - --DOMWINDOW == 94 (176A0400) [pid = 2644] [serial = 488] [outer = 00000000] [url = chrome://global/content/mozilla.xhtml]

11:17:42 INFO - --DOMWINDOW == 93 (1100C800) [pid = 2644] [serial = 506] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 92 (17D51000) [pid = 2644] [serial = 582] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 91 (1C66F800) [pid = 2644] [serial = 592] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 90 (19563C00) [pid = 2644] [serial = 591] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 89 (0F555C00) [pid = 2644] [serial = 547] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 88 (16B45800) [pid = 2644] [serial = 571] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 87 (163AA000) [pid = 2644] [serial = 537] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 86 (11B2F000) [pid = 2644] [serial = 536] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 85 (11DECC00) [pid = 2644] [serial = 542] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 84 (0F54A400) [pid = 2644] [serial = 541] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 83 (181F0000) [pid = 2644] [serial = 588] [outer = 00000000] [url = chrome://browser/content/browser.xul]

11:17:42 INFO - --DOMWINDOW == 82 (17FC1400) [pid = 2644] [serial = 587] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 81 (16593400) [pid = 2644] [serial = 533] [outer = 00000000] [url = chrome://browser/content/browser.xul]

11:17:42 INFO - --DOMWINDOW == 80 (17B2B400) [pid = 2644] [serial = 581] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 79 (163A5800) [pid = 2644] [serial = 532] [outer = 00000000] [url = https://example.com/browser/browser/base/content/test/social/social_sidebar.html]

11:17:42 INFO - --DOMWINDOW == 78 (16312400) [pid = 2644] [serial = 531] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 77 (15B41C00) [pid = 2644] [serial = 530] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 76 (1639E400) [pid = 2644] [serial = 569] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 75 (17189000) [pid = 2644] [serial = 575] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 74 (16317800) [pid = 2644] [serial = 566] [outer = 00000000] [url = chrome://browser/content/browser.xul]

11:17:42 INFO - --DOMWINDOW == 73 (0FBF1800) [pid = 2644] [serial = 521] [outer = 00000000] [url = chrome://browser/content/browser.xul]

11:17:42 INFO - --DOMWINDOW == 72 (13442C00) [pid = 2644] [serial = 555] [outer = 00000000] [url = chrome://browser/content/browser.xul]

11:17:42 INFO - --DOMWINDOW == 71 (153A6400) [pid = 2644] [serial = 424] [outer = 00000000] [url = chrome://browser/content/browser.xul]

11:17:42 INFO - --DOMWINDOW == 70 (122BEC00) [pid = 2644] [serial = 526] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 69 (11574000) [pid = 2644] [serial = 525] [outer = 00000000] [url = https://example.com/browser/browser/base/content/test/social/social_sidebar.html]

11:17:42 INFO - --DOMWINDOW == 68 (1500BC00) [pid = 2644] [serial = 560] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 67 (12169C00) [pid = 2644] [serial = 10] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 66 (17189C00) [pid = 2644] [serial = 468] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 65 (16E16000) [pid = 2644] [serial = 467] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 64 (1562F400) [pid = 2644] [serial = 564] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 63 (16317400) [pid = 2644] [serial = 576] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 62 (17DA3C00) [pid = 2644] [serial = 584] [outer = 00000000] [url = https://test1.example.com/browser/browser/base/content/test/social/social_sidebar.html]

11:17:42 INFO - --DOMWINDOW == 61 (17B17000) [pid = 2644] [serial = 578] [outer = 00000000] [url = chrome://browser/content/browser.xul]

11:17:42 INFO - --DOMWINDOW == 60 (0C92CC00) [pid = 2644] [serial = 517] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 59 (1631AC00) [pid = 2644] [serial = 514] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 58 (1955B400) [pid = 2644] [serial = 512] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 57 (11004C00) [pid = 2644] [serial = 503] [outer = 00000000] [url = chrome://browser/content/browser.xul]

11:17:42 INFO - --DOMWINDOW == 56 (0D73AC00) [pid = 2644] [serial = 518] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 55 (17DA8400) [pid = 2644] [serial = 515] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 54 (17659C00) [pid = 2644] [serial = 501] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 53 (17DA0800) [pid = 2644] [serial = 492] [outer = 00000000] [url = chrome://browser/content/browser.xul]

11:17:42 INFO - --DOMWINDOW == 52 (1704D400) [pid = 2644] [serial = 453] [outer = 00000000] [url = chrome://global/content/mozilla.xhtml]

11:17:42 INFO - --DOMWINDOW == 51 (15FA7000) [pid = 2644] [serial = 447] [outer = 00000000] [url = chrome://global/content/mozilla.xhtml]

11:17:42 INFO - --DOMWINDOW == 50 (11006400) [pid = 2644] [serial = 523] [outer = 00000000] [url = chrome://global/content/mozilla.xhtml]

11:17:42 INFO - --DOMWINDOW == 49 (13284000) [pid = 2644] [serial = 553] [outer = 00000000] [url = chrome://global/content/mozilla.xhtml]

11:17:42 INFO - --DOMWINDOW == 48 (17651800) [pid = 2644] [serial = 598] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 47 (0CC69000) [pid = 2644] [serial = 600] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 46 (1151A000) [pid = 2644] [serial = 551] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 45 (0CC6A400) [pid = 2644] [serial = 557] [outer = 00000000] [url = chrome://global/content/mozilla.xhtml]

11:17:42 INFO - --DOMWINDOW == 44 (1D316800) [pid = 2644] [serial = 602] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 43 (0C95DC00) [pid = 2644] [serial = 543] [outer = 00000000] [url = chrome://browser/content/browser.xul]

11:17:42 INFO - --DOMWINDOW == 42 (1D289000) [pid = 2644] [serial = 596] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 41 (11DEB400) [pid = 2644] [serial = 559] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 40 (17D9F800) [pid = 2644] [serial = 597] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 39 (15B3DC00) [pid = 2644] [serial = 22] [outer = 00000000] [url = about:newtab]

11:17:42 INFO - --DOMWINDOW == 38 (1562BC00) [pid = 2644] [serial = 563] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 37 (15D59400) [pid = 2644] [serial = 565] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 36 (1643D400) [pid = 2644] [serial = 570] [outer = 00000000] [url = https://example.com/browser/browser/base/content/test/social/social_sidebar.html]

11:17:42 INFO - --DOMWINDOW == 35 (17DA1000) [pid = 2644] [serial = 599] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 34 (17FC3000) [pid = 2644] [serial = 603] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 33 (153A0000) [pid = 2644] [serial = 561] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 32 (13705400) [pid = 2644] [serial = 18] [outer = 00000000] [url = about:newtab]

11:17:42 INFO - --DOMWINDOW == 31 (19559C00) [pid = 2644] [serial = 511] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 30 (17D9C400) [pid = 2644] [serial = 583] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 29 (17FBD000) [pid = 2644] [serial = 586] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 28 (1C674000) [pid = 2644] [serial = 593] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 27 (1D287400) [pid = 2644] [serial = 595] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 26 (17652800) [pid = 2644] [serial = 538] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 25 (0F348000) [pid = 2644] [serial = 540] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 24 (10FA8C00) [pid = 2644] [serial = 548] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 23 (16E89C00) [pid = 2644] [serial = 572] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 22 (17052C00) [pid = 2644] [serial = 574] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 21 (10F6C800) [pid = 2644] [serial = 522] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 20 (17DA7400) [pid = 2644] [serial = 493] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 19 (11005400) [pid = 2644] [serial = 504] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 18 (0CCCA800) [pid = 2644] [serial = 544] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 17 (13702C00) [pid = 2644] [serial = 556] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 16 (16318000) [pid = 2644] [serial = 567] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 15 (15004400) [pid = 2644] [serial = 527] [outer = 00000000] [url = about:blank]

11:17:42 INFO - --DOMWINDOW == 14 (153A3C00) [pid = 2644] [serial = 529] [outer = 00000000] [url = about:blank]

11:17:43 INFO - --DOMWINDOW == 13 (14E66000) [pid = 2644] [serial = 606] [outer = 0F33A400] [url = about:blank]

11:17:43 INFO - --DOMWINDOW == 12 (10C1D000) [pid = 2644] [serial = 607] [outer = 0F346C00] [url = about:blank]

11:17:44 INFO - ###!!! [Parent][OnMaybeDequeueOne] Error: Channel closing: too late to send/recv, messages will be lost

11:17:44 INFO - [Parent 2644] WARNING: getting z level of unregistered window: file c:\builds\moz2_slave\fx-team-w32-d-0000000000000000\build\src\xpfe\appshell\nsWindowMediator.cpp, line 591

11:17:44 INFO - --DOCSHELL 13439400 == 5 [pid = 2644] [id = 6]

11:17:44 INFO - --DOCSHELL 0E4A2C00 == 4 [pid = 2644] [id = 1]

11:17:45 INFO - --DOCSHELL 0F332000 == 3 [pid = 2644] [id = 3]

11:17:45 INFO - --DOCSHELL 14E58400 == 2 [pid = 2644] [id = 211]

11:17:45 INFO - --DOCSHELL 0F343400 == 1 [pid = 2644] [id = 4]

11:17:45 INFO - --DOCSHELL 0F672800 == 0 [pid = 2644] [id = 2]

11:17:45 INFO - --DOMWINDOW == 11 (0C6B2400) [pid = 2644] [serial = 608] [outer = 0F33A400] [url = about:blank]

11:17:45 INFO - --DOMWINDOW == 10 (0C92B000) [pid = 2644] [serial = 609] [outer = 0F346C00] [url = about:blank]

11:17:45 INFO - ###!!! [Parent][OnMaybeDequeueOne] Error: Channel closing: too late to send/recv, messages will be lost

11:17:46 INFO - ###!!! [Parent][OnMaybeDequeueOne] Error: Channel closing: too late to send/recv, messages will be lost

11:17:46 INFO - --DOMWINDOW == 9 (0F33A400) [pid = 2644] [serial = 6] [outer = 00000000] [url = about:blank]

11:17:46 INFO - --DOMWINDOW == 8 (0F346C00) [pid = 2644] [serial = 7] [outer = 00000000] [url = about:blank]

11:17:47 INFO - --DOMWINDOW == 7 (0F69D400) [pid = 2644] [serial = 4] [outer = 00000000] [url = about:blank]

11:17:47 INFO - --DOMWINDOW == 6 (1343A000) [pid = 2644] [serial = 15] [outer = 00000000] [url = about:blank]

11:17:47 INFO - --DOMWINDOW == 5 (0F69CC00) [pid = 2644] [serial = 3] [outer = 00000000] [url = chrome://browser/content/browser.xul]

11:17:47 INFO - --DOMWINDOW == 4 (13439800) [pid = 2644] [serial = 14] [outer = 00000000] [url = chrome://mochikit/content/browser-harness.xul]

11:17:47 INFO - --DOMWINDOW == 3 (14E5C000) [pid = 2644] [serial = 604] [outer = 00000000] [url = about:blank]

11:17:47 INFO - --DOMWINDOW == 2 (15004000) [pid = 2644] [serial = 605] [outer = 00000000] [url = about:blank]

11:17:47 INFO - --DOMWINDOW == 1 (0F6A1800) [pid = 2644] [serial = 5] [outer = 00000000] [url = resource://gre-resources/hiddenWindow.html]

11:17:47 INFO - --DOMWINDOW == 0 (0E4A3400) [pid = 2644] [serial = 1] [outer = 00000000] [url = resource://gre-resources/hiddenWindow.html]

11:17:47 INFO - [Parent 2644] WARNING: '!mMainThread', file c:\builds\moz2_slave\fx-team-w32-d-0000000000000000\build\src\xpcom\threads\nsThreadManager.cpp, line 404

11:17:47 INFO - nsStringStats

11:17:47 INFO - => mAllocCount: 522483

11:17:47 INFO - => mReallocCount: 40250

11:17:47 INFO - => mFreeCount: 522483

11:17:47 INFO - => mShareCount: 1115015

11:17:47 INFO - => mAdoptCount: 33523

11:17:47 INFO - => mAdoptFreeCount: 33523

11:17:47 INFO - => Process ID: 2644, Thread ID: 2876

11:17:47 INFO - TEST-INFO | Main app process: exit status 0

11:17:47 INFO - runtests.py | Application ran for: 0:01:28.451000

11:17:47 INFO - zombiecheck | Reading PID log: c:\docume~1\cltbld~1.t-x\locals~1\temp\tmpgwcn0tpidlog

11:17:56 INFO - mozcrash Saved minidump as C:\slave\test\build\blobber_upload_dir\9c4aca3c-1ca0-44e8-a5ab-ebacf071801f.dmp

11:17:56 INFO - mozcrash Saved app info as C:\slave\test\build\blobber_upload_dir\9c4aca3c-1ca0-44e8-a5ab-ebacf071801f.extra

11:17:56 WARNING - PROCESS-CRASH | Main app process exited normally | application crashed [@ ntdll.dll + 0x120e]

11:17:56 INFO - Crash dump filename: c:\docume~1\cltbld~1.t-x\locals~1\temp\tmpjbs8fa.mozrunner\minidumps\9c4aca3c-1ca0-44e8-a5ab-ebacf071801f.dmp

11:17:56 INFO - Operating system: Windows NT

11:17:56 INFO - 5.1.2600 Service Pack 3

11:17:56 INFO - CPU: x86

11:17:56 INFO - GenuineIntel family 6 model 30 stepping 5

11:17:56 INFO - 8 CPUs

11:17:56 INFO - Crash reason: EXCEPTION_BREAKPOINT

11:17:56 INFO - Crash address: 0x7c90120e

11:17:56 INFO - Assertion: Unknown assertion type 0x00000000

11:17:56 INFO - Thread 1 (crashed)

11:17:56 INFO - 0 ntdll.dll + 0x120e

11:17:56 INFO - eip = 0x7c90120e esp = 0x0660f924 ebp = 0x0660fd5c ebx = 0x00000001

11:17:56 INFO - esi = 0x00000000 edi = 0x009642b5 eax = 0x00000002 ecx = 0x00970ad9

11:17:56 INFO - edx = 0x00a30ea0 efl = 0x00000202

11:17:56 INFO - Found by: given as instruction pointer in context

11:17:56 INFO - 1 xul.dll!mozilla::ipc::MessageChannel::OnChannelErrorFromLink() [MessageChannel.cpp:ec092b22f40d : 1604 + 0x17]

11:17:56 INFO - eip = 0x0106a4d2 esp = 0x0660fd64 ebp = 0x0660fdac

11:17:56 INFO - Found by: previous frame's frame pointer

11:17:56 INFO - 2 xul.dll!mozilla::ipc::ProcessLink::OnChannelError() [MessageLink.cpp:ec092b22f40d : 405 + 0x7]

11:17:56 INFO - eip = 0x0106a457 esp = 0x0660fd84 ebp = 0x0660fdac

11:17:56 INFO - Found by: call frame info with scanning

11:17:56 INFO - 3 xul.dll!IPC::Channel::ChannelImpl::OnIOCompleted(base::MessagePumpForIO::IOContext *,unsigned long,unsigned long) [ipc_channel_win.cc:ec092b22f40d : 447 + 0x7]

11:17:56 INFO - eip = 0x010341f9 esp = 0x0660fd94 ebp = 0x0660fdac

11:17:56 INFO - Found by: call frame info

11:17:56 INFO - 4 xul.dll!base::MessagePumpForIO::WaitForIOCompletion(unsigned long,base::MessagePumpForIO::IOHandler *) [message_pump_win.cc:ec092b22f40d : 499 + 0xb]

11:17:56 INFO - eip = 0x0102eec5 esp = 0x0660fdb4 ebp = 0x0660fde4

11:17:56 INFO - Found by: call frame info

11:17:56 INFO - 5 xul.dll!base::MessagePumpForIO::WaitForWork() [message_pump_win.cc:ec092b22f40d : 478 + 0x9]

11:17:56 INFO - eip = 0x0102ef33 esp = 0x0660fdec ebp = 0x0660fe08

11:17:56 INFO - Found by: call frame info

11:17:56 INFO - 6 xul.dll!base::MessagePumpForIO::DoRunLoop() [message_pump_win.cc:ec092b22f40d : 463 + 0x6]

11:17:56 INFO - eip = 0x0102e5ec esp = 0x0660fe10 ebp = 0x0660fe34

11:17:56 INFO - Found by: call frame info

11:17:56 INFO - 7 xul.dll!base::MessagePumpWin::RunWithDispatcher(base::MessagePump::Delegate *,base::MessagePumpWin::Dispatcher *) [message_pump_win.cc:ec092b22f40d : 54 + 0x4]

11:17:56 INFO - eip = 0x0102ed29 esp = 0x0660fe1c ebp = 0x0660fe34

11:17:56 INFO - Found by: call frame info

11:17:56 INFO - 8 xul.dll!base::MessagePumpWin::Run(base::MessagePump::Delegate *) [message_pump_win.h:ec092b22f40d : 78 + 0xc]

11:17:56 INFO - eip = 0x0102ece9 esp = 0x0660fe3c ebp = 0x0660fe44

11:17:56 INFO - Found by: call frame info

11:17:56 INFO - 9 xul.dll!MessageLoop::RunInternal() [message_loop.cc:ec092b22f40d : 233 + 0xe]

11:17:56 INFO - eip = 0x01040935 esp = 0x0660fe4c ebp = 0x0660fe64

11:17:56 INFO - Found by: call frame info

11:17:56 INFO - 10 xul.dll!MessageLoop::RunHandler() [message_loop.cc:ec092b22f40d : 226 + 0x4]

11:17:56 INFO - eip = 0x010408ed esp = 0x0660fe6c ebp = 0x0660fe98

11:17:56 INFO - Found by: call frame info

11:17:56 INFO - 11 xul.dll!MessageLoop::Run() [message_loop.cc:ec092b22f40d : 200 + 0x6]

11:17:56 INFO - eip = 0x01040666 esp = 0x0660fea0 ebp = 0x0660feb8

11:17:56 INFO - Found by: call frame info

11:17:56 INFO - 12 xul.dll!base::Thread::ThreadMain() [thread.cc:ec092b22f40d : 170 + 0xa]

11:17:56 INFO - eip = 0x01051fa3 esp = 0x0660fec0 ebp = 0x0660ffac

11:17:56 INFO - Found by: call frame info

11:17:56 INFO - 13 xul.dll!`anonymous namespace'::ThreadFunc(void *) [platform_thread_win.cc:ec092b22f40d : 26 + 0x7]

11:17:56 INFO - eip = 0x0102fb35 esp = 0x0660ffb4 ebp = 0x0660ffb4

11:17:56 INFO - Found by: call frame info

11:17:56 INFO - 14 kernel32.dll + 0xb712

11:17:56 INFO - eip = 0x7c80b713 esp = 0x0660ffbc ebp = 0x0660ffec

11:17:56 INFO - Found by: call frame info

The problem seems to be the case where the child process seem to keep spinning after the parent process dies.

mccr8 says:

"There's some way to indicate that you are intentionally crashing, which keeps the test harness from freaking out. Probably mozilla::NoteIntentionalCrash()."

billm says:

"In theory, all of these problems are bugs. It would be nice to understand why they happen. Since these appear to be hard to reproduce, it will be difficult.

The parent is supposed to wait for all its children to die before it shuts down. If the children take too long, the parent is supposed to kill them. Either way, the child should never find itself in a state where the parent is dead.

I'll try to take a look at the code a bit tomorrow. If I don't find anything, then maybe we can think about workarounds."

So it'd be good if I could figure out where the parent process tries to kill the children and see how that works.

ContentParent::ShutdownProcess looks like it might be the spot.

Ah, ContentParent::ActorDestroy has this stuff:

// Destroy any processes created by this ContentParent

ContentProcessManager *cpm = ContentProcessManager::GetSingleton();

nsTArray<ContentParentId> childIDArray =

cpm->GetAllChildProcessById(this->ChildID());

for(uint32_t i = 0; i < childIDArray.Length(); i++) {

ContentParent* cp = cpm->GetContentProcessById(childIDArray[i]);

MessageLoop::current()->PostTask( FROM_HERE,

NewRunnableMethod(cp, &ContentParent::ShutDownProcess,

CLOSE_CHANNEL));

}

cpm->RemoveContentProcess(this->ChildID());

It looks to me like this ContentParent can create other ContentParents, and this uses a Runnable to call ShutDownProcess on each one, to take them all down.

So I actually think my original patch is right - if the parent suddenly goes away unexpectedly, have the child processes take themselves out.

The problem seems to be that there are cases where it looks like the Parent isn't doing a good enough job to take down the Children on a proper shutdown.

Right, so it's important to recall that my bug was to fix the former case. The latter case is, it seems, is where I'm falling down.

So it might be worth trying to instrument the clean take-down codepaths to see if there's ever a path where the child process isn't correctly killed by the parent.

So, where do I instrument, and what platforms do I test on try?

Let's do Windows XP Debug on mochitest-bc1.

In terms of instrumenting, I'm going to going old school, and just add some printf statements at a few key places:

The place in MessageChannel that does the abort

The place in ContentParent::ActorDestroy that queues the destruction of all children

ContentParent::ShutDownProcess

ContentChild::RecvShutdown

The else case in ContentParent::ShutDownProcess that elses the point where we call StartForceKillTimer

ContentParent::ForceKillTimerCallback

Various places in ContentParent::KillHard

OK, let's do it!

Try build.

Huh. No failures yet...

Ok, got one. Interesting - it doesn't look like the MessageChannel abort actually happened here.

Conclusion - the Windows XP Debug crash was an intermittent orange that I don't think I introduced.

Wait, no, that's wrong. The stack I'm seeing:

16:27:19 INFO - 1 xul.dll!mozilla::ipc::MessageChannel::OnChannelErrorFromLink() [MessageChannel.cpp:55ea3576e164 : 1605 + 0x19]

16:27:19 INFO - eip = 0x0106cdea esp = 0x0662fd64 ebp = 0x0662fdac

16:27:19 INFO - Found by: previous frame's frame pointer

16:27:19 INFO - 2 xul.dll!mozilla::ipc::ProcessLink::OnChannelError() [MessageLink.cpp:55ea3576e164 : 405 + 0x7]

16:27:19 INFO - eip = 0x0106cd62 esp = 0x0662fd84 ebp = 0x0662fdac

16:27:19 INFO - Found by: call frame info with scanning

OnChannelErrorFromLink is being called, and it's causing the abort on line 1605, which is the one right after my print statement.

So we are killing self in the child process, but my logging (I guess) didn't get printed in time. The stdout buffer probably didn't get flushed before the whole thing got taken down.

The child process does send that ShutdownFinished message though… something must be preventing it from total shutdown though.

Ok, I've instrumented the whole shutdown of the child process more closely.

Theory:

Child process is reporting that it has finished shutdown, so the parent forgets about it and doesn't put it on the chopping block. ContentParent gets ActorDestroy called on it, which sends a message down to the child to also destroy, but the child doesn't hear it in time.

Parent process shuts down, child looks for parent. Gone. Self destruct.

This seems to be a rare case. I wonder if it's as simple as setting AbortOnError to false before sending the ShutdownFinished message?

Let's try that - for both Windows XP Debug and B2G ICS Emulator opt.

Wait wait wait wait

That's wallpapering over the fact that the parent doesn't seem to know about the child process in ContentParent::ActorDestroy - that's code which is supposed to do the killing / cleanup for us. And childIDArray.Length() is just zero. Why?

I can totally reproduce that on my OS X machine - when I exit, before the childID is removed, we attempt to queue shutdown on it, but there's no children with that ID. What the hell?

nsTArray<ContentParentId>
ContentProcessManager::GetAllChildProcessById( const ContentParentId& aParentCpId)
{
MOZ_ASSERT(NS_IsMainThread());

nsTArray<ContentParentId> cpIdArray;
auto iter = mContentParentMap.find(aParentCpId);
if (NS_WARN_IF(iter == mContentParentMap.end())) {
ASSERT_UNLESS_FUZZING();
return Move(cpIdArray);
}

for ( auto cpIter = iter->second.mChildrenCpId.begin();
cpIter != iter->second.mChildrenCpId.end();
++cpIter) {
cpIdArray.AppendElement(*cpIter);
}

return Move(cpIdArray);

}

I think… I think this code is wrong. The part I've highlighted… is going to straight-up exit if there's only one item in the mChildrenCpId iterator...

Wait, this is all wrong. mChildrenCpId is something that B2G uses for grandchild processes. Christ, I've been going down another blind alley.

So billm thinks that my idea of calling SetAbortOnError with false during shutdown is a good idea - probably in ContentChild::ActorDestroy, since this is when the communications channel has definitely broken down.

Pushing that to try to see what it thinks:

Try build.

Another one that removes instrumenting so that B2G Emulator will build.

WaitWAIT WAIT WAIT

What the fuck? How come my patch is causing a crash on non-e10s Windows XP Debug? Is it because of that little fiddly bit I added that makes it possible for AbortOnError to be set to false? What happens if I revert that?

Try push...

And a try push for Win XP Debug.

What the fuck. What the actual fuck.

Ok, theory - we have some tests that run in non-e10s Mochitest runs that open remote browsers, and therefore, start content processes. If that theory is true though, how come I didn't see any of my logging stuff in the stdout?

What am I going to do about this?

AHHHHHHH - the SANDBOX. The sandbox prevents the child process from putting anything through stdout. FOR FUCKS SAKE.

And when I disable the sandbox, the output gets intermingled with the output from the parent process, like a friggin' meat grinder. It's a mess.

Gonna need to talk to jimm or bobowen about this.

Ok, there's a buildtime pref to disable the sandbox. Maybe that's all I need. We'll see.

Try build.

AH HAH. Here we go now.

19:44:30 INFO - 965 INFO TEST-START | browser/base/content/test/social/browser_addons.js

19:44:30 INFO - ++DOCSHELL 10CEEC00 == 14 [pid = 2396] [id = 14]

19:44:30 INFO - ++DOMWINDOW == 41 (10CF1400) [pid = 2396] [serial = 41] [outer = 00000000]

19:44:30 INFO - ++DOMWINDOW == 42 (13782400) [pid = 2396] [serial = 42] [outer = 10CF1400]

19:44:30 INFO - [Parent 2396] WARNING: No docshells for remote frames!: file c:\builds\moz2_slave\try-w32-d-00000000000000000000\build\src\dom\base\nsFrameLoader.cpp, line 513

19:44:30 INFO - [1148] WARNING: XPCOM objects created/destroyed from static ctor/dtor: file c:\builds\moz2_slave\try-w32-d-00000000000000000000\build\src\xpcom\base\nsTraceRefcnt.cpp, line 139

19:44:30 INFO - ### XPCOM_MEM_BLOAT_LOG defined -- logging bloat/leaks to c:\docume~1\cltbld~1.t-x\locals~1\temp\tmpmgvouq.mozrunner\runtests_leaks_tab_pid1148.log

19:44:30 INFO - [1148] WARNING: XPCOM objects created/destroyed from static ctor/dtor: file c:\builds\moz2_slave\try-w32-d-00000000000000000000\build\src\xpcom\base\nsTraceRefcnt.cpp, line 139

19:44:30 INFO - ++DOCSHELL 10CE5000 == 15 [pid = 2396] [id = 15]

19:44:30 INFO - ++DOMWINDOW == 43 (10CE5800) [pid = 2396] [serial = 43] [outer = 00000000]

19:44:30 INFO - [Child 1148] WARNING: '!compMgr', file c:/builds/moz2_slave/try-w32-d-00000000000000000000/build/src/xpcom/glue/nsComponentManagerUtils.cpp, line 63

19:44:30 INFO - [Child 1148] WARNING: NS_ENSURE_TRUE(svc) failed: file c:\builds\moz2_slave\try-w32-d-00000000000000000000\build\src\dom\ipc\nsIContentChild.cpp, line 31

19:44:30 INFO - [Child 1148] WARNING: Error constructing actor PJavaScriptChild: file c:\builds\moz2_slave\try-w32-d-00000000000000000000\build\src\obj-firefox\ipc\ipdl\PContentChild.cpp, line 985

19:44:30 INFO - ++DOMWINDOW == 44 (173D3400) [pid = 2396] [serial = 44] [outer = 10CE5800]

19:44:30 INFO - [Parent 2396] WARNING: No docshells for remote frames!: file c:\builds\moz2_slave\try-w32-d-00000000000000000000\build\src\dom\base\nsFrameLoader.cpp, line 513

19:44:30 INFO - ++DOCSHELL 15B52000 == 16 [pid = 2396] [id = 16]

19:44:30 INFO - ++DOMWINDOW == 45 (15B52400) [pid = 2396] [serial = 45] [outer = 00000000]

19:44:30 INFO - [Child 1148] WARNING: preserving unexpected JS escape sequence: file c:\builds\moz2_slave\try-w32-d-00000000000000000000\build\src\modules\libpref\prefread.cpp, line 393

19:44:30 INFO - ++DOMWINDOW == 46 (15B56000) [pid = 2396] [serial = 46] [outer = 15B52400]

19:44:30 INFO - [2332] WARNING: XPCOM objects created/destroyed from static ctor/dtor: file c:\builds\moz2_slave\try-w32-d-00000000000000000000\build\src\xpcom\base\nsTraceRefcnt.cpp, line 139

19:44:30 INFO - ### XPCOM_MEM_BLOAT_LOG defined -- logging bloat/leaks to c:\docume~1\cltbld~1.t-x\locals~1\temp\tmpmgvouq.mozrunner\runtests_leaks_tab_pid2332.log

19:44:30 INFO - [2332] WARNING: XPCOM objects created/destroyed from static ctor/dtor: file c:\builds\moz2_slave\try-w32-d-00000000000000000000\build\src\xpcom\base\nsTraceRefcnt.cpp, line 139

19:44:30 INFO - [Parent 2396] WARNING: NS_ENSURE_TRUE(mMutable) failed: file c:\builds\moz2_slave\try-w32-d-00000000000000000000\build\src\netwerk\base\nsSimpleURI.cpp, line 265

19:44:30 INFO - [Child 2332] WARNING: '!compMgr', file c:/builds/moz2_slave/try-w32-d-00000000000000000000/build/src/xpcom/glue/nsComponentManagerUtils.cpp, line 63

19:44:30 INFO - [Child 2332] WARNING: NS_ENSURE_TRUE(svc) failed: file c:\builds\moz2_slave\try-w32-d-00000000000000000000\build\src\dom\ipc\nsIContentChild.cpp, line 31

19:44:30 INFO - [Child 2332] WARNING: Error constructing actor PJavaScriptChild: file c:\builds\moz2_slave\try-w32-d-00000000000000000000\build\src\obj-firefox\ipc\ipdl\PContentChild.cpp, line 985

19:44:30 INFO - [Child 2332] WARNING: preserving unexpected JS escape sequence: file c:\builds\moz2_slave\try-w32-d-00000000000000000000\build\src\modules\libpref\prefread.cpp, line 393

19:44:30 INFO - ++DOMWINDOW == 47 (15B59000) [pid = 2396] [serial = 47] [outer = 15B52400]

19:44:30 INFO - [Parent 2396] WARNING: 1 sort operation has occurred for the SQL statement '0xf2f3530'. See https://developer.mozilla.org/En/Storage/Warnings details.: file c:\builds\moz2_slave\try-w32-d-00000000000000000000\build\src\storage\src\mozStoragePrivateHelpers.cpp, line 114

19:44:30 INFO - SystemMessageCache: init

19:44:30 INFO - [Parent 2396] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80004005: file c:\builds\moz2_slave\try-w32-d-00000000000000000000\build\src\dom\workers\ServiceWorkerManager.cpp, line 2369

19:44:30 INFO - [Parent 2396] WARNING: NS_ENSURE_SUCCESS(rv, false) failed with result 0x80004005: file c:\builds\moz2_slave\try-w32-d-00000000000000000000\build\src\netwerk\protocol\http\HttpBaseChannel.cpp, line 1952

19:44:30 INFO - [Parent 2396] WARNING: NS_ENSURE_TRUE(aURI) failed: file c:/builds/moz2_slave/try-w32-d-00000000000000000000/build/src/netwerk/dns/nsEffectiveTLDService.cpp, line 158

19:44:30 INFO - [Parent 2396] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80070057: file c:\builds\moz2_slave\try-w32-d-00000000000000000000\build\src\dom\base\ThirdPartyUtil.cpp, line 353

19:44:30 INFO - SystemMessageCache: init

19:44:30 INFO - JavaScript strict warning: resource://gre/modules/Webapps.jsm, line 1210: ReferenceError: reference to undefined property msg.app

19:44:30 INFO - -*-*- Langpacks: getAdditionalLanguages http://example.org/manifest.webapp

19:44:30 INFO - -*-*- Langpacks: Languages found: ({langs:{}})

19:44:30 INFO - -*-*- Langpacks: getAdditionalLanguages https://example.com/manifest.webapp

19:44:30 INFO - -*-*- Langpacks: Languages found: ({langs:{}})

19:44:30 INFO - -*-*- Langpacks: getAdditionalLanguages http://test1.example.org/manifest.webapp

19:44:30 INFO - -*-*- Langpacks: Languages found: ({langs:{}})

19:44:30 INFO - -*-*- Langpacks: getAdditionalLanguages http://test1.example.org:8000/manifest.webapp

19:44:30 INFO - -*-*- Langpacks: Languages found: ({langs:{}})

19:44:30 INFO - -*-*- Langpacks: getAdditionalLanguages http://sub1.test1.example.org/manifest.webapp

19:44:30 INFO - -*-*- Langpacks: Languages found: ({langs:{}})

19:44:30 INFO - -*-*- Langpacks: getAdditionalLanguages https://example.com/manifest_priv.webapp

19:44:30 INFO - -*-*- Langpacks: Languages found: ({langs:{}})

19:44:30 INFO - -*-*- Langpacks: getAdditionalLanguages https://example.com/manifest_cert.webapp

19:44:30 INFO - -*-*- Langpacks: Languages found: ({langs:{}})

19:44:30 INFO - -*-*- Langpacks: getAdditionalLanguages https://example.com/manifest_csp_cert.webapp

19:44:30 INFO - -*-*- Langpacks: Languages found: ({langs:{}})

19:44:30 INFO - -*-*- Langpacks: getAdditionalLanguages https://example.com/manifest_csp_inst.webapp

19:44:30 INFO - -*-*- Langpacks: Languages found: ({langs:{}})

19:44:30 INFO - -*-*- Langpacks: getAdditionalLanguages https://example.com/manifest_csp_priv.webapp

19:44:30 INFO - -*-*- Langpacks: Languages found: ({langs:{}})

19:44:30 INFO - -*-*- Langpacks: getAdditionalLanguages https://acertified.com/manifest.webapp

19:44:30 INFO - -*-*- Langpacks: Languages found: ({langs:{}})

19:44:30 INFO - -*-*- Langpacks: getAdditionalLanguages https://aprivileged.com/manifest.webapp

19:44:30 INFO - -*-*- Langpacks: Languages found: ({langs:{}})

19:44:30 INFO - -*-*- Langpacks: getAdditionalLanguages http://127.0.0.1:8888/sample.manifest

19:44:30 INFO - -*-*- Langpacks: Languages found: ({langs:{}})

19:44:30 INFO - -*-*- Langpacks: getAdditionalLanguages http://127.0.0.1:8888/sample.manifest

19:44:30 INFO - -*-*- Langpacks: Languages found: ({langs:{}})

19:44:30 INFO - [Child 1148] WARNING: nsWindow::GetNativeData not implemented for this type: file c:\builds\moz2_slave\try-w32-d-00000000000000000000\build\src\widget\PuppetWidget.cpp, line 915

19:44:30 INFO - ++DOCSHELL 082C9400 == 1 [pid = 1148] [id = 1]

19:44:30 INFO - ++DOMWINDOW == 1 (082C9800) [pid = 1148] [serial = 1] [outer = 00000000]

19:44:30 INFO - [Child 1148] WARNING: NS_ENSURE_TRUE(domDoc && target) failed: file c:/builds/moz2_slave/try-w32-d-00000000000000000000/build/src/dom/base/nsContentUtils.cpp, line 3595

19:44:30 INFO - [Child 1148] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80070057: file c:/builds/moz2_slave/try-w32-d-00000000000000000000/build/src/dom/base/nsContentUtils.cpp, line 3650