Question about LED

Hi, this is not realy a issu,
it is just a question.
I use your Firmware on CC2538-CC2592 Chip with pcb from https://modkam.ru/?p=1112#more-1112.
i had flashed MODKAMRU_V3_USB.hex.
The pcb had led connected to PB1..4.
in your patchfile i see you als defined leds on this ports. After plug to my control system led on PB1 to 3 lights up. about 1 second later only PB1 lights up.
now, when i control zigbee device no other led lights up. So my question is, is this the normal behavior for now?
Anyway i'm not sure at the moment if your firmware uses PB1..4 oder PB0..3?
Tank you

Anyway i'm not sure at the moment if your firmware uses PB1..4 oder PB0..3?
All 4 LEDs shoud blink on the startup. Afterward, only 3 are actualy used:

PB1 - power on;
PB3 - permit join mode;
PB4 - ZNP events.
I gues, you shoud check the connection of the PB4 LED.

Anyway i'm not sure at the moment if your firmware uses PB1..4 oder PB0..3?

All 4 LEDs shoud blink on the startup. Afterward, only 3 are actualy used:

PB1 - power on;
PB3 - permit join mode;
PB4 - ZNP events.

I gues, you shoud check the connection of the PB4 LED.

Ok tank you. Now i see also PB4 light up at startup. But the time is shorter than the other 3.
So i'm very new to ZStack.
permit join mode means the stick is ready for pairing new devices?
ZNP Event is sending or receiving zigbee telegramm?

permit join mode means the stick is ready for pairing new devices?

Correct.

ZNP Event is sending or receiving zigbee telegramm?

Any RF or serial port communication.

ok, now i got it.
Realy nice work. tank you


Add proper footer and header

We could replicate the header & footers links that we have on the kubefirst documentation but for colony.
It was empty at first caused when the docs was released, we didn't do the move to konstruct yet nor had the colony Twitter account.

Sorry I am all up in your tickets asking 8000 questions. 🥳
Are you thinking like a global Konstruct header that could work for all of the various docs urls?

Are you thinking like a global Konstruct header that could work for all of the various docs urls?

If by "docs URLs" you mean all our docs no matter the product, I don't think it will work because of color scheme and logos, and links are different from one project to the other. If you mean, on all pages an user can check in the colony docs, yes, but header + footer in Docusaurus are already like that.
Just so we are clear on terminology, this is the Docusaurus header on Colony:

And this is the footer:

Which are both empty compared to kubefirst ones. With that said, the kubefirst ones are much sexy either, but at least they good the proper information for other mediums we have.

Sorry I am all up in your tickets asking 8000 questions. 🥳

Never apologize for that :)

I think we need to also try to sort out a way to add Konstruct logos somewhere so it's obvious that even if a user is in colony or Kubefirst it also has Konstruct branding.

I think we need to also try to sort out a way to add Konstruct logos somewhere so it's obvious that even if a user is in colony or Kubefirst it also has Konstruct branding.

I created this issue since it may not be done at the same time, or in the footer/header.

Footer has been updated, header will be updated once designs with branding has been resolved.
https://linear.app/konstruct/issue/UXD-7/docs-design-konstruct-branding-for-product-pages


Rename "brigade-gw" to "brigade-github-gw"

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: danillouz sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


Speed up build-containers job

This will now only build one container to test the docker setup.


Commercial Use

Hey! Can I use Open Sora for commercial use?

It's under the Apache 2.0 license so you can go read it


Improve inferrability

The inner functions are surprisingly problematic, best to make them standalone.

Codecov Report

Merging #4 into master will decrease coverage by 17.16%.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##           master       #4       +/-   ##
===========================================
- Coverage   97.76%   80.59%   -17.17%
===========================================
  Files           4        4
  Lines         134      134
===========================================
- Hits          131      108       -23
- Misses          3       26       +23

Impacted Files
Coverage Δ

src/treemenu.jl
100.00% <ø> (ø)

src/foldingtree.jl
56.66% <100.00%> (-38.34%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f98bc49...f3af6dd. Read the comment docs.


Add raw access to a single document

In gitlab by @sylvaingaudan on Apr 26, 2017, 14:00
add a raw access to the data:
/arlas/collection/{identifier}
Returns an ArlasHit

In gitlab by @sylvaingaudan on May 17, 2017, 09:59
added ~1918909 and removed ~1918908 labels

In gitlab by @sfalquier on Jun 1, 2017, 11:03
added ~1918368 and removed ~1918909 labels

In gitlab by @sfalquier on Jun 1, 2017, 11:04
assigned to @sfalquier

In gitlab by @sfalquier on Jun 1, 2017, 15:42
mentioned in commit e02d7004d35de2d1d2b0472c6a2d52e0667bdcca

In gitlab by @sfalquier on Jun 1, 2017, 15:56
mentioned in merge request !59

In gitlab by @sfalquier on Jun 7, 2017, 13:36
mentioned in commit ff5cc040e2907aed86bdcdd857f658aadfbff76b

In gitlab by @sfalquier on Jun 7, 2017, 14:02
mentioned in commit 7b0f4f3b1af9cf1aabbcc49371644dd246c2d588

In gitlab by @MohamedHamouGisaia on Jun 8, 2017, 09:39
mentioned in commit 8ba75d7a2684b850c0772ac472607e6167482c1f

In gitlab by @MohamedHamouGisaia on Jun 8, 2017, 10:15
closed

In gitlab by @MohamedHamouGisaia on Jun 8, 2017, 10:15
removed ~1918368 label


k9mail frequently crashes when deleting mail (bulk or individual) IMAP Inbox: BUG

K9 mail client crashes almost every time I delete mail from my inbox.
Especially bulk delete.
I've done the basic things one would do to fix.
Reinstall, delete and recreate account.
There is plenty of disc space on my device.
I've enabled debugging.
I don't know what else to do.
Can you help please?

What K-9 Mail version are you using? Can you provide the debug log?
Could it be the same as issue #814?

Did just see that this bug was mentioned in my bug report => Attached a crash report to Bug #814.


https://<IP_ADDRESS>/

https://<IP_ADDRESS>/

https://<IP_ADDRESS>/app/game?game=6904189484201


TokenMismatchException in Microsoft edge browser

The edge browser throws error on TokenMismatchException on form submission.

Duplicate of the internet explorer issues.

Using a session driver like redis and making sure there are no 404 requests on page load usually fix this.


Problem: AIP creation shows too many jobs in the Enduro UI that shouldn't be visible

Describe the bug
When running an AIP creation workflow, Archivematica currently displays 124 tasks. Many of these are for decisions the user is not prompted to make (e.g. "Run File format identification?") based on the configuration, or as steps that a user does not need to see (e.g. "Moving to processing directory"). This clutters the interface and makes it harder for operators to easily glean useful workflow information.
To Reproduce
Steps to reproduce the behavior:

Start an ingest
Wait for it to complete processing
Examine the result on the package details page of the related SIP

Expected behavior
Operators should only see key tasks shown in the Enduro UI when running preservation workflows, that impart useful information on what tasks are currently being performed.
Additional context
This issue is the first in what will hopefully be a multi-part development effort, to eventually add a more curated and customized list of tasks in a preservation / AIP creation workflow, with useful outcome information (and/or error information) shown for each. New issues will be filed for future increments.
Analysis work on this topic is being done in this spreadsheet:

https://docs.google.com/spreadsheets/d/10jByMHs6kTl4B22VCnwlEd2tLPb5tdJ595wZg-J96ts/edit#gid=1083719011

For now / for this issue, any row marked in red in the "AM-Enduro list (2024)" sheet should be prevented from displaying in the Enduro user interface.

Note: even though this isn't really specific to error messages, it's a precursor to error message work that we will do in future sprints.

Radda discussed some of my questions earlier. Overall I think this  feature looks good and can go out in the next SFA release for their feedback, but we agreed to leave this ticket open for @sallain's input

These are the AM jobs we keep:
https://github.com/artefactual-sdps/enduro/blob/main/internal/am/job_tracker.go#L16-L59
We filter the jobs we get from AM creating preservation tasks only for the jobs in that list. The jobs we get depends on the processing configuration used in AM, I was using the default automated processing config, so there may be a few discrepancies on what I was able to see.
We should probably consider as much processing options and jobs as we can, so let me know if something is missing and what changes should I do in the processing configuration to see those jobs.

Looking beautiful


Release 2024-05-04

:tada: This PR is included in version 1.9.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Adding a tap gesture

Hi,
it is a really beautiful widget you have done !
How can I add a tap gesture on the circle, in addition to sliding move ?
Thanks !

Hi @nahouto
Does that mean you want to change the thumb location and value in the tap event ?
Or you want to get the event, which is tap on the circumference ? Such as, TouchUpInside.

I would like to change the thumb location and value in the tap event.
For now, I created 12 buttons all around the circleSlider to simulate a tap event.

@nahouto
I see. It had better had its function.
Now is not its function. It is scheduled to correspond to the future.
Thank you for the advice.


tests/ampi/privatization fails on mpi-linux-x86_64-smp autobuild

Original issue: https://charm.cs.illinois.edu/redmine/issues/1858

Autobuild runs this test on the 'courtesy' lab machine:
$ ./build LIBS mpi-linux-x86_64 smp -j16 --with-production --enable-error-checking -g

$ cd mpi-linux-x86_64-smp/tests/ampi/privatization

$ make test OPTS=-g TESTOPTS=+isomalloc_sync

../../../bin/testrun  ./tlsglobals +p1 +vp2  +isomalloc_sync

Running on 1 processors:  ./tlsglobals +vp2 +isomalloc_sync
charmrun>  /usr/bin/setarch x86_64 -R  mpirun -np 1  ./tlsglobals +vp2 +isomalloc_sync
Charm++> Running on MPI version: 2.1
Charm++> level of thread support used: MPI_THREAD_FUNNELED (desired: MPI_THREAD_FUNNELED)
Charm++> Running in SMP mode: numNodes 1,  1 worker threads per process
Charm++> The comm. thread both sends and receives messages
Converse/Charm++ Commit ID: 3c99e0d
Charm++> Synchronizing isomalloc memory region...
Charm++> Consolidated Isomalloc memory region: 0x2ab700000000 - 0x6aa900000000 (67051520 MB).
CharmLB> Load balancer assumes all CPUs are same.
Charm++> Running on 1 unique compute nodes (8-way SMP).
Charm++> cpu topology info is gathered in 0.000 seconds.
CharmLB> RotateLB created.
------------- Processor 1 Exiting: Called CmiAbort ------------
Reason: TCHARM_Get_num_chunks should only be called on PE 0 during setup!
[1] Stack Traceback:
  [1:0] CmiAbortHelper+0x5a  [0x61c20a]
  [1:1]   [0x61c28d]
  [1:2] TCHARM_Get_num_chunks+0x1fe  [0x4cd44e]
  [1:3] _Z14ampiCreateMainPFviPPcEPKci+0x16  [0x4e1bb6]
  [1:4] _ZN18CkIndex_TCharmMain25_call_TCharmMain_CkArgMsgEPvS0_+0x34  [0x543734]
  [1:5] _Z10_initCharmiPPc+0x2869  [0x54b7a9]
  [1:6]   [0x61f236]
  [1:7]   [0x61f3c8]
  [1:8] +0x8184  [0x2aaaaaeda184]
  [1:9] clone+0x6d  [0x2aaaabfa903d]

Original date: 2018-04-12 14:21:54

Also hanging on multicore-linux-x86_64

Original date: 2018-04-13 18:19:20

Can also happen with netlrts-linux-x86_64-smp:
$ ./tlsglobals +vp2 +p2
Charm++: standalone mode (not using charmrun)
Charm++> Running in SMP mode: numNodes 1,  2 worker threads per process
Charm++> The comm. thread both sends and receives messages
Converse/Charm++ Commit ID: v6.8.2-545-gaa6ff5636
Charm++> scheduler running in netpoll mode.
CharmLB> Load balancer assumes all CPUs are same.
Charm++> Running on 1 unique compute nodes (8-way SMP).
Charm++> cpu topology info is gathered in 0.000 seconds.
CharmLB> RotateLB created.
------------- Processor 1 Exiting: Called CmiAbort ------------
Reason: TCHARM_Get_num_chunks should only be called on PE 0 during setup!
[1] Stack Traceback:
  [1:0] CmiAbortHelper+0xbe  [0x7822f1]
  [1:1] CmiGetNonLocal+0  [0x78232a]
  [1:2] TCHARM_Get_num_chunks+0x3c  [0x5efb0b]
  [1:3] _Z14ampiCreateMainPFviPPcEPKci+0x19  [0x603cfc]
  [1:4] AMPI_Setup_Switch+0x37  [0x6035b9]
  [1:5] TCHARM_Call_fallback_setup+0x19  [0x5efac1]
  [1:6] TCHARM_User_setup+0x9  [0x8120fd]
  [1:7] tcharm_user_setup_+0x9  [0x8120ea]
  [1:8] _ZN10TCharmMainC2EP8CkArgMsg+0x3f  [0x670731]
  [1:9] _ZN18CkIndex_TCharmMain25_call_TCharmMain_CkArgMsgEPvS0_+0x40  [0x67045e]
  [1:10] _Z10_initCharmiPPc+0x1f59  [0x6797c0]
  [1:11]   [0x78205a]
  [1:12] ConverseInit+0x73e  [0x781d18]
  [1:13] main+0x3f  [0x670a6c]
  [1:14] __libc_start_main+0xf0  [0x7ffff6d6d830]
  [1:15] _start+0x29  [0x5ed0a9]
Charm++ fatal error:
TCHARM_Get_num_chunks should only be called on PE 0 during setup!
[1] Stack Traceback:
  [1:0]   [0x7830e5]
  [1:1] LrtsAbort+0x6f  [0x782a8a]
  [1:2] CmiAbort+0  [0x7822fd]
  [1:3] CmiGetNonLocal+0  [0x78232a]
  [1:4] TCHARM_Get_num_chunks+0x3c  [0x5efb0b]
  [1:5] _Z14ampiCreateMainPFviPPcEPKci+0x19  [0x603cfc]
  [1:6] AMPI_Setup_Switch+0x37  [0x6035b9]
  [1:7] TCHARM_Call_fallback_setup+0x19  [0x5efac1]
  [1:8] TCHARM_User_setup+0x9  [0x8120fd]
  [1:9] tcharm_user_setup_+0x9  [0x8120ea]
  [1:10] _ZN10TCharmMainC2EP8CkArgMsg+0x3f  [0x670731]
  [1:11] _ZN18CkIndex_TCharmMain25_call_TCharmMain_CkArgMsgEPvS0_+0x40  [0x67045e]
  [1:12] _Z10_initCharmiPPc+0x1f59  [0x6797c0]
  [1:13]   [0x78205a]
  [1:14] ConverseInit+0x73e  [0x781d18]
  [1:15] main+0x3f  [0x670a6c]
  [1:16] __libc_start_main+0xf0  [0x7ffff6d6d830]
  [1:17] _start+0x29  [0x5ed0a9]
Aborted (core dumped)

Original date: 2018-04-15 01:26:21

Notes:
The weird part is that printing out the PE # inside ampiCreateMain shows the call is from PE 0, but inside TCharm_Get_num_chunks() it thinks it's on PE 1...
By removing the TCHARMAPI() call from TCHARM_Get_num_chunks(), we get further into the startup process but then fail in TCHARM_Create_data(). Doing the same there, we get further again but fail still...
TCHARMAPI() is defined at the bottom of src/libs/ck-libs/tcharm/tcharm_impl.h and is responsible for switching the TLS pointer when context switching between user-level threads. I think we may need to handle the case where TLS isn't initialized yet (it is initialized after the first couple calls into TCHARM_* routines). I'm not sure, but there's something wrong with TCHARMAPI...
Note that all AMPI routines also call AMPI_API() macro to do the same thing (this macro does the same thing as TCHARMAPI, so the problem could be in AMPI too.
This is the furthest I was able to get things:
Thread 3 "tlsglobals" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffebfff700 (LWP 7589)]
0x00000000006027fa in CthBaseResume (t=0x7fffe4159fb0) at libthreads-default-tls.c:761
761	  CpvAccess(_numSwitches)++;
(gdb) bt
#0  0x00000000006027fa in CthBaseResume (t=0x7fffe4159fb0) at libthreads-default-tls.c:761
#1  0x000000000060339c in CthResume (t=0x7fffe4159fb0) at libthreads-default-tls.c:1782
#2  0x0000000000609e58 in CsdScheduleForever () at convcore.c:1901
#3  0x000000000060a0ed in CsdScheduler (maxmsgs=maxmsgs`entry=-1) at convcore.c:1837
#4  0x00000000006080aa in ConverseRunPE (everReturn=everReturn`entry=0) at machine-common-core.c:1531
#5  0x0000000000608115 in call_startfn (vindex=0x0) at machine-smp.c:414
#6  0x00007ffff79bf6ba in start_thread (arg=0x7fffebfff700) at pthread_create.c:333
#7  0x00007ffff696341d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Original date: 2018-04-18 00:26:32

This may be the fix: ~~https://charm.cs.illinois.edu/gerrit/4019~~ https://github.com/UIUC-PPL/charm/commit/c21713633d855e77129356789a42cc475aa76353

Original date: 2018-04-18 07:22:11

That fixes the failures on multicore-linux-x86_64. I believe mpi-linux-x86_64-smp failures were the same.

Original date: 2018-04-18 07:38:06

mpi-linux-x86_64-smp is still failing with the patch for './tlsglobals +vp2':
Charm++> -tlsglobals enabled for privatization of thread-local variables.

Thread 3 "tlsglobals" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffebfff700 (LWP 15259)]
0x000000000060c05a in CthBaseResume (t=0x7fffe4159840) at libthreads-default-tls.c:762
762	  CpvAccess(_numSwitches)++;
(gdb) bt
#0  0x000000000060c05a in CthBaseResume (t=0x7fffe4159840) at libthreads-default-tls.c:762
#1  0x000000000060ccbc in CthResume (t=0x7fffe4159840) at libthreads-default-tls.c:1720
#2  0x0000000000613b38 in CsdScheduleForever () at convcore.c:1902
#3  0x0000000000613dcd in CsdScheduler (maxmsgs=maxmsgs`entry=-1) at convcore.c:1838
#4  0x0000000000611cea in ConverseRunPE (everReturn=everReturn`entry=0) at machine-common-core.c:1535
#5  0x0000000000611d55 in call_startfn (vindex=0x0) at machine-smp.c:414
#6  0x00007ffff79bf6ba in start_thread (arg=0x7fffebfff700) at pthread_create.c:333
#7  0x00007ffff696341d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

It even fails the same with +vp1.

Original date: 2018-04-18 16:51:53

Removing the unused _numSwitches variable still results in a crash:
Starting program: /home/evan/charm/mpi-linux-x86_64-smp/tests/ampi/privatization/tlsglobals
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff41b0700 (LWP 18712)]
[New Thread 0x7ffff39af700 (LWP 18713)]
Charm++> Running on MPI version: 3.1
Charm++> level of thread support used: MPI_THREAD_FUNNELED (desired: MPI_THREAD_FUNNELED)
Charm++> Running in SMP mode: numNodes 1,  1 worker threads per process
Charm++> The comm. thread both sends and receives messages
[New Thread 0x7fffe9098700 (LWP 18714)]
Converse/Charm++ Commit ID: v6.8.2-592-g45f0fd545
CharmLB> Load balancer assumes all CPUs are same.
Charm++> Running on 1 unique compute nodes (12-way SMP).
Charm++> cpu topology info is gathered in 0.000 seconds.
CharmLB> RotateLB created.
Charm++> -tlsglobals enabled for privatization of thread-local variables.

Thread 4 "tlsglobals" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffe9098700 (LWP 18714)]
0x00005555558f729a in CthFixData (t=0x7fffdc166c70) at libthreads-default-tls.c:350
350	  size_t newsize = CthCpvAccess(CthDatasize);
(gdb) p CthCpvAccess(CthDatasize)
$1 = 56

I compiled with -g3 so that macro information would be included with debug symbols.

Original date: 2018-04-18 16:58:04

Yeah removing the stuff I pointed out on the patch doesn't fix this, but we could still remove it separately.

Original date: 2018-04-18 17:02:41

My patch fixes the TCharm crash on netlrts-linux-x86_64-smp in addition to multicore, but that build also exhibits a new problem:
$ ./charmrun ++local +n2 ++ppn 1 ./tlsglobals +vp2
Charmrun> scalable start enabled.
Charmrun> started all node programs in 0.033 seconds.
Charm++> Running in SMP mode: numNodes 2,  1 worker threads per process
Charm++> The comm. thread both sends and receives messages
Converse/Charm++ Commit ID: v6.8.2-592-g45f0fd545
Warning> Randomization of virtual memory (ASLR) is turned on in the kernel, thread migration may not work! Run 'echo 0 > /proc/sys/kernel/randomize_va_space' as root to disable it, or try running with '+isomalloc_sync'.
Charm++> scheduler running in netpoll mode.
CharmLB> Load balancer assumes all CPUs are same.
Charm++> Running on 1 unique compute nodes (12-way SMP).
Charm++> cpu topology info is gathered in 0.001 seconds.
CharmLB> RotateLB created.
Charm++> -tlsglobals enabled for privatization of thread-local variables.
------------- Processor 0 Exiting: Called CmiAbort ------------
Reason: AMPI> The application provided a custom AMPI_Setup() method, but AMPI is built with shared library support. This is an unsupported configuration. Please recompile charm++/AMPI without `-build-shared` or remove the AMPI_Setup() function from your application.

[0] Stack Traceback:
  [0:0] CmiAbortHelper+0xc4  [0x55be65eb19a7]
  [0:1] CmiGetNonLocal+0  [0x55be65eb19e2]
  [0:2] _ZN17MPI_threadstart_t5startEv+0x75  [0x55be65d73de3]
  [0:3] AMPI_threadstart+0x37  [0x55be65d31c04]
  [0:4] +0x1f7c41  [0x55be65d1bc41]
  [0:5] CthStartThread+0x58  [0x55be65ead943]
  [0:6] make_fcontext+0x2f  [0x55be65eadddf]
Fatal error on PE 0> AMPI> The application provided a custom AMPI_Setup() method, but AMPI is built with shared library support. This is an unsupported configuration. Please recompile charm++/AMPI without `-build-shared` or remove the AMPI_Setup() function from your application.

If +n is set to 1, ++ppn is increased above 1, or +vp is set to 1, this problem does not occur.

Original date: 2018-04-18 17:24:39

The netlrts issue seems to be that AMPI_Setup_Switch is executed on a process other than the one containing PE 0.
EDIT: Upon closer inspection, more than one process thinks it has/is PE 0.

Original date: 2018-04-18 17:37:41

_numSwitches removal: ~~https://charm.cs.illinois.edu/gerrit/4025~~ https://github.com/UIUC-PPL/charm/commit/ea3bc43e9d3f2c66bf1e3cf33ea14529588781b3
netlrts-linux-x86_64-smp workaround: ~~https://charm.cs.illinois.edu/gerrit/4026~~ https://github.com/UIUC-PPL/charm/commit/d1507d4a1c364bb3ca3f27e0193d2355e984d791

Original date: 2018-04-18 17:54:19

$ valgrind -- ./tlsglobals
==1917== Memcheck, a memory error detector
==1917== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==1917== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==1917== Command: ./tlsglobals
==1917==
==1917== Conditional jump or move depends on uninitialised value(s)
==1917==    at 0x66F2C36: opal_value_unload (in /usr/lib/x86_64-linux-gnu/openmpi/lib/libopen-pal.so.20.10.1)
==1917==    by 0x580F3FA: ompi_proc_complete_init (in /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so.20.10.1)
==1917==    by 0x5813598: ompi_mpi_init (in /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so.20.10.1)
==1917==    by 0x5835B44: PMPI_Init_thread (in /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so.20.10.1)
==1917==    by 0x4B35AC: LrtsInit (machine.c:1440)
==1917==    by 0x4B0FB9: ConverseInit (machine-common-core.c:1286)
==1917==    by 0x392A5C: main (main.C:9)
==1917==
Charm++> Running on MPI version: 3.1
Charm++> level of thread support used: MPI_THREAD_FUNNELED (desired: MPI_THREAD_FUNNELED)
Charm++> Running in SMP mode: numNodes 1,  1 worker threads per process
Charm++> The comm. thread both sends and receives messages
Converse/Charm++ Commit ID: v6.8.2-593-geeb7b8fca
Warning> Randomization of virtual memory (ASLR) is turned on in the kernel, thread migration may not work! Run 'echo 0 > /proc/sys/kernel/randomize_va_space' as root to disable it, or try running with '+isomalloc_sync'.
CharmLB> Load balancer assumes all CPUs are same.
Charm++> Running on 1 unique compute nodes (12-way SMP).
Charm++> cpu topology info is gathered in 0.056 seconds.
CharmLB> RotateLB created.
Charm++> -tlsglobals enabled for privatization of thread-local variables.
==1917== Thread 4:
==1917== Invalid read of size 8
==1917==    at 0x4AB2A8: CthFixData (libthreads-default-tls.c:350)
==1917==    by 0x4AC398: CthBaseResume (libthreads-default-tls.c:758)
==1917==    by 0x4ACABC: CthResume (libthreads-default-tls.c:1715)
==1917==    by 0x4B6FC6: CthResumeNormalThread (convcore.c:2110)
==1917==    by 0x4B672D: CmiHandleMessage (convcore.c:1654)
==1917==    by 0x4B6B29: CsdScheduleForever (convcore.c:1902)
==1917==    by 0x4B6A44: CsdScheduler (convcore.c:1838)
==1917==    by 0x4B1812: ConverseRunPE (machine-common-core.c:1535)
==1917==    by 0x4ADDFD: call_startfn (machine-smp.c:414)
==1917==    by 0x50457FB: start_thread (pthread_create.c:465)
==1917==    by 0x616FB5E: clone (clone.S:95)
==1917==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==1917==
==1917== Invalid read of size 4
==1917==    at 0x4C2B5C: CpdAborting (debug-conv.c:368)
==1917==    by 0x4B1A40: CmiAbortHelper (machine-common-core.c:1670)
==1917==    by 0x4B3458: KillOnAllSigs (machine.c:1360)
==1917==    by 0x505114F: ??? (in /lib/x86_64-linux-gnu/libpthread-2.26.so)
==1917==    by 0x4AB2A7: CthFixData (libthreads-default-tls.c:350)
==1917==    by 0x4AC398: CthBaseResume (libthreads-default-tls.c:758)
==1917==    by 0x4ACABC: CthResume (libthreads-default-tls.c:1715)
==1917==    by 0x4B6FC6: CthResumeNormalThread (convcore.c:2110)
==1917==    by 0x4B672D: CmiHandleMessage (convcore.c:1654)
==1917==    by 0x4B6B29: CsdScheduleForever (convcore.c:1902)
==1917==    by 0x4B6A44: CsdScheduler (convcore.c:1838)
==1917==    by 0x4B1812: ConverseRunPE (machine-common-core.c:1535)
==1917==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==1917==
==1917==
==1917== Process terminating with default action of signal 11 (SIGSEGV)
==1917==  Access not within mapped region at address 0x0
==1917==    at 0x4C2B5C: CpdAborting (debug-conv.c:368)
==1917==    by 0x4B1A40: CmiAbortHelper (machine-common-core.c:1670)
==1917==    by 0x4B3458: KillOnAllSigs (machine.c:1360)
==1917==    by 0x505114F: ??? (in /lib/x86_64-linux-gnu/libpthread-2.26.so)
==1917==    by 0x4AB2A7: CthFixData (libthreads-default-tls.c:350)
==1917==    by 0x4AC398: CthBaseResume (libthreads-default-tls.c:758)
==1917==    by 0x4ACABC: CthResume (libthreads-default-tls.c:1715)
==1917==    by 0x4B6FC6: CthResumeNormalThread (convcore.c:2110)
==1917==    by 0x4B672D: CmiHandleMessage (convcore.c:1654)
==1917==    by 0x4B6B29: CsdScheduleForever (convcore.c:1902)
==1917==    by 0x4B6A44: CsdScheduler (convcore.c:1838)
==1917==    by 0x4B1812: ConverseRunPE (machine-common-core.c:1535)
==1917==  If you believe this happened as a result of a stack
==1917==  overflow in your program's main thread (unlikely but
==1917==  possible), you can try to increase the size of the
==1917==  main thread stack using the --main-stacksize= flag.
==1917==  The main thread stack size used in this run was 8388608.
==1917==
==1917== HEAP SUMMARY:
==1917==     in use at exit: 4,705,476 bytes in 14,366 blocks
==1917==   total heap usage: 27,194 allocs, 12,828 frees, 12,906,096 bytes allocated
==1917==
==1917== LEAK SUMMARY:
==1917==    definitely lost: 8,585 bytes in 71 blocks
==1917==    indirectly lost: 391 bytes in 13 blocks
==1917==      possibly lost: 6,088 bytes in 98 blocks
==1917==    still reachable: 4,690,412 bytes in 14,184 blocks
==1917==                       of which reachable via heuristic:
==1917==                         newarray           : 880,040 bytes in 2 blocks
==1917==         suppressed: 0 bytes in 0 blocks
==1917== Rerun with --leak-check=full to see details of leaked memory
==1917==
==1917== For counts of detected and suppressed errors, rerun with: -v
==1917== Use --track-origins=yes to see where uninitialised values come from
==1917== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 0 from 0)
Segmentation fault (core dumped)

Original date: 2018-04-18 22:26:02

Does running with "+isomalloc_sync" help, and/or with valgrind and "--leak-check=full --track-origins=yes"?

Original date: 2018-04-18 23:15:23

Does running with "+isomalloc_sync" help

No difference

and/or with valgrind and "--leak-check=full --track-origins=yes"?

Nothing relevant to the issue

Original date: 2018-04-19 23:59:14

We should check if -tlsglobals ever worked on mpi-linux-x86_64-smp in any way

Original date: 2018-04-20 00:28:53

If the MPI library is itself using TLS (some MPI libraries use threads internally) will our ‘-tlsglobals’ runtime switching of the TLS pointer interfere with it? I’m not sure why it would be different for non-SMP though, if so

Original date: 2018-04-24 18:41:05

Evan cleaned up some of the valgrind output here: ~~https://charm.cs.illinois.edu/gerrit/#/c/charm/+/4046/~~ https://github.com/UIUC-PPL/charm/commit/cd4c8be457f590791129eb51536eacca622fa2c6

Original date: 2018-04-24 22:42:43

I ran the tlsglobals test with TSan. There is a lot of output that seems to have the same few root causes but I'm not sure where to go with it.

Original date: 2018-04-24 22:54:07

Hmm, yeah that's pretty opaque. Most of it seems to be from inside "mca_" which is a low-level component of OpenMPI. Could you run tsan on an MPI (not AMPI) program that just calls MPI_Init() and MPI_Finalize() to compare?

Original date: 2018-04-24 23:06:47

This log seems to be the beginning portion of the tlsglobals log, before any Converse code is run.

Original date: 2018-04-24 23:27:25

Here's the AMPI-related things in the mpi-linux-x86_64-smp tsan log you posted above:
==================
WARNING: ThreadSanitizer: data race (pid=19329)
  Read of size 4 at 0x55d6c100702c by thread T3:
    #0 ampiProcInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/libs/ck-libs/ampi/ampi.C:957 (tlsglobals+0x000000228db4)
    #1 InitCallTable::enumerateInitCalls() /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/init.C:1037 (tlsglobals+0x0000002d8618)
    #2 _initCharm(int, char**) /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/init.C:1317 (tlsglobals+0x0000002dbae7)
    #3 ConverseRunPE /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1534 (tlsglobals+0x00000048e058)
    #4 call_startfn /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-smp.c:414 (tlsglobals+0x000000487b41)
    #5 <null> <null> (libtsan.so.0+0x000000025aab)
  Previous write of size 4 at 0x55d6c100702c by main thread:
    #0 ampiProcInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/libs/ck-libs/ampi/ampi.C:957 (tlsglobals+0x000000228ddd)
    #1 InitCallTable::enumerateInitCalls() /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/init.C:1037 (tlsglobals+0x0000002d8618)
    #2 _initCharm(int, char**) /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/init.C:1317 (tlsglobals+0x0000002dbae7)
    #3 ConverseRunPE /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1528 (tlsglobals+0x00000048dffe)
    #4 ConverseInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1430 (tlsglobals+0x00000048d932)
    #5 main /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/main.C:9 (tlsglobals+0x0000002cb286)

  Location is global 'Csv_CtvOffsampiPtr_' of size 4 at 0x55d6c100702c (tlsglobals+0x0000008a702c)

  Thread T3 (tid=19339, running) created by main thread at:
    #0 pthread_create <null> (libtsan.so.0+0x0000000290c3)
    #1 CmiStartThreads /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-smp.c:506 (tlsglobals+0x000000487e4f)
    #2 ConverseInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1428 (tlsglobals+0x00000048d928)
    #3 main /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/main.C:9 (tlsglobals+0x0000002cb286)
SUMMARY: ThreadSanitizer: data race /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/libs/ck-libs/ampi/ampi.C:957 in ampiProcInit
==================
==================
WARNING: ThreadSanitizer: data race (pid=19329)
  Read of size 4 at 0x55d6c1007030 by thread T3:
    #0 ampiProcInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/libs/ck-libs/ampi/ampi.C:958 (tlsglobals+0x000000228e1d)
    #1 InitCallTable::enumerateInitCalls() /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/init.C:1037 (tlsglobals+0x0000002d8618)
    #2 _initCharm(int, char**) /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/init.C:1317 (tlsglobals+0x0000002dbae7)
    #3 ConverseRunPE /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1534 (tlsglobals+0x00000048e058)
    #4 call_startfn /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-smp.c:414 (tlsglobals+0x000000487b41)
    #5 <null> <null> (libtsan.so.0+0x000000025aab)
  Previous write of size 4 at 0x55d6c1007030 by main thread:
    #0 ampiProcInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/libs/ck-libs/ampi/ampi.C:958 (tlsglobals+0x000000228e46)
    #1 InitCallTable::enumerateInitCalls() /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/init.C:1037 (tlsglobals+0x0000002d8618)
    #2 _initCharm(int, char**) /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/init.C:1317 (tlsglobals+0x0000002dbae7)
    #3 ConverseRunPE /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1528 (tlsglobals+0x00000048dffe)
    #4 ConverseInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1430 (tlsglobals+0x00000048d932)
    #5 main /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/main.C:9 (tlsglobals+0x0000002cb286)

  Location is global 'Csv_CtvOffsampiInitDone_' of size 4 at 0x55d6c1007030 (tlsglobals+0x0000008a7030)

  Thread T3 (tid=19339, running) created by main thread at:
    #0 pthread_create <null> (libtsan.so.0+0x0000000290c3)
    #1 CmiStartThreads /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-smp.c:506 (tlsglobals+0x000000487e4f)
    #2 ConverseInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1428 (tlsglobals+0x00000048d928)
    #3 main /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/main.C:9 (tlsglobals+0x0000002cb286)
SUMMARY: ThreadSanitizer: data race /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/libs/ck-libs/ampi/ampi.C:958 in ampiProcInit
==================
==================
WARNING: ThreadSanitizer: data race (pid=19329)
  Read of size 4 at 0x55d6c1007038 by thread T3:
    #0 ampiProcInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/libs/ck-libs/ampi/ampi.C:959 (tlsglobals+0x000000228e86)
    #1 InitCallTable::enumerateInitCalls() /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/init.C:1037 (tlsglobals+0x0000002d8618)
    #2 _initCharm(int, char**) /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/init.C:1317 (tlsglobals+0x0000002dbae7)
    #3 ConverseRunPE /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1534 (tlsglobals+0x00000048e058)
    #4 call_startfn /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-smp.c:414 (tlsglobals+0x000000487b41)
    #5 <null> <null> (libtsan.so.0+0x000000025aab)
  Previous write of size 4 at 0x55d6c1007038 by main thread:
    #0 ampiProcInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/libs/ck-libs/ampi/ampi.C:959 (tlsglobals+0x000000228eaf)
    #1 InitCallTable::enumerateInitCalls() /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/init.C:1037 (tlsglobals+0x0000002d8618)
    #2 _initCharm(int, char**) /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/init.C:1317 (tlsglobals+0x0000002dbae7)
    #3 ConverseRunPE /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1528 (tlsglobals+0x00000048dffe)
    #4 ConverseInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1430 (tlsglobals+0x00000048d932)
    #5 main /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/main.C:9 (tlsglobals+0x0000002cb286)

  Location is global 'Csv_CtvOffsampiFinalized_' of size 4 at 0x55d6c1007038 (tlsglobals+0x0000008a7038)

  Thread T3 (tid=19339, running) created by main thread at:
    #0 pthread_create <null> (libtsan.so.0+0x0000000290c3)
    #1 CmiStartThreads /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-smp.c:506 (tlsglobals+0x000000487e4f)
    #2 ConverseInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1428 (tlsglobals+0x00000048d928)
    #3 main /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/main.C:9 (tlsglobals+0x0000002cb286)
SUMMARY: ThreadSanitizer: data race /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/libs/ck-libs/ampi/ampi.C:959 in ampiProcInit
==================
==================
WARNING: ThreadSanitizer: data race (pid=19329)
  Read of size 4 at 0x55d6c1007034 by thread T3:
    #0 ampiProcInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/libs/ck-libs/ampi/ampi.C:960 (tlsglobals+0x000000228eef)
    #1 InitCallTable::enumerateInitCalls() /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/init.C:1037 (tlsglobals+0x0000002d8618)
    #2 _initCharm(int, char**) /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/init.C:1317 (tlsglobals+0x0000002dbae7)
    #3 ConverseRunPE /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1534 (tlsglobals+0x00000048e058)
    #4 call_startfn /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-smp.c:414 (tlsglobals+0x000000487b41)
    #5 <null> <null> (libtsan.so.0+0x000000025aab)
  Previous write of size 4 at 0x55d6c1007034 by main thread:
    #0 ampiProcInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/libs/ck-libs/ampi/ampi.C:960 (tlsglobals+0x000000228f18)
    #1 InitCallTable::enumerateInitCalls() /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/init.C:1037 (tlsglobals+0x0000002d8618)
    #2 _initCharm(int, char**) /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/init.C:1317 (tlsglobals+0x0000002dbae7)
    #3 ConverseRunPE /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1528 (tlsglobals+0x00000048dffe)
    #4 ConverseInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1430 (tlsglobals+0x00000048d932)
    #5 main /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/main.C:9 (tlsglobals+0x0000002cb286)

  Location is global 'Csv_CtvOffsstackBottom_' of size 4 at 0x55d6c1007034 (tlsglobals+0x0000008a7034)

  Thread T3 (tid=19339, running) created by main thread at:
    #0 pthread_create <null> (libtsan.so.0+0x0000000290c3)
    #1 CmiStartThreads /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-smp.c:506 (tlsglobals+0x000000487e4f)
    #2 ConverseInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1428 (tlsglobals+0x00000048d928)
    #3 main /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/main.C:9 (tlsglobals+0x0000002cb286)
SUMMARY: ThreadSanitizer: data race /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/libs/ck-libs/ampi/ampi.C:960 in ampiProcInit

Those 4 variables are Ctv variables that are currently initialized in AMPI's [initproc] routine in ampi.C:
 static void ampiProcInit(void){
   CtvInitialize(ampiParent*, ampiPtr);
   CtvInitialize(bool,ampiInitDone);
   CtvInitialize(bool,ampiFinalized);
   CtvInitialize(void*,stackBottom);

I'm not sure where else to initialize them though...

Original date: 2018-04-24 23:35:28

And here's the rest of the Charm++/Converse-related tsan output. I think we should really fix all of these, but the ''CmiThreadIs_flag' one is the most relevant to -tlsglobals:
WARNING: ThreadSanitizer: data race (pid=19329)
  Read of size 4 at 0x55d6c101c624 by main thread:
    #0 CthBaseInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/libthreads-default-tls.c:569 (tlsglobals+0x000000484420)
    #1 CthInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/libthreads-default-tls.c:1645 (tlsglobals+0x000000485643)
    #2 ConverseRunPE /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1487 (tlsglobals+0x00000048dd75)
    #3 ConverseInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1430 (tlsglobals+0x00000048d932)
    #4 main /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/main.C:9 (tlsglobals+0x0000002cb286)
  Previous write of size 4 at 0x55d6c101c624 by thread T3:
    #0 CthBaseInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/libthreads-default-tls.c:569 (tlsglobals+0x000000484437)
    #1 CthInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/libthreads-default-tls.c:1645 (tlsglobals+0x000000485643)
    #2 ConverseRunPE /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1487 (tlsglobals+0x00000048dd75)
    #3 call_startfn /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-smp.c:414 (tlsglobals+0x000000487b41)
    #4 <null> <null> (libtsan.so.0+0x000000025aab)

  Location is global 'CmiThreadIs_flag' of size 4 at 0x55d6c101c624 (tlsglobals+0x0000008bc624)

  Thread T3 (tid=19339, running) created by main thread at:
    #0 pthread_create <null> (libtsan.so.0+0x0000000290c3)
    #1 CmiStartThreads /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-smp.c:506 (tlsglobals+0x000000487e4f)
    #2 ConverseInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1428 (tlsglobals+0x00000048d928)
    #3 main /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/main.C:9 (tlsglobals+0x0000002cb286)
SUMMARY: ThreadSanitizer: data race /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/libthreads-default-tls.c:569 in CthBaseInit
==================
==================
WARNING: ThreadSanitizer: data race (pid=19329)
  Write of size 4 at 0x55d6c101d4d0 by main thread (mutexes: write M327):
    #0 CmiArgInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/convcore.c:359 (tlsglobals+0x000000492da9)
    #1 ConverseCommonInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/convcore.c:3781 (tlsglobals+0x00000049fa68)
    #2 ConverseRunPE /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1497 (tlsglobals+0x00000048df50)
    #3 ConverseInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1430 (tlsglobals+0x00000048d932)
    #4 main /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/main.C:9 (tlsglobals+0x0000002cb286)
  Previous read of size 4 at 0x55d6c101d4d0 by thread T3:
    #0 CmiAddCLA /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/convcore.c:298 (tlsglobals+0x000000492880)
    #1 CmiGetArgFlagDesc /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/convcore.c:549 (tlsglobals+0x000000493932)
    #2 CmiTimerInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/convcore.c:1013 (tlsglobals+0x000000495d41)
    #3 ConverseCommonInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/convcore.c:3796 (tlsglobals+0x00000049fc76)
    #4 ConverseRunPE /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1497 (tlsglobals+0x00000048df50)
    #5 call_startfn /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-smp.c:414 (tlsglobals+0x000000487b41)
    #6 <null> <null> (libtsan.so.0+0x000000025aab)

  Location is global 'usageChecked' of size 4 at 0x55d6c101d4d0 (tlsglobals+0x0000008bd4d0)

  Mutex M327 (0x7b0c0000b850) created at:
    #0 pthread_mutex_init <null> (libtsan.so.0+0x0000000299de)
    #1 LrtsCreateLock /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1910 (tlsglobals+0x00000048ea73)
    #2 CmiStartThreads /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-smp.c:455 (tlsglobals+0x000000487c15)
    #3 ConverseInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1428 (tlsglobals+0x00000048d928)
    #4 main /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/main.C:9 (tlsglobals+0x0000002cb286)
  Thread T3 (tid=19339, running) created by main thread at:
    #0 pthread_create <null> (libtsan.so.0+0x0000000290c3)
    #1 CmiStartThreads /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-smp.c:506 (tlsglobals+0x000000487e4f)
    #2 ConverseInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1428 (tlsglobals+0x00000048d928)
    #3 main /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/main.C:9 (tlsglobals+0x0000002cb286)
SUMMARY: ThreadSanitizer: data race /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/convcore.c:359 in CmiArgInit
==================
==================
WARNING: ThreadSanitizer: data race (pid=19329)
  Read of size 8 at 0x55d6c1008c48 by thread T3:
    #0 CmiWallTimer /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/convcore.c:1068 (tlsglobals+0x000000496111)
    #1 CcdModuleInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/conv-conds.c:430 (tlsglobals+0x0000004a2b3e)
    #2 ConverseCommonInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/convcore.c:3800 (tlsglobals+0x00000049fc93)
    #3 ConverseRunPE /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1497 (tlsglobals+0x00000048df50)
    #4 call_startfn /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-smp.c:414 (tlsglobals+0x000000487b41)
    #5 <null> <null> (libtsan.so.0+0x000000025aab)
  Previous write of size 8 at 0x55d6c1008c48 by main thread:
    #0 CmiWallTimer /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/convcore.c:1071 (tlsglobals+0x000000496163)
    #1 initTraceCore /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/traceCoreCommon.C:44 (tlsglobals+0x0000004b882a)
    #2 traceInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/trace-common.C:499 (tlsglobals+0x0000002cebd2)
    #3 ConverseCommonInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/convcore.c:3824 (tlsglobals+0x00000049fde5)
    #4 ConverseRunPE /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1497 (tlsglobals+0x00000048df50)
    #5 ConverseInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1430 (tlsglobals+0x00000048d932)
    #6 main /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/main.C:9 (tlsglobals+0x0000002cb286)

  Location is global 'lastT' of size 8 at 0x55d6c1008c48 (tlsglobals+0x0000008a8c48)

  Thread T3 (tid=19339, running) created by main thread at:
    #0 pthread_create <null> (libtsan.so.0+0x0000000290c3)
    #1 CmiStartThreads /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-smp.c:506 (tlsglobals+0x000000487e4f)
    #2 ConverseInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1428 (tlsglobals+0x00000048d928)
    #3 main /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/main.C:9 (tlsglobals+0x0000002cb286)
SUMMARY: ThreadSanitizer: data race /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/convcore.c:1068 in CmiWallTimer
==================
==================
WARNING: ThreadSanitizer: data race (pid=19329)
  Write of size 4 at 0x55d6c101e18c by thread T3:
    #0 ConverseCommonInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/convcore.c:3842 (tlsglobals+0x00000049fe22)
    #1 ConverseRunPE /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1497 (tlsglobals+0x00000048df50)
    #2 call_startfn /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-smp.c:414 (tlsglobals+0x000000487b41)
    #3 <null> <null> (libtsan.so.0+0x000000025aab)
  Previous write of size 4 at 0x55d6c101e18c by main thread:
    #0 ConverseCommonInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/convcore.c:3842 (tlsglobals+0x00000049fe22)
    #1 ConverseRunPE /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1497 (tlsglobals+0x00000048df50)
    #2 ConverseInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1430 (tlsglobals+0x00000048d932)
    #3 main /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/main.C:9 (tlsglobals+0x0000002cb286)

  Location is global 'ccsRunning' of size 4 at 0x55d6c101e18c (tlsglobals+0x0000008be18c)

  Thread T3 (tid=19339, running) created by main thread at:
    #0 pthread_create <null> (libtsan.so.0+0x0000000290c3)
    #1 CmiStartThreads /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-smp.c:506 (tlsglobals+0x000000487e4f)
    #2 ConverseInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1428 (tlsglobals+0x00000048d928)
    #3 main /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/main.C:9 (tlsglobals+0x0000002cb286)
SUMMARY: ThreadSanitizer: data race /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/convcore.c:3842 in ConverseCommonInit
==================
==================
WARNING: ThreadSanitizer: data race (pid=19329)
  Write of size 4 at 0x55d6c101b480 by main thread:
    #0 _loadbalancerInit() /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/LBDatabase.C:186 (tlsglobals+0x0000003e9209)
    #1 _initCharm(int, char**) /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/init.C:1192 (tlsglobals+0x0000002db6f4)
    #2 ConverseRunPE /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1528 (tlsglobals+0x00000048dffe)
    #3 ConverseInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1430 (tlsglobals+0x00000048d932)
    #4 main /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/main.C:9 (tlsglobals+0x0000002cb286)
  Previous write of size 4 at 0x55d6c101b480 by thread T3:
    #0 _loadbalancerInit() /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/LBDatabase.C:186 (tlsglobals+0x0000003e9209)
    #1 _initCharm(int, char**) /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/init.C:1192 (tlsglobals+0x0000002db6f4)
    #2 ConverseRunPE /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1534 (tlsglobals+0x00000048e058)
    #3 call_startfn /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-smp.c:414 (tlsglobals+0x000000487b41)
    #4 <null> <null> (libtsan.so.0+0x000000025aab)

  Location is global '_lb_args' of size 104 at 0x55d6c101b460 (tlsglobals+0x0000008bb480)

  Thread T3 (tid=19339, running) created by main thread at:
    #0 pthread_create <null> (libtsan.so.0+0x0000000290c3)
    #1 CmiStartThreads /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-smp.c:506 (tlsglobals+0x000000487e4f)
    #2 ConverseInit /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/machine-common-core.c:1428 (tlsglobals+0x00000048d928)
    #3 main /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/main.C:9 (tlsglobals+0x0000002cb286)
SUMMARY: ThreadSanitizer: data race /home/evan/charm/mpi-linux-x86_64-smp-tsan/tmp/LBDatabase.C:186 in _loadbalancerInit()

There are a bunch more races on _lb_args inside _loadbalancerInit() too.

Original date: 2018-04-24 23:49:08

CmiThreadIs_flag looks to be a static global variable, which all ranks read and write to without any sort of synchronization. It should be written to the same value by all threads, so I think it's actually safe but tsan can't tell that?

Original date: 2018-04-25 19:24:40

I tried running with LLDB instead of GDB, which provided useful information:
$ lldb -- ./tlsglobals
(lldb) target create "./tlsglobals"
Current executable set to './tlsglobals' (x86_64).
(lldb) r
Process 24193 launched: './tlsglobals' (x86_64)
Charm++> Running on MPI version: 3.1
Charm++> level of thread support used: MPI_THREAD_FUNNELED (desired: MPI_THREAD_FUNNELED)
Charm++> Running in SMP mode: numNodes 1,  1 worker threads per process
Charm++> The comm. thread both sends and receives messages
Converse/Charm++ Commit ID: v6.8.2-620-gc5f37d228
CharmLB> Load balancer assumes all CPUs are same.
Charm++> Running on 1 unique compute nodes (12-way SMP).
Charm++> cpu topology info is gathered in 0.000 seconds.
CharmLB> RotateLB created.
Charm++> -tlsglobals enabled for privatization of thread-local variables.
Process 24193 stopped
* thread #4, name = 'tlsglobals', stop reason = signal SIGSEGV: invalid address (fault address: 0x0)
    frame #0: tlsglobals`CthFixData(t=0x00007fffdc166dd0) at libthreads-default-tls.c:350
   347 	#endif
   348 	static void CthFixData(CthThread t)
   349 	{
-> 350 	  size_t newsize = CthCpvAccess(CthDatasize);
   351 	  size_t oldsize = B(t)->datasize;
   352 	  if (oldsize < newsize) {
   353 	    newsize = 2*newsize;
(lldb) p CthCpvAccess(CthDatasize)
error: Couldn't materialize: couldn't get the value of variable Cpv_CthDatasize_: No TLS data currently exists for this thread.
error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression

Original date: 2018-04-25 19:44:00

Note the verbs autobuilds failed last night in tests/ampi/privatization too.

Original date: 2018-04-25 19:46:20

They failed in swapglobals, which means charmc must not have detected a problem scenario in the toolchain.

Original date: 2018-04-25 21:05:46

I suspect the issue has something to do with the MPI build using more threads than other targets, and also the use of "pthread-specific thread local storage":https://dulanja.blogspot.com/2011/09/how-to-use-thread-local-storage-tls-in.html.
mpi-linux-x86_64-smp:
(lldb) thread list
Process 24193 stopped
  thread #1: tid = 24193, 0x00007fffe9cebbcf mca_pml_ob1.so`mca_pml_ob1_recv_req_start + 1263, name = 'tlsglobals'
  thread #2: tid = 24201, 0x00007ffff68de951 libc.so.6`__GI___poll(fds=0x00007fffec000b10, nfds=2, timeout=3599733) at poll.c:29, name = 'tlsglobals'
  thread #3: tid = 24202, 0x00007ffff68eacd7 libc.so.6`__GI_epoll_pwait(epfd=12, events=0x0000555555d32720, maxevents=32, timeout=-1, set=0x0000000000000000) at epoll_pwait.c:42, name = 'tlsglobals'
* thread #4: tid = 24203, 0x00005555558f6876 tlsglobals`CthFixData(t=0x00007fffdc166dd0) at libthreads-default-tls.c:350, name = 'tlsglobals', stop reason = signal SIGSEGV: invalid address (fault address: 0x0)

netlrts-linux-x86_64-smp
(lldb) thread list
Process 9223 stopped
* thread #1: tid = 9223, 0x000055555574a8c6 tlsglobals`::test_privatization(rank=0, my_wth=0, global=0x00000004012ff170) at test.C:51, name = 'tlsglobals', stop reason = breakpoint 1.1
  thread #2: tid = 9226, 0x00005555558dd59c tlsglobals`PCQueuePop(Q=0x0000555555c9ed50) at pcqueue.h:226, name = 'tlsglobals'

Original date: 2018-04-26 20:46:21

Interesting comment here:
$ lldb -- ./tlsglobals
(lldb) target create "./tlsglobals"
Current executable set to './tlsglobals' (x86_64).
(lldb) r
Process 1460 launched: './tlsglobals' (x86_64)
Charm++> Running on MPI version: 3.1
Charm++> level of thread support used: MPI_THREAD_FUNNELED (desired: MPI_THREAD_FUNNELED)
Charm++> Running in SMP mode: numNodes 1,  1 worker threads per process
Charm++> The comm. thread both sends and receives messages
Converse/Charm++ Commit ID: v6.8.2-621-g9b61967fa
CharmLB> Load balancer assumes all CPUs are same.
Charm++> Running on 1 unique compute nodes (12-way SMP).
Charm++> cpu topology info is gathered in 0.000 seconds.
CharmLB> RotateLB created.
Charm++> -tlsglobals enabled for privatization of thread-local variables.
Process 1460 stopped
* thread #4, name = 'tlsglobals', stop reason = signal SIGSEGV: invalid address (fault address: 0x0)
    frame #0: tlsglobals`CthFixData(t=0x00007fffdc167b00) at libthreads-default-tls.c:350
   347 	#endif
   348 	static void CthFixData(CthThread t)
   349 	{
-> 350 	  size_t newsize = CthCpvAccess(CthDatasize);
   351 	  size_t oldsize = B(t)->datasize;
   352 	  if (oldsize < newsize) {
   353 	    newsize = 2*newsize;
(lldb) bt
* thread #4, name = 'tlsglobals', stop reason = signal SIGSEGV: invalid address (fault address: 0x0)
  * frame #0: tlsglobals`CthFixData(t=0x00007fffdc167b00) at libthreads-default-tls.c:350
    frame #1: tlsglobals`CthBaseResume(t=0x00007fffdc167b00) at libthreads-default-tls.c:775
    frame #2: tlsglobals`CthResume(t=0x00007fffdc167b00) at libthreads-default-tls.c:1732
    frame #3: tlsglobals`CthResumeNormalThread(token=0x00007fffdc167c00) at convcore.c:2110
    frame #4: tlsglobals`CmiHandleMessage(msg=0x00007fffdc167c00) at convcore.c:1654
    frame #5: tlsglobals`CsdScheduleForever at convcore.c:1902
    frame #6: tlsglobals`CsdScheduler(maxmsgs=-1) at convcore.c:1838
    frame #7: tlsglobals`ConverseRunPE(everReturn=0) at machine-common-core.c:1535
    frame #8: tlsglobals`call_startfn(vindex=0x0000000000000000) at machine-smp.c:414
    frame #9: 0x00007ffff79bb7fc libpthread.so.0`start_thread + 220
    frame #10: libc.so.6`__GI___clone at clone.S:95
(lldb) f 1
frame #1: tlsglobals`CthBaseResume(t=0x00007fffdc167b00) at libthreads-default-tls.c:775
   772 	  for(l=B(t)->listener;l!=NULL;l=l->next){
   773 	    if (l->resume) l->resume(l);
   774 	  }
-> 775 	  CthFixData(t); /*Thread-local storage may have changed in other thread.*/
   776 	  CthCpvAccess(CthCurrent) = t;
   777 	  CthCpvAccess(CthData) = B(t)->data;
   778 	  CthAliasEnable(B(t));

Original date: 2018-04-26 21:43:47

The failure occurs on the first CthResume that shows up in any backtrace from switchTLS.
mpi-linux-x86_64-smp:
$ gdb ./tlsglobals
GNU gdb (Ubuntu 8.0.1-0ubuntu1) 8.0.1
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./tlsglobals...done.
(gdb) br switchTLS
Breakpoint 1 at 0x3d073f: file cmitls.c, line 110.
(gdb) commands
Type commands for breakpoint(s) 1, one per line.
End with a line saying just "end".
>bt
>c
>end
(gdb) r
Starting program: /home/evan/charm/mpi-linux-x86_64-smp/tests/ampi/privatization/tlsglobals
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff41b0700 (LWP 24317)]
[New Thread 0x7ffff39af700 (LWP 24318)]
Charm++> Running on MPI version: 3.1
Charm++> level of thread support used: MPI_THREAD_FUNNELED (desired: MPI_THREAD_FUNNELED)
Charm++> Running in SMP mode: numNodes 1,  1 worker threads per process
Charm++> The comm. thread both sends and receives messages
[New Thread 0x7fffe9098700 (LWP 24319)]
Converse/Charm++ Commit ID: v6.8.2-621-g9b61967fa
CharmLB> Load balancer assumes all CPUs are same.
Charm++> Running on 1 unique compute nodes (12-way SMP).
Charm++> cpu topology info is gathered in 0.000 seconds.
CharmLB> RotateLB created.
[Switching to Thread 0x7fffe9098700 (LWP 24319)]

Thread 4 "tlsglobals" hit Breakpoint 1, switchTLS (cur=0x7fffe9097348, next=0x7fffdc001660) at cmitls.c:110
110	                : "r"(next->memseg));
#0  switchTLS (cur=0x7fffe9097348, next=0x7fffdc001660) at cmitls.c:110
#1  0x00005555558f6e78 in CtgInstallMainThreadTLS (cur=0x7fffe9097348) at libthreads-default-tls.c:528
#2  0x000055555575fe0d in TCharmAPIRoutine::TCharmAPIRoutine (this=0x7fffe9097340, routineName=0x5555559929ee "TCHARM_Get_num_chunks", libraryName=0x55555599298d "tcharm") at tcharm_impl.h:337
#3  0x000055555575a1c7 in TCHARM_Get_num_chunks () at tcharm.C:677
#4  0x000055555576e770 in ampiCreateMain (mainFn=0x55555576de40 <AMPI_Fallback_Main(int, char**)>, name=0x555555995a18 "default", nameLen=7) at ampi.C:1057
#5  0x000055555576dec2 in AMPI_Setup_Switch () at ampi.C:841
#6  0x000055555575a194 in TCHARM_Call_fallback_setup () at tcharm.C:664
#7  0x0000555555991f5b in TCHARM_User_setup () at compat_us.c:4
#8  0x0000555555991f48 in tcharm_user_setup_ () at compat_fus.c:4
#9  0x00005555557dd6c5 in TCharmMain::TCharmMain (this=0x7fffdc1658e0, msg=0x7fffdc08b580) at tcharmmain.C:43
#10 0x00005555557dd3a4 in CkIndex_TCharmMain::_call_TCharmMain_CkArgMsg (impl_msg=0x7fffdc08b580, impl_obj_void=0x7fffdc1658e0) at tcharmmain.def.h:53
#11 0x00005555557e7c7c in _initCharm (unused_argc=1, argv=0x7fffffffe468) at init.C:1538
#12 0x00005555558fce2f in ConverseRunPE (everReturn=0) at machine-common-core.c:1534
#13 0x00005555558f942e in call_startfn (vindex=0x0) at machine-smp.c:414
#14 0x00007ffff79bb7fc in start_thread (arg=0x7fffe9098700) at pthread_create.c:465
#15 0x00007ffff68eab5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 4 "tlsglobals" hit Breakpoint 1, switchTLS (cur=0x7fffe90972f0, next=0x7fffe9097348) at cmitls.c:110
110	                : "r"(next->memseg));
#0  switchTLS (cur=0x7fffe90972f0, next=0x7fffe9097348) at cmitls.c:110
#1  0x00005555558f6eb1 in CtgInstallCthTLS (cur=0x7fffe90972f0, next=0x7fffe9097348) at libthreads-default-tls.c:534
#2  0x000055555575febc in TCharmAPIRoutine::~TCharmAPIRoutine (this=0x7fffe9097340, __in_chrg=<optimized out>) at tcharm_impl.h:368
#3  0x000055555575a244 in TCHARM_Get_num_chunks () at tcharm.C:677
#4  0x000055555576e770 in ampiCreateMain (mainFn=0x55555576de40 <AMPI_Fallback_Main(int, char**)>, name=0x555555995a18 "default", nameLen=7) at ampi.C:1057
#5  0x000055555576dec2 in AMPI_Setup_Switch () at ampi.C:841
#6  0x000055555575a194 in TCHARM_Call_fallback_setup () at tcharm.C:664
#7  0x0000555555991f5b in TCHARM_User_setup () at compat_us.c:4
#8  0x0000555555991f48 in tcharm_user_setup_ () at compat_fus.c:4
#9  0x00005555557dd6c5 in TCharmMain::TCharmMain (this=0x7fffdc1658e0, msg=0x7fffdc08b580) at tcharmmain.C:43
#10 0x00005555557dd3a4 in CkIndex_TCharmMain::_call_TCharmMain_CkArgMsg (impl_msg=0x7fffdc08b580, impl_obj_void=0x7fffdc1658e0) at tcharmmain.def.h:53
#11 0x00005555557e7c7c in _initCharm (unused_argc=1, argv=0x7fffffffe468) at init.C:1538
#12 0x00005555558fce2f in ConverseRunPE (everReturn=0) at machine-common-core.c:1534
#13 0x00005555558f942e in call_startfn (vindex=0x0) at machine-smp.c:414
#14 0x00007ffff79bb7fc in start_thread (arg=0x7fffe9098700) at pthread_create.c:465
#15 0x00007ffff68eab5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 4 "tlsglobals" hit Breakpoint 1, switchTLS (cur=0x7fffe9097328, next=0x7fffdc001660) at cmitls.c:110
110	                : "r"(next->memseg));
#0  switchTLS (cur=0x7fffe9097328, next=0x7fffdc001660) at cmitls.c:110
#1  0x00005555558f6e78 in CtgInstallMainThreadTLS (cur=0x7fffe9097328) at libthreads-default-tls.c:528
#2  0x000055555575fe0d in TCharmAPIRoutine::TCharmAPIRoutine (this=0x7fffe9097320, routineName=0x555555992a9f "TCHARM_Create_data", libraryName=0x55555599298d "tcharm") at tcharm_impl.h:337
#3  0x000055555575a3eb in TCHARM_Create_data (nThreads=1, threadFn=1, threadData=0x7fffdc165910, threadDataLen=8) at tcharm.C:735
#4  0x000055555576e7d4 in ampiCreateMain (mainFn=0x55555576de40 <AMPI_Fallback_Main(int, char**)>, name=0x555555995a18 "default", nameLen=7) at ampi.C:1061
#5  0x000055555576dec2 in AMPI_Setup_Switch () at ampi.C:841
#6  0x000055555575a194 in TCHARM_Call_fallback_setup () at tcharm.C:664
#7  0x0000555555991f5b in TCHARM_User_setup () at compat_us.c:4
#8  0x0000555555991f48 in tcharm_user_setup_ () at compat_fus.c:4
#9  0x00005555557dd6c5 in TCharmMain::TCharmMain (this=0x7fffdc1658e0, msg=0x7fffdc08b580) at tcharmmain.C:43
#10 0x00005555557dd3a4 in CkIndex_TCharmMain::_call_TCharmMain_CkArgMsg (impl_msg=0x7fffdc08b580, impl_obj_void=0x7fffdc1658e0) at tcharmmain.def.h:53
#11 0x00005555557e7c7c in _initCharm (unused_argc=1, argv=0x7fffffffe468) at init.C:1538
#12 0x00005555558fce2f in ConverseRunPE (everReturn=0) at machine-common-core.c:1534
#13 0x00005555558f942e in call_startfn (vindex=0x0) at machine-smp.c:414
#14 0x00007ffff79bb7fc in start_thread (arg=0x7fffe9098700) at pthread_create.c:465
#15 0x00007ffff68eab5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
Charm++> -tlsglobals enabled for privatization of thread-local variables.

Thread 4 "tlsglobals" hit Breakpoint 1, switchTLS (cur=0x7fffe90972c0, next=0x7fffe9097328) at cmitls.c:110
110	                : "r"(next->memseg));
#0  switchTLS (cur=0x7fffe90972c0, next=0x7fffe9097328) at cmitls.c:110
#1  0x00005555558f6eb1 in CtgInstallCthTLS (cur=0x7fffe90972c0, next=0x7fffe9097328) at libthreads-default-tls.c:534
#2  0x000055555575febc in TCharmAPIRoutine::~TCharmAPIRoutine (this=0x7fffe9097320, __in_chrg=<optimized out>) at tcharm_impl.h:368
#3  0x000055555575a496 in TCHARM_Create_data (nThreads=1, threadFn=1, threadData=0x7fffdc165910, threadDataLen=8) at tcharm.C:735
#4  0x000055555576e7d4 in ampiCreateMain (mainFn=0x55555576de40 <AMPI_Fallback_Main(int, char**)>, name=0x555555995a18 "default", nameLen=7) at ampi.C:1061
#5  0x000055555576dec2 in AMPI_Setup_Switch () at ampi.C:841
#6  0x000055555575a194 in TCHARM_Call_fallback_setup () at tcharm.C:664
#7  0x0000555555991f5b in TCHARM_User_setup () at compat_us.c:4
#8  0x0000555555991f48 in tcharm_user_setup_ () at compat_fus.c:4
#9  0x00005555557dd6c5 in TCharmMain::TCharmMain (this=0x7fffdc1658e0, msg=0x7fffdc08b580) at tcharmmain.C:43
#10 0x00005555557dd3a4 in CkIndex_TCharmMain::_call_TCharmMain_CkArgMsg (impl_msg=0x7fffdc08b580, impl_obj_void=0x7fffdc1658e0) at tcharmmain.def.h:53
#11 0x00005555557e7c7c in _initCharm (unused_argc=1, argv=0x7fffffffe468) at init.C:1538
#12 0x00005555558fce2f in ConverseRunPE (everReturn=0) at machine-common-core.c:1534
#13 0x00005555558f942e in call_startfn (vindex=0x0) at machine-smp.c:414
#14 0x00007ffff79bb7fc in start_thread (arg=0x7fffe9098700) at pthread_create.c:465
#15 0x00007ffff68eab5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 4 "tlsglobals" hit Breakpoint 1, switchTLS (cur=0x7fffdc0016f8, next=0x7fffdc166f48) at cmitls.c:110
110	                : "r"(next->memseg));
#0  switchTLS (cur=0x7fffdc0016f8, next=0x7fffdc166f48) at cmitls.c:110
#1  0x00005555558f80df in CthResume (t=0x7fffdc166ed0) at libthreads-default-tls.c:1728
#2  0x00005555559025f7 in CthResumeNormalThread (token=0x7fffdc166fd0) at convcore.c:2110
#3  0x0000555555901d5e in CmiHandleMessage (msg=0x7fffdc166fd0) at convcore.c:1654
#4  0x000055555590215a in CsdScheduleForever () at convcore.c:1902
#5  0x0000555555902075 in CsdScheduler (maxmsgs=-1) at convcore.c:1838
#6  0x00005555558fce43 in ConverseRunPE (everReturn=0) at machine-common-core.c:1535
#7  0x00005555558f942e in call_startfn (vindex=0x0) at machine-smp.c:414
#8  0x00007ffff79bb7fc in start_thread (arg=0x7fffe9098700) at pthread_create.c:465
#9  0x00007ffff68eab5f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 4 "tlsglobals" received signal SIGSEGV, Segmentation fault.
0x00005555558f6876 in CthFixData (t=0x7fffdc166ed0) at libthreads-default-tls.c:350
350	  size_t newsize = CthCpvAccess(CthDatasize);
(gdb)

For comparison, netlrts-linux-x86_64-smp:
$ gdb ./tlsglobals
GNU gdb (Ubuntu 8.0.1-0ubuntu1) 8.0.1
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./tlsglobals...done.
(gdb) br switchTLS
Breakpoint 1 at 0x3b1bea: file cmitls.c, line 110.
(gdb) commands
Type commands for breakpoint(s) 1, one per line.
End with a line saying just "end".
>bt
>c
>end
(gdb) r
Starting program: /home/evan/charm/netlrts-linux-x86_64-smp/tests/ampi/privatization/tlsglobals
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Charm++> No provisioning arguments specified. Running with a single PE.
         Use +auto-provision to fully subscribe resources or +p1 to silence this message.
Charm++: standalone mode (not using charmrun)
Charm++> Running in SMP mode: numNodes 1,  1 worker threads per process
Charm++> The comm. thread both sends and receives messages
[New Thread 0x7ffff6ce0700 (LWP 21742)]
Converse/Charm++ Commit ID: v6.8.2-621-g9b61967fa
Charm++> scheduler running in netpoll mode.
CharmLB> Load balancer assumes all CPUs are same.
Charm++> Running on 1 unique compute nodes (12-way SMP).
Charm++> cpu topology info is gathered in 0.000 seconds.
CharmLB> RotateLB created.

Thread 1 "tlsglobals" hit Breakpoint 1, switchTLS (cur=0x7fffffffddf8, next=0x555555c9c130) at cmitls.c:110
110	                : "r"(next->memseg));
#0  switchTLS (cur=0x7fffffffddf8, next=0x555555c9c130) at cmitls.c:110
#1  0x00005555558dbe80 in CtgInstallMainThreadTLS (cur=0x7fffffffddf8) at libthreads-default-tls.c:528
#2  0x0000555555752bd7 in TCharmAPIRoutine::TCharmAPIRoutine (this=0x7fffffffddf0, routineName=0x555555973aae "TCHARM_Get_num_chunks", libraryName=0x555555973a4d "tcharm") at tcharm_impl.h:337
#3  0x000055555574d195 in TCHARM_Get_num_chunks () at tcharm.C:677
#4  0x00005555557612d2 in ampiCreateMain (mainFn=0x555555760ae4 <AMPI_Fallback_Main(int, char**)>, name=0x555555976a90 "default", nameLen=7) at ampi.C:1057
#5  0x0000555555760b5d in AMPI_Setup_Switch () at ampi.C:841
#6  0x000055555574d162 in TCHARM_Call_fallback_setup () at tcharm.C:664
#7  0x0000555555972ff3 in TCHARM_User_setup () at compat_us.c:4
#8  0x0000555555972fe0 in tcharm_user_setup_ () at compat_fus.c:4
#9  0x00005555557cdffb in TCharmMain::TCharmMain (this=0x555555d2a430, msg=0x555555d29d90) at tcharmmain.C:43
#10 0x00005555557cdd04 in CkIndex_TCharmMain::_call_TCharmMain_CkArgMsg (impl_msg=0x555555d29d90, impl_obj_void=0x555555d2a430) at tcharmmain.def.h:53
#11 0x00005555557d72e2 in _initCharm (unused_argc=1, argv=0x7fffffffe478) at init.C:1538
#12 0x00005555558e0d28 in ConverseRunPE (everReturn=0) at machine-common-core.c:1534
#13 0x00005555558e09e2 in ConverseInit (argc=1, argv=0x7fffffffe478, fn=0x5555557d52ff <_initCharm(int, char**)>, usched=0, initret=0) at machine-common-core.c:1430
#14 0x00005555557ce34e in main (argc=1, argv=0x7fffffffe478) at main.C:9

Thread 1 "tlsglobals" hit Breakpoint 1, switchTLS (cur=0x7fffffffdda0, next=0x7fffffffddf8) at cmitls.c:110
110	                : "r"(next->memseg));
#0  switchTLS (cur=0x7fffffffdda0, next=0x7fffffffddf8) at cmitls.c:110
#1  0x00005555558dbeb9 in CtgInstallCthTLS (cur=0x7fffffffdda0, next=0x7fffffffddf8) at libthreads-default-tls.c:534
#2  0x0000555555752c7b in TCharmAPIRoutine::~TCharmAPIRoutine (this=0x7fffffffddf0, __in_chrg=<optimized out>) at tcharm_impl.h:368
#3  0x000055555574d212 in TCHARM_Get_num_chunks () at tcharm.C:677
#4  0x00005555557612d2 in ampiCreateMain (mainFn=0x555555760ae4 <AMPI_Fallback_Main(int, char**)>, name=0x555555976a90 "default", nameLen=7) at ampi.C:1057
#5  0x0000555555760b5d in AMPI_Setup_Switch () at ampi.C:841
#6  0x000055555574d162 in TCHARM_Call_fallback_setup () at tcharm.C:664
#7  0x0000555555972ff3 in TCHARM_User_setup () at compat_us.c:4
#8  0x0000555555972fe0 in tcharm_user_setup_ () at compat_fus.c:4
#9  0x00005555557cdffb in TCharmMain::TCharmMain (this=0x555555d2a430, msg=0x555555d29d90) at tcharmmain.C:43
#10 0x00005555557cdd04 in CkIndex_TCharmMain::_call_TCharmMain_CkArgMsg (impl_msg=0x555555d29d90, impl_obj_void=0x555555d2a430) at tcharmmain.def.h:53
#11 0x00005555557d72e2 in _initCharm (unused_argc=1, argv=0x7fffffffe478) at init.C:1538
#12 0x00005555558e0d28 in ConverseRunPE (everReturn=0) at machine-common-core.c:1534
#13 0x00005555558e09e2 in ConverseInit (argc=1, argv=0x7fffffffe478, fn=0x5555557d52ff <_initCharm(int, char**)>, usched=0, initret=0) at machine-common-core.c:1430
#14 0x00005555557ce34e in main (argc=1, argv=0x7fffffffe478) at main.C:9

Thread 1 "tlsglobals" hit Breakpoint 1, switchTLS (cur=0x7fffffffddd8, next=0x555555c9c130) at cmitls.c:110
110	                : "r"(next->memseg));
#0  switchTLS (cur=0x7fffffffddd8, next=0x555555c9c130) at cmitls.c:110
#1  0x00005555558dbe80 in CtgInstallMainThreadTLS (cur=0x7fffffffddd8) at libthreads-default-tls.c:528
#2  0x0000555555752bd7 in TCharmAPIRoutine::TCharmAPIRoutine (this=0x7fffffffddd0, routineName=0x555555973b5f "TCHARM_Create_data", libraryName=0x555555973a4d "tcharm") at tcharm_impl.h:337
#3  0x000055555574d3b3 in TCHARM_Create_data (nThreads=1, threadFn=1, threadData=0x555555e03dd0, threadDataLen=8) at tcharm.C:735
#4  0x0000555555761336 in ampiCreateMain (mainFn=0x555555760ae4 <AMPI_Fallback_Main(int, char**)>, name=0x555555976a90 "default", nameLen=7) at ampi.C:1061
#5  0x0000555555760b5d in AMPI_Setup_Switch () at ampi.C:841
#6  0x000055555574d162 in TCHARM_Call_fallback_setup () at tcharm.C:664
#7  0x0000555555972ff3 in TCHARM_User_setup () at compat_us.c:4
#8  0x0000555555972fe0 in tcharm_user_setup_ () at compat_fus.c:4
#9  0x00005555557cdffb in TCharmMain::TCharmMain (this=0x555555d2a430, msg=0x555555d29d90) at tcharmmain.C:43
#10 0x00005555557cdd04 in CkIndex_TCharmMain::_call_TCharmMain_CkArgMsg (impl_msg=0x555555d29d90, impl_obj_void=0x555555d2a430) at tcharmmain.def.h:53
#11 0x00005555557d72e2 in _initCharm (unused_argc=1, argv=0x7fffffffe478) at init.C:1538
#12 0x00005555558e0d28 in ConverseRunPE (everReturn=0) at machine-common-core.c:1534
#13 0x00005555558e09e2 in ConverseInit (argc=1, argv=0x7fffffffe478, fn=0x5555557d52ff <_initCharm(int, char**)>, usched=0, initret=0) at machine-common-core.c:1430
#14 0x00005555557ce34e in main (argc=1, argv=0x7fffffffe478) at main.C:9
Charm++> -tlsglobals enabled for privatization of thread-local variables.

Thread 1 "tlsglobals" hit Breakpoint 1, switchTLS (cur=0x7fffffffdd70, next=0x7fffffffddd8) at cmitls.c:110
110	                : "r"(next->memseg));
#0  switchTLS (cur=0x7fffffffdd70, next=0x7fffffffddd8) at cmitls.c:110
#1  0x00005555558dbeb9 in CtgInstallCthTLS (cur=0x7fffffffdd70, next=0x7fffffffddd8) at libthreads-default-tls.c:534
#2  0x0000555555752c7b in TCharmAPIRoutine::~TCharmAPIRoutine (this=0x7fffffffddd0, __in_chrg=<optimized out>) at tcharm_impl.h:368
#3  0x000055555574d45e in TCHARM_Create_data (nThreads=1, threadFn=1, threadData=0x555555e03dd0, threadDataLen=8) at tcharm.C:735
#4  0x0000555555761336 in ampiCreateMain (mainFn=0x555555760ae4 <AMPI_Fallback_Main(int, char**)>, name=0x555555976a90 "default", nameLen=7) at ampi.C:1061
#5  0x0000555555760b5d in AMPI_Setup_Switch () at ampi.C:841
#6  0x000055555574d162 in TCHARM_Call_fallback_setup () at tcharm.C:664
#7  0x0000555555972ff3 in TCHARM_User_setup () at compat_us.c:4
#8  0x0000555555972fe0 in tcharm_user_setup_ () at compat_fus.c:4
#9  0x00005555557cdffb in TCharmMain::TCharmMain (this=0x555555d2a430, msg=0x555555d29d90) at tcharmmain.C:43
#10 0x00005555557cdd04 in CkIndex_TCharmMain::_call_TCharmMain_CkArgMsg (impl_msg=0x555555d29d90, impl_obj_void=0x555555d2a430) at tcharmmain.def.h:53
#11 0x00005555557d72e2 in _initCharm (unused_argc=1, argv=0x7fffffffe478) at init.C:1538
#12 0x00005555558e0d28 in ConverseRunPE (everReturn=0) at machine-common-core.c:1534
#13 0x00005555558e09e2 in ConverseInit (argc=1, argv=0x7fffffffe478, fn=0x5555557d52ff <_initCharm(int, char**)>, usched=0, initret=0) at machine-common-core.c:1430
#14 0x00005555557ce34e in main (argc=1, argv=0x7fffffffe478) at main.C:9

Thread 1 "tlsglobals" hit Breakpoint 1, switchTLS (cur=0x555555c9ecc8, next=0x555555ca0cd8) at cmitls.c:110
110	                : "r"(next->memseg));
#0  switchTLS (cur=0x555555c9ecc8, next=0x555555ca0cd8) at cmitls.c:110
#1  0x00005555558dcec1 in CthResume (t=0x555555ca0c60) at libthreads-default-tls.c:1728
#2  0x00005555558e904b in CthResumeNormalThread (token=0x555555e05270) at convcore.c:2110
#3  0x00005555558e88ba in CmiHandleMessage (msg=0x555555e05270) at convcore.c:1654
#4  0x00005555558e8c50 in CsdScheduleForever () at convcore.c:1902
#5  0x00005555558e8b96 in CsdScheduler (maxmsgs=-1) at convcore.c:1838
#6  0x00005555558e0d3c in ConverseRunPE (everReturn=0) at machine-common-core.c:1535
#7  0x00005555558e09e2 in ConverseInit (argc=1, argv=0x7fffffffe478, fn=0x5555557d52ff <_initCharm(int, char**)>, usched=0, initret=0) at machine-common-core.c:1430
#8  0x00005555557ce34e in main (argc=1, argv=0x7fffffffe478) at main.C:9

Thread 1 "tlsglobals" hit Breakpoint 1, switchTLS (cur=0x4010fff98, next=0x555555c9c130) at cmitls.c:110
110	                : "r"(next->memseg));
#0  switchTLS (cur=0x4010fff98, next=0x555555c9c130) at cmitls.c:110
#1  0x00005555558dbe80 in CtgInstallMainThreadTLS (cur=0x4010fff98) at libthreads-default-tls.c:528
#2  0x0000555555752bd7 in TCharmAPIRoutine::TCharmAPIRoutine (this=0x4010fff90, routineName=0x555555975390 "AMPI_Init", libraryName=0x5555559777c6 "ampi") at tcharm_impl.h:337
#3  0x000055555576b3cc in MPI_Init (p_argc=0x4010ffffc, p_argv=0x4010ffff0) at ampi.C:3618
#4  0x000055555574ab9b in AMPI_Main_cpp (argc=1, argv=0x555555d0c500) at test.C:200
#5  0x0000555555760b09 in AMPI_Fallback_Main (argc=1, argv=0x555555d0c500) at ampi.C:825
#6  0x00005555557a2e45 in MPI_threadstart_t::start (this=0x401100098) at ampi.C:1031
#7  0x00005555557612b6 in AMPI_threadstart (data=0x555555e04c10) at ampi.C:1051
#8  0x000055555574b661 in startTCharmThread (msg=0x555555e04bf0) at tcharm.C:175
#9  0x00005555558dcf8b in CthStartThread (arg=...) at libthreads-default-tls.c:1770
#10 0x00005555558dd41f in make_fcontext () at make_x86_64_sysv_elf_gas.S:70
#11 0x0000000000000000 in ?? ()

(and much more afterward)

Original date: 2018-04-27 21:04:52

My current hypothesis is that the crash is related to the fact that the MPI machine layer uses the first thread as the comm thread, unlike other machine layers. This pushes the initial setup of our custom TLS functionality to thread 2, and I suspect somehow this prevents the normal procedure from completing successfully.
mpi-linux-x86_64-smp: (using MPICH, because OpenMPI uses additional threads internally)
(gdb) info threads
  Id   Target Id         Frame
  1    Thread 0x7ffff7fd4dc0 (LWP 22660) "tlsglobals" 0x00007ffff6936951 in __GI___poll (fds=0x555555cb5080, nfds=1, timeout=0) at ../sysdeps/unix/sysv/linux/poll.c:29
* 2    Thread 0x7ffff5b74700 (LWP 22661) "tlsglobals" allocNewTLSSeg (t=0x7ffff0166fc8, th=0x7ffff0166f50) at cmitls.c:52

netlrts-linux-x86_64-smp:
(gdb) info threads
  Id   Target Id         Frame
* 1    Thread 0x7ffff7fd6d80 (LWP 22664) "tlsglobals" allocNewTLSSeg (t=0x555555ca0cd8, th=0x555555ca0c60) at cmitls.c:52
  2    Thread 0x7ffff6ce0700 (LWP 22668) "tlsglobals" PCQueuePop (Q=0x555555c9d110) at pcqueue.h:226

Original date: 2018-04-27 21:34:51

I tried the following hack:
diff --git a/src/conv-core/threads.c b/src/conv-core/threads.c
index 8713a286c..af9b770ac 100644
--- a/src/conv-core/threads.c
+++ b/src/conv-core/threads.c
`` -1691,6 +1691,16 `` void CthInit(char **argv)
     CmiThreadIs_flag |= CMI_THREAD_IS_ALIAS;
 #endif
  }
+
+#if CMK_CONVERSE_MPI && CMK_THREADS_BUILD_TLS
+  {
+    tlsseg_t temp;
+    allocNewTLSSeg(&B(t)->tlsseg,t);
+    switchTLS(&temp, &B(t)->tlsseg);
+    CthCpvAccess(CthCurrent)=t; // crashes here, "No TLS data currently exists for this thread."
+    switchTLS(&B(t)->tlsseg, &temp);
+  }
+#endif
 }

 static void CthThreadFree(CthThread t)

It runs on both threads and crashes on both.

Original date: 2018-04-27 21:45:42

My current hypothesis is that the crash is related to the fact that the MPI machine layer uses the first thread as the comm thread, unlike other machine layers.

Yeah, the way we call MPI_Init_thread (with MPI_THREAD_FUNNELED) necessitates that all calls we make to MPI must be done on the master thread of the process (thread 0), or else the MPI standard says the behavior is undefined.

Original date: 2018-04-27 21:58:41

Switching the machine layer to use MPI_THREAD_SERIALIZED instead proves the hypothesis incorrect, as it still crashes.
(gdb) info threads
  Id   Target Id         Frame
* 1    Thread 0x7ffff7fd4dc0 (LWP 1386) "tlsglobals" 0x00005555558e8646 in CthFixData (t=0x555555cae100) at libthreads-default-tls.c:350
  2    Thread 0x7ffff5b74700 (LWP 1390) "tlsglobals" 0x00007ffff6936951 in __GI___poll (fds=0x555555cb5080, nfds=1, timeout=0) at ../sysdeps/unix/sysv/linux/poll.c:29

diff --git a/src/arch/mpi/machine.c b/src/arch/mpi/machine.c
index 4646e9c74..5264cb70b 100644
--- a/src/arch/mpi/machine.c
+++ b/src/arch/mpi/machine.c
`` -1431,7 +1431,7 `` void LrtsInit(int *argc, char ***argv, int *numNodes, int *myNodeID) {
 #if CMK_MPI_INIT_THREAD
 #if CMK_SMP
     if (Cmi_smp_mode_setting == COMM_THREAD_SEND_RECV)
-        thread_level = MPI_THREAD_FUNNELED;
+        thread_level = MPI_THREAD_SERIALIZED;
       else
         thread_level = MPI_THREAD_MULTIPLE;
 #else
diff --git a/src/arch/util/machine-smp.c b/src/arch/util/machine-smp.c
index 255561e58..8b229be7e 100644
--- a/src/arch/util/machine-smp.c
+++ b/src/arch/util/machine-smp.c
`` -472,7 +472,7 `` static void CmiStartThreads(char **argv)
   Cmi_state_vector = (CmiState *)calloc(_Cmi_mynodesize+1, sizeof(CmiState));
 #if CMK_CONVERSE_MPI
       /* main thread is communication thread */
-  if(!CharmLibInterOperate) {
+  if(!CharmLibInterOperate && _thread_provided == MPI_THREAD_FUNNELED) {
     CmiStateInit(_Cmi_mynode+CmiNumPes(), _Cmi_mynodesize, &Cmi_mystate);
     Cmi_state_vector[_Cmi_mynodesize] = &Cmi_mystate;
   } else
`` -491,7 +491,7 `` static void CmiStartThreads(char **argv)
 #endif
   tocreate = _Cmi_mynodesize;
 #if CMK_CONVERSE_MPI
-  if(!CharmLibInterOperate) {
+  if(!CharmLibInterOperate && _thread_provided == MPI_THREAD_FUNNELED) {
     start = 0;
     end = tocreate - 1;                    /* skip comm thread */
   } else
`` -512,7 +512,7 `` static void CmiStartThreads(char **argv)
   }
 #if ! (CMK_HAS_TLS_VARIABLES && !CMK_NOT_USE_TLS_THREAD)
 #if CMK_CONVERSE_MPI
-  if(!CharmLibInterOperate)
+  if(!CharmLibInterOperate && _thread_provided == MPI_THREAD_FUNNELED)
     pthread_setspecific(Cmi_state_key, Cmi_state_vector+_Cmi_mynodesize);
   else
 #endif

Original date: 2018-04-30 15:24:24

Semi-related: I was just looking at -tlsglobals support in charmc, and it looks like we are adding "-Wl,--allow-multiple-definition" to the linker flags. Do we know if this is actually necessary? We sometimes have to add that flag to the linker flags when using Isomalloc, but other times it causes problems

Original date: 2018-04-30 21:43:40

I was able to resolve the issue with the following change:
diff --git a/src/arch/mpi-linux-x86_64/conv-mach-smp.h b/src/arch/mpi-linux-x86_64/conv-mach-smp.h
index f72decc97..b1fd3ccae 100644
--- a/src/arch/mpi-linux-x86_64/conv-mach-smp.h
+++ b/src/arch/mpi-linux-x86_64/conv-mach-smp.h
`` -10,3 +10,7 ``
 #undef CMK_TIMER_USE_SPECIAL
 #define CMK_TIMER_USE_GETRUSAGE                            1
 #define CMK_TIMER_USE_SPECIAL                              0
+
+#undef CMK_NOT_USE_TLS_THREAD
+#define CMK_NOT_USE_TLS_THREAD   1
+

I am unsure if this is a proper solution, since it is not needed for this purpose on any other platform. Would this change come at any performance cost?

Original date: 2018-04-30 21:49:18

~~https://charm.cs.illinois.edu/gerrit/4088~~ https://github.com/UIUC-PPL/charm/commit/2692b6e709cc03c3e9ba8fe20829841fe1036b9a

Original date: 2018-04-30 21:56:23

-It looks like non-SMP mpi-linux-x86_64 already defines CMK_NOT_USE_TLS_THREAD to 1. I'm not sure why this did not carry over to SMP builds, but at least I am more confident now in the change being okay.-
Disregard this, I was looking at mpi-darwin-x86_64.

Original date: 2018-04-30 21:59:22

Can you try out an mpi-crayxc build too, to see if that should also set this? We might need all MPI builds to do it...

Original date: 2018-05-01 19:29:48

Sam White wrote:

Can you try out an mpi-crayxc build too, to see if that should also set this? We might need all MPI builds to do it...

mpi-crayxc and mpi-crayxc-smp do not exhibit the issue.


add node v20 to CI matrix

Codecov Report
Patch coverage: 88.88% and no project coverage change.

Comparison is base (f95b15b) 98.55% compared to head (8d68325) 98.55%.
 Consider uploading reports for the commit 39a2e5b to get more accurate results

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #234   +/-   ##
=======================================
  Coverage   98.55%   98.55%
=======================================
  Files          16       16
  Lines        1035     1037    +2
  Branches      199      200    +1
=======================================
+ Hits         1020     1022    +2
  Misses         15       15

Files Changed
Coverage Δ

src/Encoder.ts
98.14% <ø> (ø)

src/decode.ts
100.00% <ø> (ø)

src/utils/utf8.ts
96.87% <ø> (ø)

src/Decoder.ts
99.04% <66.66%> (ø)

src/CachedKeyDecoder.ts
100.00% <100.00%> (ø)

src/ExtData.ts
100.00% <100.00%> (ø)

src/decodeAsync.ts
100.00% <100.00%> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


How to decode transaction instruction data

I need to decode transaction instruction data.
For example instruction is like this:
[
  {
    "accounts": [
      2,
      3,
      1,
      0
    ],
    "data": "g7gn3dpVguJSb",
    "programIdIndex": 4
  }
]

And instruction data bs58 decode result is:
0c010000000000000000

And transaction instruction decode result is:
{
  "info": {
    "authority": "3mqxFEsLMsahLSTU7Xhmeb8Zi9LJnKc2QYtuzn3Lxts2",
    "destination": "A5cmqXq4fepJBSfpjNYtc4qJo5NrW4m6AW6BUY5dimDh",
    "mint": "CAvvk8RGWCGERB4CHBxh2FX2vXc8Qkj7soEqLKTnYgq2",
    "source": "Bvu5cd49cWq7vLAQQmtGYXHkCguimzyUWA73hjHokkAn",
    "tokenAmount": {
      "amount": "1",
      "decimals": 0,
      "uiAmount": 1,
      "uiAmountString": "1"
    }
  },
  "type": "transferChecked"
}

How to get this result? Please help me.

reconstructing would require you to know the program instructions. the data for every program is different.
if this is really a transfer checked instruction then it is a system program ix, then you would be able to decode the data by deconstructing it using solana._layouts template. precisely the _TRANFER_LAYOUT.parse(data) function

reconstructing would require you to know the program instructions. the data for every program is different.
if this is really a transfer checked instruction then it is a system program ix, then you would be able to decode the data by deconstructing it using solana._layouts template. precisely the _TRANFER_LAYOUT.parse(data) function

Thanks. It's works fine.


Adding script to build and publish docker image

Adding docker support to build image.

Where do we publish the docker image? Apache Incubator had the restriction on releasing non-apache approved releases to public. Some guideline from the general@incubator for docker is the following.
Docker

Artefacts need to be placed in https://hub.docker.com/r/apache/<project> or https://hub.docker.com/u/apache<project>/<project>

To comply with ASF release and distributions please ensue the following:
- The overview should include the incubator disclaimer.
- The docker file should include an ASF header.
- The docker file should include the incubator disclaimer.
- "docker pull apache/<project>" should not install an artefact containing unapproved code.
- Release candidates, nightly or snapshots need to be clearly tagged.
- The latest tag should not point to an artefact containing unapproved code e.g. to master or dev branches or to a RC or snapshot.

@snleee, thanks for the pointers. I agree that we should all published releases under https://hub.docker.com/r/apache/ must be based on the released code. But individuals are free to take this and publish it under a different repo right?
I am assuming we will have an instruction page such as https://zeppelin.apache.org/docs/0.7.0/install/docker.html

Codecov Report

Merging #4200 into master will increase coverage by 0.04%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #4200      +/-   ##
============================================
+ Coverage     67.24%   67.28%   +0.04%
  Complexity       20       20
============================================
  Files          1041     1041
  Lines         51514    51514
  Branches       7216     7216
============================================
+ Hits          34640    34663      +23
+ Misses        14506    14485      -21
+ Partials       2368     2366       -2

Impacted Files
Coverage Δ
Complexity Δ

...r/validation/RealtimeSegmentValidationManager.java
43.33% <0%> (-36.67%)
0% <0%> (ø)

...a/manager/realtime/RealtimeSegmentDataManager.java
75% <0%> (-25%)
0% <0%> (ø)

...apache/pinot/common/metrics/ValidationMetrics.java
20.28% <0%> (-23.19%)
0% <0%> (ø)

...lix/EmptyBrokerOnlineOfflineStateModelFactory.java
86.66% <0%> (-13.34%)
0% <0%> (ø)

.../impl/dictionary/LongOffHeapMutableDictionary.java
83.92% <0%> (-10.72%)
0% <0%> (ø)

...nsport/netty/PooledNettyClientResourceManager.java
85.41% <0%> (-4.17%)
0% <0%> (ø)

.../org/apache/pinot/transport/netty/NettyServer.java
80.8% <0%> (-3.04%)
0% <0%> (ø)

...org/apache/pinot/common/metrics/MetricsHelper.java
72.5% <0%> (-2.5%)
0% <0%> (ø)

...esthandler/ConnectionPoolBrokerRequestHandler.java
86.33% <0%> (-1.87%)
0% <0%> (ø)

.../impl/dictionary/FloatOnHeapMutableDictionary.java
89.13% <0%> (ø)
0% <0%> (ø)
:arrow_down:

... and 22 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 45a8430...0b5d164. Read the comment docs.

I think we should maintain an official Pinot dockerhub account and make docker image publish as part of the release process.
This script could also be a reference for pinot users to publish their own images.

I think we should maintain an official Pinot dockerhub account and make docker image publish as part of the release process.
This script could also be a reference for pinot users to publish their own images.

+1 on making it part of our release process. I'll create an issue for this to be tackled in our 0.3 release.

Synced with @fx19880617 offline. Running the script will be no-op (permission denied). Users will set their custom tag for publishing to their own repository.
For the official apache release, we will need to set up apache/pinot repository and make the image publish to the repository as part of the release process.
The push refers to repository [docker.io/library/pinot]
b7c6cccc78d8: Preparing
6f2fe2187e57: Preparing
604345bc37d0: Preparing
e8f1623f0c2c: Preparing
e3b08ec2022d: Preparing
4230ff7f2288: Waiting
2c719774c1e1: Waiting
ec62f19bb3aa: Waiting
f94641f1fe1f: Waiting
denied: requested access to the resource is denied```


chore(deps): update gocloak to v11 to fix CVE-2020-26160

Description

This fix the Authorization bypass in github.com/dgrijalva/jwt-go issue.
jwt-go allows attackers to bypass intended access restrictions in
situations with []string{} for m["aud"] (which is allowed by the
specification). Because the type assertion fails, "" is the value of
aud. This is a security problem if the JWT token is presented to a
service that lacks its own audience check. There is no patch available
and users of jwt-go are advised to migrat to golang-jwt at version 3.2.1
which we already have in our go.mod
Verification Steps

Unit and integration test should pass
Code review from one of @akoserwal or @ziccardi as I'd like to get their expertly opinion on this upgrade of gocloak.

Checklist (Definition of Done)

[ ] All acceptance criteria specified in JIRA have been completed
[ ] Unit and integration tests added that prove the fix is effective or the feature works (tested against emulated and non-emulated OCM environment)
[ ] Documentation added for the feature
[ ] CI and all relevant tests are passing
[ ] Code Review completed
[ ] Verified independently by reviewer
[ ] Required metrics/dashboards/alerts have been added (or PR created).
[ ] Required Standard Operating Procedure (SOP) is added.
[ ] JIRA has created for changes required on the client side

@machi1990 - did you run the integration tests against your changes?

@machi1990 - did you run the integration tests against your changes?

CI completed successful so we should be good on that.

Adding a link to the the CVE-2020-26160


Crash bug

I've been trying to clean up the errors HoI4 throws. Early on, we had almost 9000 errors, but HoI4 loaded. Somewhere along the line of fixing errors, HoI4 stopped loading.

Fixes to decisions (b278e7697b984ad68078d5feaf69477c66716dbc) is good

Blank some division names (871924b6e85022e246f2b0d7e4ffe2e59e3c1fbc) is not good, therefore caused the crash.

Confirmed, removing the blanked files fixes the crash. But I don't know why this should be.

I suggest generating the files for the countries in the mod, by basing it on the primary culture. The file is supposed to use historical names for divisions, although in an althist they'd be mostly senseless.
The coding is:
[TAG]_INF_01 =
{
	name = "[Infantry Division]"
	for_countries = { [TAG] }
	can_use = { always = yes }
	division_types = { "infantry" }
	fallback_name = "%d [Infantry Division]"
	ordered = { } #Can be left blank
}

This is repeated in [TAG]_CAV_01 (Cavalry Division/"cavalry"), [TAG]_MOT_01 (Motor Division/"motorized"), [TAG]_MEC_01 (Mechanized Division/"mechanized"), [TAG]_ARM_01 (Armoured Division/"light_amor" "medium_armor" "heavy_armor" "modern_armor"), [TAG]_PAR_01 (Airborne Division/"paratrooper"), [TAG]_MAR_01 (Marine Division/"marine"), and [TAG]_MNT_01 (Mountain Division/"mountaineers").
The culturetoGraphics file could be updated with the entry-name for the equivalent of "Infantry Division" and so forth.

Right, long-term that's what we want to do. But in the short term, we need to remove the existing ones, and the simple method of just blanking the files causes HoI4 to crash.

As we did by "Jigme Wanchuck", we could simply generate a copy of the German division file as placeholder for now for the countries ([TAG] simply means the tag of the country, with the file itself being named [TAG]_names_divisions.txt): https://pastebin.com/iQ0Fc6p8

Which, again works for the countries we have in the mod. The immediate problem is the files for the countries that don't exist. They create 1289 lines worth of errors in the log, and we need some way of blanking them. Once that's done, then we can worry about adding new ones.


Add GitHub Action workflow to build and deploy actions

Bootstrap a bash script that runs hub build to compile and push docker
images for the actions that changed on Quay.io
The workflow only runs on push against the main branch.
Signed-off-by: Gianluca Arbezzano<EMAIL_ADDRESS>

Ok, it should work. I am merging it straight away because I want to see how it works "in prod"


Length function

Fixes: #60

make test returns this error
cd wdl2cwl/tests && cwltest --tool cwltool --test cwl_tests.yaml /bin/sh: 1: cwltest: not found make: *** [Makefile:142: cwltest] Error 127

@Th3nn3ss If you edit the top comment to include the phrase "Fixes: #60 " then that issue will automatically be closed when the PR gets merged

Should I add all the other task workflows even though they are not explicitly handled by the code I just added? I believe there are two more tasks in the bcftools.wdl I left out.

Only if they contain length() WDL calls

It's my pleasure.
Should I always delete a branch after is't been merged?

Should I always delete a branch after it's been merged?

I would, but it is your fork, so that is up to you.
Now that you've made two contribution to this repository I'm going to give you permission to make branches directly here (and therefore you won't need your fork). We've set branches in this repository to automatically be deleted after merging.


Added enabled parameter to EnvironmentVariableEntry

This parameter is important because some environment options that are disabled can become enabled after using xctestplanner

Thanks for the contribution 🚀


Update enzyme with jest configuration

Codecov Report

Merging #32 into master will increase coverage by 25%.
The diff coverage is 100%.

@@          Coverage Diff           @@
##           master    #32    +/-   ##
======================================
+ Coverage      75%   100%   +25%
======================================
  Files           2      3     +1
  Lines          16     17     +1
======================================
+ Hits           12     17     +5
+ Misses          4      0     -4

Impacted Files
Coverage Δ

src/__tests__/setup.js
100% <100%> (ø)

src/index.js
100% <0%> (+80%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6f24b85...80a3ccc. Read the comment docs.