problem starting pipenv on MacPorts

Hi,
I can't seem to be able to start pipenv after installing it. I don't know if I'm missing a step in the installation process or maybe my setup is not compatible.
I am using pip via MacPorts so my priority PATH locations are  /opt/local/bin and /opt/local/sbin. I am also using fish as my shell replacement (I actually was introduced to it and started using it because of the shell completion line in your documentation) and using virtualfish with the compatibility aliases activated to allow standard virtualenvwrapper commands (workon, deactivate.. etc).
I installed pipenv by doing sudo pip install pipenv. But after I try typing any of the pipenv commands I get fish: Unknown command 'pipenv'. Fish seems to recognize all my other usual commands except pipenv for some reason.

Hey @gruntfarm, thanks for taking the time to open this issue. So first off, I would strongly advise never using sudo with pip install commands. This can have unexpected side effects to system level modules and should be unneeded on macOS.
First we'll need to determine where sudo pip install actually put pipenv, and then determine whether that location is on the PATH. I'm assuming it's not because it can't find pipenv. Have you tried installing pipenv without sudo?

Hi @nateprewitt thanks for your response. using sudo pip install seems to have placed pipenv into /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pipenv
If I do just pip install pipenv, the install fails because using the MacPorts version of pip seems to want to place all pip installs into that /opt/local/ directory.
I can do pip install pipenv --user and that installs it into /Users/$USER/Library/Python/2.7/lib/python/site-packages/pipenv but again I still can't activate the pipenv command because that location is not on my PATH

@gruntfarm, so this very much seems to be an issue with your environment/macports, rather than pipenv. According to this Stackoverflow, you may need to add this directory to your PYTHONPATH.
$ export PYTHONPATH=/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages

Either way, I don't know if there's much for us to do from the pipenv side, so I'm going to close this out for now. Feel free to leave additional comments if you find any additional information not already provided here.

It looks like alternatively, you may supposed to be using python-select to make sure you're using the correct Python install. I'm not positive though because I don't have any experience with MacPorts.

Hm yes I have both that path and /opt/local/lib/python/site-packages on my PYTHONPATH.  Not sure what could be causing the issue. Looks like I'll have to stick to using virtualenvwrapper and pip separeately for now.

Ok new development. I hope I can post in here even though the issue is closed.
So I was able to run pipenv with this command python -m pipenv install, but when I do, this is the error I get:
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pipenv/__main__.py", line 4, in <module>
    cli()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pipenv/cli.py", line 960, in shell
    ensure_project(three=three, python=python, validate=False)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pipenv/cli.py", line 162, in ensure_project
    ensure_virtualenv(three=three, python=python)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pipenv/cli.py", line 137, in ensure_virtualenv
    do_create_virtualenv(three=three, python=python)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pipenv/cli.py", line 395, in do_create_virtualenv
    c = delegator.run(cmd, block=False)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pipenv/vendor/delegator.py", line 247, in run
    c.run(block=block, binary=binary)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pipenv/vendor/delegator.py", line 146, in run
    s = PopenSpawn(self._popen_args, **pexpect_kwargs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pipenv/vendor/pexpect/popen_spawn.py", line 45, in __init__
    self.proc = subprocess.Popen(cmd, **kwargs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 390, in __init__
    errread, errwrite)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1024, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

@gruntfarm, can you tell me what the output of python -m pipenv --venv is?

I get this message
No virtualenv has been created for this project yet!
Also just running pipenv like that without any arguments gives me the standard usage/commands doc.

Ok so I installed pipsi and pew (via pipsi) and added the pew and pipsi location to the PATH (different from the /opt/local one) and now the install command works!
But I still have to prepend with python -m, and just pipenv still isn't recognized as a shell command.

@gruntfarm want to hop on a hangouts?

Ooh I'd love to except I'm at my 9-5 right now which doesn't have anything to do with development (so I couldn't pass it off as work-related). But I could probably do it tonight around 8pm EST (sorry not sure what timezone you are in)

@gruntfarm ping me then and i'll see if i'm online!


Provided entry to create new text file in shell context menu

Since Windows Notepad can be uninstalled, uninstalling Notepad removes the entry to create new text file in shell context menu. This PR enables Notepads to provide this capability to users even if Windows Notepad is uninstalled
PR Type
What kind of change does this PR introduce?
Feature
Other information
If both Windows Notepad and Notepads are installed, there is only one context menu entry displayed(no duplicate entries). The context menu in file explorer may not be updated as soon as app is installed/uninstalled and explorer may require a restart.

“If both Windows Notepad and Notepads are installed, there is only one context menu entry displayed(no duplicate entries”
What if the translation of "Text Document" is different from the one comes in Windows Notepad? I guess it does not matter right?

What if the translation of "Text Document" is different from the one comes in Windows Notepad? I guess it does not matter right?

Yes, it doesn't matter.


Update fonts query string automatically

In 3.2.x we introduced {% static_with_version %} template tag for cache-busting purposes. Same needed to be done for static assets referenced in css, such as an icon font. Since then it was forgotten, so this PR introduces a postcss plugin to do that automatically.
Version will be read from cms/__init__.py

Coverage remained the same at 87.839% when pulling c90fe46077a2e6c7915953f952b703756c1932c9 on vxsx:feature/postcss-plugin-update-font-paths into 3ce6a53d2308ec4898a870bf82439808f2cc91c5 on divio:develop.

LGTM


If I set wifi auth==wpa ,node can't connect to root in esp mesh network (IDFGH-11521)

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.
v5.2
Espressif SoC revision.
ESP32
Operating System used.
Windows
How did you build your project?
VS Code IDE
If you are using Windows, please specify command line type.
None
Development Kit.
ESP32_DEVKIT_V4
Power Supply used.
USB
What is the expected behavior?
node connect to root
What is the actual behavior?
node can not connect to root
Steps to reproduce.
I use esp-idf\examples\mesh\internal_communication project.
Call esp_mesh_fix_root(true);
Call esp_mesh_set_type(MESH_NODE);(another development board call call esp_mesh_set_type(MESH_ROOT);)
Set Mesh AP Authentication Mode to CONFIG_WIFI_AUTH_WPA_PSK in "Example Configuration" in menuconfig.
Compile esp-idf\examples\mesh\internal_communication and flash.
Debug Logs.
ets Jun  8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:7140
load:0x40078000,len:15500
load:0x40080400,len:4
load:0x40080404,len:3904
entry 0x40080640
I (29) boot: ESP-IDF v5.2-dev-3903-g66992aca7a 2nd stage bootloader
I (29) boot: compile time Nov 23 2023 09:39:41
I (31) boot: Multicore bootloader
I (35) boot: chip revision: v1.0
I (38) boot.esp32: SPI Speed      : 40MHz
I (43) boot.esp32: SPI Mode       : DIO
I (48) boot.esp32: SPI Flash Size : 2MB
I (52) boot: Enabling RNG early entropy source...
I (58) boot: Partition Table:
I (61) boot: ## Label            Usage          Type ST Offset   Length
I (68) boot:  0 nvs              WiFi data        01 02 00009000 00006000
I (76) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (83) boot:  2 factory          factory app      00 00 00010000 00100000
I (91) boot: End of partition table
I (95) esp_image: segment 0: paddr=00010020 vaddr=3f400020 size=2c730h (182064) map
I (166) esp_image: segment 1: paddr=0003c758 vaddr=3ffb0000 size=038c0h ( 14528) load
I (172) esp_image: segment 2: paddr=00040020 vaddr=400d0020 size=ac1e0h (704992) map
I (413) esp_image: segment 3: paddr=000ec208 vaddr=3ffb38c0 size=0045ch (  1116) load
I (414) esp_image: segment 4: paddr=000ec66c vaddr=40080000 size=1658ch ( 91532) load
I (466) boot: Loaded app from partition at offset 0x10000
I (466) boot: Disabling RNG early entropy source...
I (478) cpu_start: Multicore app
I (487) cpu_start: Pro cpu start user code
I (487) cpu_start: cpu freq: 160000000 Hz
I (487) cpu_start: Application information:
I (490) cpu_start: Project name:     internal_communication
I (496) cpu_start: App version:      1
I (501) cpu_start: Compile time:     Nov 23 2023 09:45:01
I (507) cpu_start: ELF file SHA256:  28edbd050...
I (512) cpu_start: ESP-IDF:          v5.2-dev-3903-g66992aca7a
I (519) cpu_start: Min chip rev:     v0.0
I (523) cpu_start: Max chip rev:     v3.99
I (528) cpu_start: Chip rev:         v1.0
I (533) heap_init: Initializing. RAM available for dynamic allocation:
I (540) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
I (546) heap_init: At 3FFB9A00 len 00026600 (153 KiB): DRAM
I (552) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (559) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (565) heap_init: At 4009658C len 00009A74 (38 KiB): IRAM
I (573) spi_flash: detected chip: generic
I (576) spi_flash: flash io: dio
W (580) spi_flash: Detected size(4096k) larger than the size in the binary image header(2048k). Using the size in the binary image header.
I (594) main_task: Started on CPU0
I (604) main_task: Calling app_main()
I (654) wifi:wifi driver task: 3ffc1d04, prio:23, stack:6656, core=0
I (674) wifi:wifi firmware version: b43e2ce
I (674) wifi:wifi certification version: v7.0
I (674) wifi:config NVS flash: enabled
I (674) wifi:config nano formating: disabled
I (684) wifi:Init data frame dynamic rx buffer num: 32
I (684) wifi:Init static rx mgmt buffer num: 5
I (694) wifi:Init management short buffer num: 32
I (694) wifi:Init dynamic tx buffer num: 32
I (694) wifi:Init static rx buffer size: 1600
I (704) wifi:Init static rx buffer num: 10
I (704) wifi:Init dynamic rx buffer num: 32
I (714) wifi_init: rx ba win: 6
I (714) wifi_init: tcpip mbox: 32
I (714) wifi_init: udp mbox: 6
I (724) wifi_init: tcp mbox: 6
I (724) wifi_init: tcp tx win: 5744
I (734) wifi_init: tcp rx win: 5744
I (734) wifi_init: tcp mss: 1440
I (734) wifi_init: WiFi IRAM OP enabled
I (744) wifi_init: WiFi RX IRAM OP enabled
I (744) phy_init: phy_version 4780,16b31a7,Sep 22 2023,20:42:16
I (834) wifi:mode : sta (c4:4f:33:3e:cb:01) + softAP (c4:4f:33:3e:cb:02)
I (834) wifi:enable tsf
I (834) wifi:Total power save buffer number: 16
I (834) wifi:Init max length of beacon: 752/752
I (834) wifi:Init max length of beacon: 752/752
I (844) mesh: <nvs>read layer:0, err:0x1102
I (844) mesh: <nvs>read assoc:0, err:0x1102
I (854) wifi:Set ps type: 0, coexist: 0

I (854) mesh: [CONFIG]fail to set node, err:0x4008
I (874) wifi:Total power save buffer number: 16
I (1134) wifi:mode : sta (c4:4f:33:3e:cb:01)
I (1144) mesh: <MESH_NWK_LOOK_FOR_NETWORK>need_scan:0x1, need_scan_router:0x0, look_for_nwk_count:1
I (1144) mesh_main: <MESH_EVENT_MESH_STARTED>ID:77:77:77:77:77:77
I (1144) mesh_main: mesh starts successfully, heap:171536, root fixed<0>(tree), ps:0
I (1154) main_task: Returned from app_main()
I (1444) mesh: [FIND][ch:1]AP:7, otherID:0, MAP:0, idle:0, candidate:0, root:0[00:00:00:00:00:00][FIXED-ROOT]
I (1444) mesh: [FIND:1]fail to find a network, channel:0, cfg<channel:1, router:12345678, 00:00:00:00:00:00>

I (1454) mesh: <MESH_NWK_LOOK_FOR_NETWORK>need_scan:0x3, need_scan_router:0x1, look_for_nwk_count:2
I (1584) mesh: [FIND][ch:1]AP:0, otherID:0, MAP:0, idle:0, candidate:0, root:0[00:00:00:00:00:00][FIXED-ROOT]<scan router>
I (1584) mesh: [FIND:2]fail to find a network, channel:0, cfg<channel:1, router:12345678, 00:00:00:00:00:00>

I (1594) mesh: <MESH_NWK_LOOK_FOR_NETWORK>need_scan:0x3, need_scan_router:0x0, look_for_nwk_count:3
I (1904) mesh: [FIND][ch:1]AP:6, otherID:0, MAP:0, idle:0, candidate:0, root:0[00:00:00:00:00:00][FIXED-ROOT]
I (1904) mesh: [FIND:3]fail to find a network, channel:0, cfg<channel:1, router:12345678, 00:00:00:00:00:00>

I (1914) mesh: <MESH_NWK_LOOK_FOR_NETWORK>need_scan:0x3, need_scan_router:0x1, look_for_nwk_count:4
I (2044) mesh: [FIND][ch:1]AP:0, otherID:0, MAP:0, idle:0, candidate:0, root:0[00:00:00:00:00:00][FIXED-ROOT]<scan router>
I (2044) mesh: [FIND:4]fail to find a network, channel:0, cfg<channel:1, router:12345678, 00:00:00:00:00:00>

I (2054) mesh: <MESH_NWK_LOOK_FOR_NETWORK>need_scan:0x3, need_scan_router:0x0, look_for_nwk_count:5
I (2364) mesh: [FIND][ch:1]AP:8, otherID:0, MAP:0, idle:0, candidate:0, root:0[00:00:00:00:00:00][FIXED-ROOT]
I (2364) mesh: [FIND:5]fail to find a network, channel:0, cfg<channel:1, router:12345678, 00:00:00:00:00:00>

I (2374) mesh: <MESH_NWK_LOOK_FOR_NETWORK>need_scan:0x3, need_scan_router:0x1, look_for_nwk_count:6
I (2504) mesh: [FIND][ch:1]AP:0, otherID:0, MAP:0, idle:0, candidate:0, root:0[00:00:00:00:00:00][FIXED-ROOT]<scan router>
I (2504) mesh: [FIND:6]fail to find a network, channel:0, cfg<channel:1, router:12345678, 00:00:00:00:00:00>

I (2514) mesh: <MESH_NWK_LOOK_FOR_NETWORK>need_scan:0x3, need_scan_router:0x0, look_for_nwk_count:7
I (2824) mesh: find root:ESPM_3EECF5, root_cap:1(max:300), new channel:1, old channel:0
I (2824) mesh: [FIND][ch:1]AP:8, otherID:0, MAP:1, idle:0, candidate:0, root:1[00:00:00:00:00:00][FIXED-ROOT]
I (2824) mesh: [FIND:7]find a network, channel:1, cfg<channel:1, router:12345678, 00:00:00:00:00:00>

I (2834) mesh_main: <MESH_EVENT_FIND_NETWORK>new channel:1, router BSSID:00:00:00:00:00:00
I (2834) wifi:mode : sta (c4:4f:33:3e:cb:01) + softAP (c4:4f:33:3e:cb:02)
W (2854) wifi:<MESH AP>adjust channel:1, secondary channel offset:1(40U)
I (2864) wifi:Total power save buffer number: 16
W (2874) wifi:<MESH AP>adjust channel:1, secondary channel offset:1(40U)
I (2884) wifi:Total power save buffer number: 16
I (3184) mesh: [SCAN][ch:1]AP:8, other(ID:0, RD:0), MAP:1, idle:0, candidate:1, root:1, topMAP:0[c:0,i:0][00:00:00:00:00:00][FIXED-ROOT]<>
I (3184) mesh: 7370[selection]try rssi_threshold:-78, backoff times:0, max:5<-78,-82,-85>
I (3194) mesh: [DONE]connect to parent:ESPM_3EECF5, channel:1, rssi:-22, c4:4f:33:3e:ec:f6[layer:1, assoc:0], my_vote_num:0/voter_num:0, rc[00:00:00:00:00:00/-120/0]
I (4734) mesh: [wifi]disconnected reason:201(), continuous:1/max:12, non-root, vote(,)<><>
I (4734) mesh_main: <MESH_EVENT_PARENT_DISCONNECTED>reason:201
I (6234) mesh: [wifi]disconnected reason:201(), continuous:2/max:12, non-root, vote(,)<><>
I (6234) mesh_main: <MESH_EVENT_PARENT_DISCONNECTED>reason:201
I (7744) mesh: [wifi]disconnected reason:201(), continuous:3/max:12, non-root, vote(,)<><>
I (7744) mesh_main: <MESH_EVENT_PARENT_DISCONNECTED>reason:201
I (9244) mesh: [wifi]disconnected reason:201(), continuous:4/max:12, non-root, vote(,)<><>
I (9244) mesh_main: <MESH_EVENT_PARENT_DISCONNECTED>reason:201
I (10744) mesh: [wifi]disconnected reason:201(), continuous:5/max:12, non-root, vote(,)<><>
I (10744) mesh_main: <MESH_EVENT_PARENT_DISCONNECTED>reason:201

More Information.
No response

If I set wifi auth==WPA/WPA2, it works.

This issue does not seem to be caused by the auth mode. The error with this issue is that my AP password is randomly selected between "!" and "~". When I am randomly selected between "0" and "9", no matter which authentication method is used, it can work normally.

update!
I can connect when password include '0'-'9' and auth is wpa/wpa2
I can't connect when password include '!'-'~' and auth is wpa/wpa2
I can't connect when auth is wpa,no matter what password

@fzboffice Can you reproduce it just running the internal_communication project? I have tested on the internal_communication example, and set the password to mypass!word, mypass~word, !!!!~~~~, !~!~!~!~, they all can connect successfully.

@zhangyanjiaoesp not just '!''~',it is start from'!' to '~'.

@zhangyanjiaoesp Do you set auth mode to wpa?

@zhangyanjiaoesp Do you set auth mode to wpa?

Just test with the authmode set to WPA, and the node device can't found the AP indeed. And we will solve it ASAP.

@zhangyanjiaoesp Has this issue been solved?

The fix has been merged internally, and it will take some time to sync to Github.


[SPARK-6662][YARN] Allow variable substitution in spark.yarn.historyServer.address

In Spark on YARN, explicit hostname and port number need to be set for "spark.yarn.historyServer.address" in SparkConf to make the HISTORY link. If the history server address is known and static, this is usually not a problem.
But in cloud, that is usually not true. Particularly in EMR, the history server always runs on the same node as with RM. So I could simply set it to ${yarn.resourcemanager.hostname}:18080 if variable substitution is allowed.
In fact, Hadoop configuration already implements variable substitution, so if this property is read via YarnConf, this can be easily achievable.

Can one of the admins verify this patch?

@tgravescs @srowen
I just updated my patch that implementing Tom's suggestion-

The spark.yarn. properties are no longer copied into yarn configs.
A helper function that substitutes Hadoop variables ${hadoop- .. } is added to SparkHadoopUtil.

Do you think this is acceptable? Please let me know.

ok to test

Test build #29640 has started for   PR 5321 at commit 10e2917.

Test build #29640 has finished for   PR 5321 at commit 10e2917.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.
This patch does not change any dependencies.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29640/
Test PASSed.

@tgravescs , thank you very much for the review! I incorporated all your comments, and please let me know if you have any more comments.

Test build #29809 has started for   PR 5321 at commit 79757c6.

Test build #29809 has finished for   PR 5321 at commit 79757c6.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.
This patch does not change any dependencies.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29809/
Test PASSed.

Thank you Sandy. I made the change.

Test build #29885 has started for   PR 5321 at commit e37de75.

Thanks @piaozhexiu.
This seems reasonable to me.

Test build #29885 has finished for   PR 5321 at commit e37de75.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds no public classes.
This patch does not change any dependencies.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29885/
Test FAILed.

Jenkins failure seems due to a transient maven resolution error-
sbt.ResolveException: unresolved dependency: org.apache.spark#spark-core_2.10;1.2.0: not found

ok to test

sorry for the delay, It seems my permission to tell jenkins to run aren't working. hopefully I can get that resolved soon.
Otherwise this looks good.

jenkins, test this please

Test build #30171 has started for   PR 5321 at commit e37de75.

Test build #30171 has finished for   PR 5321 at commit e37de75.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.
This patch does not change any dependencies.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30171/
Test PASSed.

Looks good.

Thank you @tgravescs !


Wawawa option

Added wawawa option, as found in this video: https://www.youtube.com/watch?v=2v7e8LU-USI
This provides a more compact way to view your additional stats and achievement progress, as well as new stats. All credit for this goes to Wawawa (@wawawawawawawa).

Thanks for your patience. I've been working on getting a large amount of tasks done for v0.8. I am going to take a closer look at this over the weekend and see how difficult it will be to merge it all in.

I've got this working in the 0.8.3 branch. I will publish it to Muledump Preview after Thanksgiving.

This feature has been manually added to 0.8.3 rather than merged in.


Update dependencies, resolve TypeScript issues

This updates the service library and provides some fixes to the code to allow it to compile. It's unclear if this works: the unit tests aren't properly implemented and I have yet to test this against the Carebox API.

Having now done a manual test, this appears to still work, so merging.


running the CastVideo App on custom hardware

Hi,
I would like to run this CastVideo Sender App on snapdragon 820 based Custom Android hardware. So what are the dependencies to run it on Custom hardware?
Do i need to do the CTS? And licensing and certification also required from Google to run this Cast Sender App?
Thanks in advance.

Cast requires Google Play services. So your hardware would have to be certified by Google.

@entertailion Thanks!


Added directives to build the nFLUID package on Linux

Added two targets in Makefile:

apt-nfluid-deps
nfluid

@opazSG I have modified the script to install pyside from wheel.  Now the build can complete within the Travis time limit given to public repository.  However the nfluid is not yet tested in the framework:
Isn't "nfluid" to be loaded as a simphony plugin like simphony.visualisation.aviz and simphony.visualisation.mayavi_tools?  If so, please add "nfluid" to an entry point and create a test for it.  For example, entry points in simphony.engine and simphony.visualisation are tested in simphony-framework.tests.test_engine_import and simphony-framework.tests.test_visualisation_import.
nfluid may not fall into these categories and you may want to create a simphony.pre_processing entry point (I made it up).  In that case, please make a pull request in simphony-common.
And please add tests for nfluid so that the tests can be loaded and run here, https://github.com/simphony/nfluid/issues/32
Thanks!

@opazSG is there any update on this PR? It would be nice have this merged.

@kitchoi @itziakos Thanks for the comments, I will take this from now on and I will have it done ASAP

@kitchoi @itziakos This pr should be ready but I think it's failling because of some paraview related test suite, could you please help me out?

@SGGgarcia it's a known problem. I know how to fix it. I'll take it from here.

For reference https://github.com/simphony/simphony-paraview/issues/34
Action is to just restart the build until it passes. It's a paraview problem we can't fix at the moment, until someone replies.

Thanks @stefanoborini for the answer, I understand know. I'll keep restarting it until it passes


Photo Carousel Card Component

Nobody was assigned this component

I feel like there are about 3 of these on the site, surely this has already been done.


[Feature][Seatunnel-core-flink/spark] format usage help message

Purpose of this pull request
related issue: https://github.com/apache/incubator-seatunnel/issues/1787
With this PR, we can get a better output of usage help message
Before
./start-seatunnel-flink.sh -h
Usage: start-seatunnel-flink.sh [options] Options: -t, --check check config Default: false start-seatunnel-flink.sh start-seatunnel-spark.sh start-seatunnel-sql.sh -c, --config Config file -h, --help Show the usage message -r, --run-mode job run mode, run or run-application Default: RUN Possible Values: [RUN, APPLICATION_RUN] -i, --variable variable substitution, such as -i city=beijing, or -i date=20190318 Default: []

After
Usage: start-seatunnel-sql.sh [options]
  Options:
    -t, --check    check config (default: false)
  * -c, --config   Config file
    -h, --help     Show the usage message
    -r, --run-mode job run mode, run or run-application (default: RUN)
                   (values: [RUN, APPLICATION_RUN])
    -i, --variable variable substitution, such as -i city=beijing, or -i
                   date=20190318 (default: [])

Check list

[x] Code changed are covered with tests, or it does not need tests for reason:
[x] If any new Jar binary package adding in your PR, please add License Notice according
New License Guide
[x] If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs

Hi, @ruanwenjun , would you help review this tiny PR when you're available?
Thanks,
Kelu.

@legendtkl Which system are you using, Windows?

macOS Monterey  @ruanwenjun

macOS Monterey @ruanwenjun

It looks well on my laptop(macOS x86).

it's wired... This is my case.

Sorry, you are right, I didn't update my code.


ARROW-12626: [C++] Support toolchain xsimd, update toolchain version to version 8.1.0

13244

 Quality gate: Passed
1 Issues

 1 Code issues

Details

 Quality gate: Passed
1 Issues

 1 Code issues

Details


Add User States for game play

Ensure state changes in Move Boxes

Added 4 states:

Pending Dice roll by user
Pending Piece Selection by user to move
Nemesis Play state
Game over state


why dosen't logger's Println function return error when l.output() goes wrong

Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
1.6.3
What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOHOSTOS="linux"
What did you do?
I use logger to log some information in xxx.log file.
Then I keep the go process running, and remove the xxx.log file.
After that, xxx.log file cannot be create automatically, and nothing error was report!
If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.
What did you expect to see?
I expect that, if a logger is writing fail, some error should be return. Like:
func (l *Logger) Println(v ...interface{}) error {
return l.Output(2, fmt.Sprintln(v...))
}
What did you see instead?
I do nothing yet.

I suggest you to read man 2 unlink if you don't know this behavior.

unlink()  deletes  a  name  from the file system.  If that name was the
last link to a file and no processes have the file  open  the  file  is
deleted and the space it was using is made available for reuse.
If  the  name  was the last link to a file but any processes still have
the file open the file will remain in existence  until  the  last  file
descriptor referring to it is closed.
If the name referred to a symbolic link the link is removed.

If  the  name  referred  to a socket, fifo or device the name for it is
removed but processes which have the object open may  continue  to  use
it.

So you need to reopen the file (if you want to create new log file). One of the way to do, implement logger which open/close the file every logger.Println (etc).
In generally, utilities for rotating log files (ex: logrotate) send SIGHUP to the processes which open the file. If you want to handle the rotate of the file by them, you can catch the SIGHUP to re-open log files.

Checking errors for all log prints would be too tedious. The log system should be reliable. (What would you do if log returned an error? Log about it? Kill the program? Then just make the log sink kill the program on error.)
In any case, we can't change the API now, even if we wanted to.
Let's move discussion of log rotation & log implementations to golang-nuts or other forums. (https://golang.org/wiki/Questions)

thanks @mattn , I will try your suggestion.
thanks @bradfitz .


Pretrained model

@loveorchids @firmamentqj Thank you for your hard work,
Can you upload a pretrained model so I can test.
Waiting for your reply

Hi, as we did not spend much effort on OCR and Key Information Extraction tasks, only text detection model is available below. The result is expected to be around 94% in F1-score on SROIE test set:
https://drive.google.com/open?id=1uZnBuLm_DLKwNukYhwbrbDxUmLa2XtPe
Feel free to ask here if you meet any further problems.

Thanks,
Though how to use the pretrained model?

First, make sure you installed the repo and prepared the data, test data should be placed under ~/Pictures/dataset/ocr/SROIE2019_test by default).
Second, create a folder under ~/Pictures/dataset/ocr called "_text_detection".
Then place the pre-trained model under that folder and test with the data.
Feel free to ask here if you meet any further problems.

@loveorchids
I copied  this sample  to SROIE2019_test, the your task1_epoch_0203.pth to _text_detection folder, finally I ran tb_test.py and got error, have a look:
(sr) home@home-desktop:~/Documents/sroie2019/researches/ocr/textbox$ python tb_test.py
/home/home/Documents/sroie2019/omni_torch/utils/__init__.py:165: UserWarning: Cannot find models
  warnings.warn("Cannot find models")
Traceback (most recent call last):
  File "tb_test.py", line 278, in <module>
    test_rotation(opt)
  File "tb_test.py", line 140, in test_rotation
    if "module." + key not in weight_dict:
TypeError: argument of type 'SSD' is not iterable

Please paste your command to run the tb_test.py? I would like to check if any argument was mistaken.
For example python3 tb_test.py -mpl XXXXXXX

python C:/Users/35389/Documents/sroie2019/researches/ocr/textbox/tb_test.py --test_size 1000
C:/Users/35389/Documents/sroie2019\omni_torch\utils_init_.py:165: UserWarning: Cannot find models
warnings.warn("Cannot find models")
Traceback (most recent call last):
File "C:/Users/35389/Documents/sroie2019/researches/ocr/textbox/tb_test.py", line 256, in
test_rotation(opt)
File "C:/Users/35389/Documents/sroie2019/researches/ocr/textbox/tb_test.py", line 141, in test_rotation
if "module." + key not in weight_dict:
TypeError: argument of type 'SSD' is not iterable
getting this error please help me


feat(coding-guidelines): add guideline for C++

Signed-off-by: Takagi, Isamu<EMAIL_ADDRESS>Description
Add guideline for C++.
Related to https://github.com/autowarefoundation/autoware/discussions/2824.
Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.

[x] I've confirmed the contribution guidelines.
[x] The PR follows the pull request guidelines.

In-review checklist for the PR reviewers
The Reviewers must check the checkboxes below before approval.

[ ] The PR follows the pull request guidelines.

Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.

[ ] There are no open discussions or they are tracked via tickets.

After all checkboxes are checked, anyone who has write access can merge the PR.

@isamu-takagi May I update this PR to be a similar style to AUTOSAR's guidelines and C++ Core Guidelines?

@kenji-miyake Sounds good. Can you please?

@isamu-takagi Pushed my proposals. How about this?

@kenji-miyake Nice work! I think it's good.

Okay, then @xmfcx Could you briefly take a look?


Extract benchmarks to separate node module

I need to be able to run the benchmarks while developing, and this will allow me to remove the failing fitness test from the core helix-pi suite, which means it will run at proper unit test speeds!

Done!


syslog forwarding continues after removal of the syslog_forwarder job

Because rsyslog is always running on BOSH instance groups, and the syslog_forwarder job doesn't remove its rsyslog.conf from /etc/rsyslog.d/, a deploy that removes the syslog_forwarder job doesn't actually stop syslog from getting sent to the drain.
You can easily reproduce this by colocating the syslog_forwarder on some instance group, and then deploying again to remove the syslog_forwarder. You'll see that your drain (e.g. papertrail) continues to receive logs.
Could a drain script remove the rsyslog.conf and restart the rsyslog process?

@dsabeti is this something customers still want to do? If you redeploy with disabled, it should get removed.

Hey @Benjamintf1. I'm mostly speaking for myself here -- no users or customers have brought this up with me.
As far as using the disabled parameter, I assume you mean this one. It looks like a potential workaround, but I don't think anybody would think to look for that parameter as a way to stop syslogs from being forwarded to a drain.
Maybe what I'm saying is that this behavior is surprising because it's so different from most other (all other?) BOSH jobs. Operators expect that removing a job from an instance group should change the behavior. I should also mention that this a potentially expensive surprise, if you're forwarding logs to a service that charges you per log line: I originally stumbled onto this behavior when I filled up my papertrail quota.
I think you have a good implicit point that production operators are unlikely to remove the syslog forwarder from their deployments. Still, there are plenty of development teams that only want to set up syslog temporarily -- for example, to validate that new logs have the desired format or contain the correct information -- that it's worth addressing this behavior for them.

Yeah, it's something we've looked into and thought about. I'm not necessarily opposed to doing this in any sense, and think this actually might be a good idea. I'm just wondering how much need drives this, and how much this need has been customer validate, and if we're ok with the downtime we would get between stopping and starting(given the security implications surrounding the downtime)

We could clean up after ourselves in the drain, and probably should - I find the "I want to stop forwarding logs because something I'm paying for is full/expensive" use case compelling, and suspect that most users will accomplish this by just removing the ops file in cf-d. (Our commercial users do this by enabling the "disabled" property, but that's categorically the wrong approach for OSS.)
We'll talk about this in IPM. We've got good cleanup logic in our pre-start now, so it would be trivial to spread it around.

Manually made a story at https://www.pivotaltracker.com/story/show/156637315

Story was accepted.


Spliting the inverse solution of mixed source space into surface and volume space

How can the inverse solution( or "stcs") of mixed source space can be split into inverse solution of surface surce space and inverse solution of volume source space seperately. So that these two can be visualised seperately over MRI?
please find the code.
import numpy as np
import os.path as op
import matplotlib.pyplot as plt
import mne
from mne.datasets import sample
from mne import setup_source_space, setup_volume_source_space
from mne import make_forward_solution
from mne.minimum_norm import make_inverse_operator, apply_inverse
from nilearn.plotting import plot_stat_map
from nilearn.image import index_img
# Set dir
data_path = sample.data_path()
subject = 'sample'
data_dir = op.join(data_path, 'MEG', subject)
subjects_dir = op.join(data_path, 'subjects')
bem_dir = op.join(subjects_dir, subject, 'bem')
# Set file names
fname_mixed_src = op.join(bem_dir, '%s-oct-6-mixed-src.fif' % subject)
fname_aseg = op.join(subjects_dir, subject, 'mri', 'aseg.mgz')
fname_model = op.join(bem_dir, '%s-5120-bem.fif' % subject)
fname_bem = op.join(bem_dir, '%s-5120-bem-sol.fif' % subject)
fname_evoked = data_dir + '/sample_audvis-ave.fif'
fname_trans = data_dir + '/sample_audvis_raw-trans.fif'
fname_fwd = data_dir + '/sample_audvis-meg-oct-6-mixed-fwd.fif'
fname_cov = data_dir + '/sample_audvis-shrunk-cov.fif'
# Get a surface-based source space.
surf = setup_source_space(subject, spacing='ico5', add_dist=False, subjects_dir=subjects_dir)
# Get a volume-based source space.
labels_vol = ['Left-Hippocampus', 'Right-Hippocampus']
hippocampus= setup_volume_source_space(subject, mri=fname_aseg, pos=3.0, bem=fname_model,
                                 volume_label=labels_vol, mindist=3.0, subjects_dir=subjects_dir, verbose=True)
# Generate the mixed source space
src = surf+hippocampus
n = sum(src[i]['nuse'] for i in range(len(src)))
print('the src space contains %d spaces and %d points' % (len(src), n))
# Compute the fwd matrix
fwd = make_forward_solution(fname_evoked, fname_trans, src, fname_bem, mindist=5.0,
                                   meg=True, eeg=False, n_jobs=1)
leadfield = fwd['sol']['data']
print("Leadfield size : %d sensors x %d dipoles" % leadfield.shape)
src_fwd = fwd['src']
n = sum(src_fwd[i]['nuse'] for i in range(len(src_fwd)))
print('the fwd src space contains %d spaces and %d points' % (len(src_fwd), n))
# Load data
condition = 'Left Auditory'
evoked = mne.read_evokeds(fname_evoked, condition=condition, baseline=(None, 0))
noise_cov = mne.read_cov(fname_cov)
# Compute inverse solution and for each epoch
snr = 3.0           # use smaller SNR for raw data
inv_method = 'MNE'       # sLORETA, MNE, dSPM
lambda2 = 1.0 / snr ** 2
# Compute inverse operator
inverse_operator = make_inverse_operator(evoked.info, fwd, noise_cov,
                                         loose=1.0, depth=0.8, fixed=False)
stcs = apply_inverse(evoked, inverse_operator, lambda2, inv_method,
                                        pick_ori=None)
####################################################################
# Export result as a 4D nifti object
img = stcs.as_volume(src, mri_resolution=False)  # set True for full MRI resolution
# Save it as a nifti file
# nib.save(img, 'mne_%s_inverse.nii.gz' % method)
t1_fname = data_path + '/subjects/sample/mri/T1.mgz'
# Plotting with nilearn
plot_stat_map(index_img(img, 61), t1_fname, threshold=8., title='%s (t=%.1f s.)' % (method, stc.times[61]))
plt.show()

@poojaprabhu9 please don't put code on .docx files but like I did above or using https://gist.github.com/
basically the need is to go from the stcs above into 2 stcs one for the surface par and one for the volume part.
I would go with:
stc_surf, stc_vol = stcs.split(src)

for the background the objective is to include deep structures in MNE inverse and be able to get a surface stc that can be visualized with stc.plot
any volunteer to help? maybe @SherazKhan ?

Sure @agramfort . I will try with that.
Thanks

We have .volume and .surface methods nowadays, so this should be fixed

We have .volume and .surface methods nowadays, so this should be fixed


correctly resolve for babel-loader in example

As per the webpack documentation, node_modules is expected to be in the parent folder of webpack.config.js. Without this change, when I tried to run the project's example I got the following error:
    ERROR in ./~/html-webpack-plugin/lib/loader.js!../index.ejs
    Module not found: Error: Cannot resolve module 'babel-loader' in /Users/rbose85/Code/opensrc/html-webpack-template
     @ ./~/html-webpack-plugin/lib/loader.js!../index.ejs 1:8-92

Thanks for the fix!


Port to 1.13.2 forge with wasteland meteors

Any chance

Highly unlikely, forge took too long on 1.13 and I don't have much time or interest to play that version of Minecraft currently .. if many mods are released I may reconsider but I suspect most large ones will skip 1.13 because 1.14 will be out before they stabilize

What if i port it myself

The license of this project [2 clause BSD] was deliberately picked.  Allowing fairly free access to the source, without asking me, and with a clause to say you are mostly on your own.
Of course if you add an interesting feature into the base mod I'd be interested (if provided under the terms of the same BSD license) in a patch for future versions as I do intend to port this forward to forge 1.14.* and continue the fabric port.  (I admit infinite time I might make the stepping stone version to 1.13, however  my day job has way too much coding, and sometimes I need to play a games too)


[SPARK-33579][CORE] Fix executor blank page behind proxy

What changes were proposed in this pull request?

Fix some "hardcoded" API urls in Web UI.
More specifically, we avoid the use of location.origin when constructing URLs for internal API calls within the JavaScript.
Instead, we use apiRoot global variable.
Why are the changes needed?

On one hand, it allows us to build relative URLs. On the other hand, apiRoot reflects the Spark property spark.ui.proxyBase which can be set to change the root path of the Web UI.
If spark.ui.proxyBase is actually set, original URLs become incorrect, and we end up with an executors blank page.
I encounter this bug when accessing the Web UI behind a proxy (in my case a Kubernetes Ingress).
See the following link for more context:
https://github.com/jupyterhub/jupyter-server-proxy/issues/57#issuecomment-699163115
Does this PR introduce any user-facing change?

Yes, as all the changes introduced are in the JavaScript for the Web UI.
How the changes have been tested ?
I modified/debugged the JavaScript as in the commit with the help of the developer tools in Google Chrome, while accessing the Web UI of my Spark app behind my k8s ingress.

Can one of the admins verify this patch?

Jenkins test this please

Test build #131887 has started for PR 30523 at commit 346b007.

Seems OK. This would not change behavior if the proxy is not set, right? I'm trying to think of any case this would break, like framing the Spark UI in another app?
Can you make a JIRA?

There are already other uses of apiRoot for constructing URLs, especially in historypage.js.

Build finished.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/36484/

Test build #131887 has finished for PR 30523 at commit 346b007.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Build finished.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/131887/

Jenkins retest this please

Test build #131914 has started for PR 30523 at commit 346b007.

Build finished.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/36511/

Test build #131914 has finished for PR 30523 at commit 346b007.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Build finished.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/131914/

Seems fine to me too. cc @gengliangwang and @sarutak FYI

Seems OK. Let me try it on my local setup with reverse proxy.

Merged to master and branch-3.0. Thanks all !

Happy to contribute :+1: :star:


Fix push-pop logic in flow

Q
A

Bug fix?
yes

Breaking change?

New feature?

Deprecations?

Spec compliancy?

Tests added/pass?
yes

Fixed tickets
Fixes #404

License
MIT

@andy-ms I think that should be correct now, and tests seem to still work.

We moved the previous 7.0 branch to be master now, and master is now 6.x

Hey @danez! It looks like one or more of your builds have failed.

Node vlts/* - linux
Node vlts/* - linux


Expand c_void import to fix breakage on nightly

nightly changed std::os::raw::c_void from being a re-export to being a type alias. This causes Rust to be unsure whether the user meant to use libc::c_void or std::os::raw::c_void as both of those were imported using glob imports. Fixing this issue involves expanding one of those glob imports to avoid the glob import conflict.

published as jerk 0.2.3


Added microbatchsize CLI option to override micro_batch_size.

Not sure what your policy is for using globals to pass around settings, but this commit allows setting micro_batch_size with the microbatchsize CLI option with the fewest changes to the original code.
Tested all three finetune scripts with the stablelm-base-alpha-3b model and Alpaca fine tuning dataset; verified that GPU memory usage went down as --microbatchsize was reduced.

I vote for not making the change: The philosophy behind the scripts is that they are minimal and easy to understand. They are templates that are meant to be adapted and transformed to the needs of the researcher, nano-GPT style. My personal opinion is that, while it is possible to expose every single hyperparameter as a CLI argument in these scripts, it will make lit-gpt appear more as a framework which it is not meant to be, and take away from the hackability. Furthermore, every use case is different and the field is moving forward fast, so hacking the scripts will almost certainly be always needed downstream beyond just changing hyperparameters.
Specifically here, for sure changing the micro-batch size is important as this will impact the GPU memory usage. We could also do a better job in explaining its use, e.g. as a comment in-line or give recommendations for common GPU types in the tutorials.


Submitting SDK 9.1 builds to itunes fails on CFBundleExecutable

ERROR ITMS-90535: "Unexpected CFBundleExecutable Key. The bundle at 'Payload/Application.app/MixpanelResources.bundle' does not contain a bundle executable. If this bundle intentionally does not contain an executable, consider removing the CFBundleExecutable key from its Info.plist and using a CFBundlePackageType of BNDL. If this bundle is part of a third-party framework, consider contacting the developer of the framework for an update to address this issue."

Are you using Carthage or Cocoapods?

Resolved in #417

This is now released as v2.9.2, thanks for your patience!


Adding a channel for scans.

There was a problem doing mutations while a scan was ongoing.  Adding a channel specifically for scans.

Renamed the channels and added a comment in BigtableGrpcClient.

LGTM


Android back button closing the app

User reported:
At some point, AppStudio has got to support the Android back button properly. No Android user would ever expect the back button to close an app except from the app's home screen. It is seriously annoying and is going to be problematic for our users.
But since the back button should at least close the app cleanly...open the Esri Campus Map. Tap the Get Started button. On the list of photos, tap a photo and then very quickly tap the Android back button. The app freezes, and you have to kill the process if you want to use it again.

Resolved


'Composite: true + Incremental: true' excessively increases compilation times

Bug Report

Background

We're in attempt to establish project references by splitting codes into multiple tsconfig projects. First migration attempt trying to make small project, then reference existing monolithic codebase. Enabling composite option to existing tsconfig however, makes overall compilation time excessively longer which we can't try this approach.

tsconfig
Please note few names are redacted.

{
	"include": [
		"bin/**/*",
		"js/**/*",
		"webpack/**/*"
	],
	"exclude": ["**/node_modules", "**/.*", "js/ts-plugins/**/*"],
	"compilerOptions": {
		"allowJs": true,
		"allowSyntheticDefaultImports": true,
		"baseUrl": ".",
		"esModuleInterop": true,
		"importHelpers": true,
		"jsx": "react",
		"lib": ["dom", "es2020"],
		"downlevelIteration": true,
		"module": "esnext",
		"moduleResolution": "node",
		"target": "es5",
		"composite": true,
		"incremental": true,
		"outDir": "./.ts",
		"resolveJsonModule": true,
		"paths": {
			"@libs/*": ["js/libs/*"],
			"@docs/*": ["js/${...}/app/docs/*"],
                         ...//few other pathMappings
		},
		"noImplicitAny": true,
		"noImplicitReturns": true,
		"noImplicitThis": true,
		"noUnusedLocals": true,
		"noUnusedParameters": true,
		"strictNullChecks": true,
		"strictPropertyInitialization": true,
		"plugins": [
			{
				"name": "../../js/ts-plugins/dist/custom-plugin/index"
			}
		]
	}
}

Compilation time

Compilation executed via
node --max-old-space-size=16384 ./node_modules/typescript/bin/tsc -p ./tsconfig.json --extendedDiagnostics

due to there are some memory pressure causes OOM without increasing heap size.
composite: false, incremental: false

Files:                        15454
Lines:                      2360040
Nodes:                      8724975
Identifiers:                2858297
Symbols:                    5053080
Types:                      2160538
Instantiations:             6523057
Memory used:               7364254K
Assignability cache size:   1029646
Identity cache size:          80555
Subtype cache size:           98716
Strict subtype cache size:   125795
Tracing time:               222.07s
I/O Read time:                1.33s
Parse time:                   9.58s
ResolveModule time:           9.42s
ResolveTypeReference time:    0.08s
Program time:                29.52s
Bind time:                    6.74s
Check time:                 478.03s
transformTime time:          11.61s
commentTime time:            63.43s
I/O Write time:               5.99s
printTime time:             228.94s
Emit time:                  228.94s
Dump types time:            185.83s
Total time:                 743.23s

composite: true, incremental: true

Files:                         15454
Lines:                       2360040
Nodes:                       8724975
Identifiers:                 2858297
Symbols:                     5067727
Types:                       2208351
Instantiations:              6654654
Memory used:                5015179K
Assignability cache size:    1030212
Identity cache size:          131336
Subtype cache size:            98687
Strict subtype cache size:    125733
Tracing time:                259.28s
I/O Read time:                 3.18s
Parse time:                   10.27s
ResolveModule time:           10.52s
ResolveTypeReference time:     0.11s
Program time:                 34.22s
Bind time:                     6.90s
Check time:                  562.21s
transformTime time:        14012.56s
commentTime time:            261.38s
printTime time:            14893.35s
Emit time:                 14894.61s
I/O Write time:               13.35s
Dump types time:             377.50s
Total time:                15497.94s

When enabled, other stats are nearly similar but only transformTime takes excessively longer on same machine.
Below's an example file from generated trace shows single file's transformNodes duration differences.
composite: false, incremental: false

{
  "pid": 1,
  "tid": 1,
  "ph": "X",
  "cat": "emit",
  "ts": 560591824.5819993,
  "name": "transformNodes",
  "dur": 1967.9180011749268,
  "args": {
    "path": "/users/.../${redacted}.js"
  }
}

composite: true, incremental: true

{
  "pid": 1,
  "tid": 1,
  "ph": "X",
  "cat": "emit",
  "ts":<PHONE_NUMBER>9.568998,
  "name": "transformNodes",
  "dur": 134794797.68300056,
  "args": {
    "path": "/users/.../${redacted}.js"
  }
}

This looks like mostly occurring around .js files since we have allowJs: true and majority of files are still .js, but some of .ts and .tsx are large offenders around long transformTime as well.
It looks like https://github.com/microsoft/TypeScript/issues/39969 is somewhat similar while it doesn't look like exact same.
🔎 Search Terms
Incremental build slow

🕗 Version & Regression Information

Experiencing tsc >= 4.1.3

⏯ Playground Link

💻 Code

// We can quickly address your report if:
//  - The code sample is short. Nearly all TypeScript bugs can be demonstrated in 20-30 lines of code!
//  - It doesn't use external libraries. These are often issues with the type definitions rather than TypeScript bugs.
//  - The incorrectness of the behavior is readily apparent from reading the sample.
// Reports are slower to investigate if:
//  - We have to pare too much extraneous code.
//  - We have to clone a large repo and validate that the problem isn't elsewhere.
//  - The sample is confusing or doesn't clearly demonstrate what's wrong.

Due to internal policies, it is not possible to share actual code unfortunately. (and it's large)
🙁 Actual behavior

Whole compilation time takes more than hours
🙂 Expected behavior

Compilation time would increase, but within range of original compile time which takes less than 10 min in worst case.

Interesting, thanks for the pointer. It looks like somewhat related and would like to try out those changes.


[release-v1.11] Add makefile target for kafka keda tests

This is an attempt at adding a makefile target for the keda tests.

openshift/e2e-common.sh: line 75: INSTALL_KEDA: unbound variable


Delete manifests/j/JetBrains/Toolbox/<IP_ADDRESS>528 directory

this is an EAP version
Checklist for Pull Requests

[ ] Have you signed the Contributor License Agreement?
[ ] Is there a linked Issue?

Manifests

[ ] Have you checked that there aren't other open pull requests for the same manifest update/change?
[ ] This PR only modifies one (1) manifest
[ ] Have you validated your manifest locally with winget validate --manifest <path>?
[ ] Have you tested your manifest locally with winget install --manifest <path>?
[ ] Does your manifest conform to the 1.6 schema?

Note: <path> is the directory's name containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

URL: https://download.jetbrains.com/toolbox/jetbrains-toolbox-<IP_ADDRESS>528.exe
Status Code: 200
(Automated message - build 897)

URL: https://download.jetbrains.com/toolbox/jetbrains-toolbox-<IP_ADDRESS>528.exe Status Code: 200
(Automated message - build 897)
Edit: Hi @mysmlz,
This package is still available - why is it being removed?

It's an EAP(Early Access Program) version,not a stable version.

URL: https://download.jetbrains.com/toolbox/jetbrains-toolbox-<IP_ADDRESS>528.exe
Status Code: 200
(Automated message - build 897)

It's an EAP(Early Access Program) version,not a stable version.

Thanks - should it be readded under a similar PackageIdentifier - possibly JetBrains.Toolbox.EAP?


dns: Don't log DNS requests and responses

The logs are too noisy with all the request and response logs,
especially at scale.

@ejj requesting a review from you since you wrote this initially


  We have no mechanism to free them.
We should free dead Gs if they sit in the global free queue for long enough.  Or maybe free all of them at each GC?
I noticed this while debugging #8832.  The stacks for dead Gs are freed at GC time.  This leads to a fragmented heap because spans for G storage and stack storage alternate in the heap.  When only the stacks are freed, the resulting free spans won't coalesce because the spans for storing the Gs aren't freed.

@dvyukov

Rephrased as "dead Gs cause span fragmentation". It's not clear we can easily free dead G's - it's hard to be sure all the pointers to them have been found. I expect there is code in the runtime that depends on G's never being freed (reused to hold other types of data). I don't want to debug that.
How big a problem is the fragmentation?

The fragmentation can be pretty bad, the case in #8832 makes the heap into a repeated pattern of one page allocated (for Gs) and eight pages free (once used for stacks).  So although the heap is ~90% free, we'd have to grow the heap for any large allocation.
That said, I don't think it is terribly common to use lots of Gs for a while and then never use them again.
I agree this may be hard.  But it is something to think about if we can find a way.

That said, I don't think it is terribly common to use lots of Gs for a while and then never use them again.

Most network servers tend to create a new goroutine per incoming request (E.g., the HTTP server). If there's a sudden huge spike in requests (often not intentional, but due to a misbehaving external service), it can cause a huge number of goroutines to be created, and even after the requests have been served, the process consumes a huge amount of memory. I'm assuming this is due to the fragmentation described in this issue.
As an example, we saw one process in production with a system usage of > 100GB, while the Go runtime thought it was using < 4GB.
Pprof output:

ps output:

I am seeing a similar issue I think. runtime.malg is the most prolific. It is possible that this is in response to a sudden influx of requests.

@mandarjog : Why do you think this is related to fragmentation?
malg allocates goroutine stacks.  Maybe you just have a lot of goroutines and thus lots of stack.
This issue is about what happens after all those goroutines complete.  The syndrome I would expect to see is that the heap is mostly unused, including having little memory used by stacks, but large allocations still grow the heap.

After the go routines are done, I expect the usage to come down.
Should we expect the amount active set of 'G' to be at the high watermark of go routines?
https://user-images.githubusercontent.com/18554027/31787111-a33ae35e-b4d8-11e7-9cf1-2410427c1473.png
We see here that externally measured memory comes back down, but does not go all the way down.
I am not certain that fragmentation is the issue, but it is possible.

Once a G is allocated, it is never freed, so yes the number of Gs at any point is the high water mark of the execution so far.
But, that's only for the G descriptor itself.  That does not include a G's stack.  Stacks of finished Gs are freed during garbage collection, so that space should not remain at the high water mark.
Stacks are 90%+ of the space used for a goroutine, so I don't think that can account for your graph - you're retaining at least 30% of the high water mark.
This issue is unlikely to affect the amount of memory used as reported by the OS.  The fragmentation described in this issue is only for virtual addresses - all the holes described here can (and are) given back to the OS by the scavenger. (Unless you're running on an OS where the page size is >8K).
If you'd like to continue investigating, please open a new issue.

I'm seeing this issue in an application I wrote as well. It is an HTTP server which essentially kicks off a specialized background web crawler. A "scan" request comes to the http server which throws the crawler into a goroutine and responds to the http request with "scan started successfully". Unfortunately, once it allocates memory, it never releases it no matter how long I wait. I wrote a simple program to show what is happening below. I posted on golang-nuts and was led here. Below is my post in its entirety. For me, this is a showstopper issue. If Go won't release memory from a very simple goroutine and it takes up gigs of memory (on my production app), I'll have to rewrite using something else.
Golang-nuts post:
I've got a nasty memory leak. It appears that Go won't reclaim memory when goroutines are called in a highly parallel fashion.
To illustrate the issue I've written a simple example below. When I run this on my Windows system, I get 800MB to 1 GB of memory consumption.
What's fascinating is that if I change the time.Sleep to 100 milliseconds instead of 1, the process never goes above 6 MB of memory. This is what I would expect since there are no objects here that should be retained in memory. And, it doesn't matter how long I wait, the garbage collector never cleans up the mess either.
I've tried to profile it using pprof and it didn't help me. It helped me find and fix other issues, but not this. It's entirely possible I did it wrong though since I am new to using that tool.
Help! Thanks in advance!
package main
import (
"bytes"
"fmt"
"io/ioutil"
"math/rand"
"runtime"
"time"
)
func main() {
for i := 0; i < 1000; i++ {
time.Sleep(time.Millisecond * 1)
go fakeGetAndSaveData()
}
runtime.GC()
time.Sleep(10 * time.Minute)
}
func fakeGetAndSaveData() {
var buf bytes.Buffer
for i := 0; i < 40000; i++ {
buf.WriteString(fmt.Sprintf("the number is %d\n", i))
}
ioutil.WriteFile(fmt.Sprintf("%d.txt", rand.Int()), buf.Bytes(), 0644)
}

@robarchibald I'm pretty sure what you are seeing is not related to this issue.  Please open a new one.  The reason I'm pretty sure is that the space used is all for bytes.Buffer, it isn't for the goroutine descriptors or their stacks.  Even if we retained the descriptors and their stacks, it is only ~8MB (stacks on Windows are 8K to start).
How are you measuring the memory used by the process?  The Go runtime should give all the unused memory back to the OS after 5 minutes.  Although I admit I'm unsure how this is handled on Windows.
The difference between a 1ms sleep and a 100ms sleep is probably that in the latter, the each goroutine finishes before the next one is generated, so there is only one goroutine at a time.  In the former, there are lots of goroutines simultaneously, each one using almost 1MB for its buffer.

Thanks @randall77. I'm measuring memory using the OS. The results are the same whether in Ubuntu 16.04 (production) or Windows 10 (my dev box). And, I specifically set the time.Sleep to 10 minutes because I know that Garbage collection is supposed to cleanup after 5 minutes. Plus, I explicitly call runtime.GC() immediately in case I could get it to collect the trash earlier. No dice either way. Go never releases the memory even after waiting a week. Sorry, I wasn't patient enough to wait longer than that. :)
I can certainly open a new issue, but this isn't an issue with bytes.Buffer. This is a problem with goroutine cleanup. Sure, this particular issue was opened because the descriptor and stack wasn't cleaned up, but I think I'm showing here that the issue is MUCH bigger than that. Nothing is getting cleaned up if the system is busy enough or if the parallelism is high enough... at least that's what I'm assuming here since I can change the delay to be longer and the problem goes away.
It isn't a bytes.Buffer issue. It isn't an ioutil.WriteFile issue. I only used bytes.Buffer for this example and that's not what I'm using in my real application. And, I can do a fmt.Println instead of ioutil.WriteFile and it still does the same thing too. As @prashantv mentioned, he's seen 100 GB of memory usage due to what sounds like a similar issue to mine.

@6degreeshealth Show us exactly how you are measuring memory.  Ideally on Ubuntu.  Don't use words to tell us what you are doing, show us the exact commands that you are using, and the exact output that you see.  Thanks.

On Linux, I'm using top -b | grep memoryLeak command and on Windows, I use Task Manager. I just ran this again on Ubuntu 16.04 and it looks like it isn't exhibiting the problem after all. When I ran it yesterday it crashed my VM, but it wasn't due to out of memory as I'd assumed. But, I do see the problem clearly on Windows. Sorry, I'll put together a different example for Linux.
Here's what I see on Windows. This is after it's gone idle. Nothing new is being spawned. It's just sitting there. It stays like this for 10 minutes until the program closes.

I've opened #22439 to discuss @6degreeshealth 's issues.
Let's leave this bug for G fragmentation issues.

I'm seeing a similar problem with a golang program . The program spawns a good number of goroutines throughout the day . I've noticed a 200mb increase in memory hold at the end of the day. pprof heap returns :

Showing nodes accounting for 189.55MB, 85.32% of 222.16MB total
Dropped 41 nodes (cum <= 1.11MB)
Showing top 10 nodes out of 89
flat  flat%   sum%        cum   cum%
93.53MB 42.10% 42.10%    93.53MB 42.10%  runtime.malg

200mb pileup on a daily basis is a huge amount . Is there any update on this issue ? Any possible workarounds?

@prashanthjbabu
The fact that malg is the culprit means it probably isn't this issue.
This issue is about fragmentation after the goroutine stacks are freed. You're seeing goroutine stacks that haven't been freed yet (malg allocates goroutine stacks).
Could you open a new issue, ideally with code to reproduce the problem?

@randall77
Thanks for your response ! I ended up in this issue while looking at https://github.com/golang/go/issues/16022 which also talks about malg holding up too much memory. In my case too malg is holding up memory . So I thought my issue is similar to https://github.com/golang/go/issues/16022 and since that issue was pointing to this one I thought I'd post here.

@prashanthjbabu I guess it is possible your stacks are being freed, and the space use you're seeing is just G descriptors. That would take close to 1 million goroutines.  Do you have that many goroutines live at once? Or perhaps you have a goroutine leak?
It could also be a bug in the runtime that leaks descriptors. To track such a thing down, we'd need code to reproduce your bug.

@randall77
Thanks for your response ! After you mentioned the possibilities I ran the pprof goroutine profile and realized that there was indeed a goroutine leak like you mentioned . Thanks for the help!

I think I have the similar picture in my go web app:
kubectl top pod repolite-5f9f6bb846-tfnvz --containers

POD                         NAME          CPU(cores)   MEMORY(bytes)
repolite-5f9f6bb846-tfnvz   repolite      2m           1543Mi

Maybe I am wrong, but I don't see anything strange:
 ~ curl "http://<HOST>/debug/pprof/goroutine?debug=1"
goroutine profile: total 4
1 @ 0x439b36 0x432253 0x464fa9 0x4ca172 0x4cb4da 0x4cb4c8 0x58aca9 0x597965 0x66386d 0x54e3c3 0x54e51d 0x669396 0x46a4e1
#	0x464fa8	internal/poll.runtime_pollWait+0x88		/usr/local/go/src/runtime/netpoll.go:234
#	0x4ca171	internal/poll.(*pollDesc).wait+0x31		/usr/local/go/src/internal/poll/fd_poll_runtime.go:84
#	0x4cb4d9	internal/poll.(*pollDesc).waitRead+0x259	/usr/local/go/src/internal/poll/fd_poll_runtime.go:89
#	0x4cb4c7	internal/poll.(*FD).Read+0x247			/usr/local/go/src/internal/poll/fd_unix.go:167
#	0x58aca8	net.(*netFD).Read+0x28				/usr/local/go/src/net/fd_posix.go:56
#	0x597964	net.(*conn).Read+0x44				/usr/local/go/src/net/net.go:183
#	0x66386c	net/http.(*connReader).Read+0x16c		/usr/local/go/src/net/http/server.go:780
#	0x54e3c2	bufio.(*Reader).fill+0x102			/usr/local/go/src/bufio/bufio.go:101
#	0x54e51c	bufio.(*Reader).Peek+0x5c			/usr/local/go/src/bufio/bufio.go:139
#	0x669395	net/http.(*conn).serve+0xc35			/usr/local/go/src/net/http/server.go:1955

1 @ 0x439b36 0x432253 0x464fa9 0x4ca172 0x4cd4cc 0x4cd4b9 0x58c455 0x5a09e8 0x59fbbd 0x66e134 0x66dd5d 0x82a669 0x834885 0x439767 0x46a4e1
#	0x464fa8	internal/poll.runtime_pollWait+0x88		/usr/local/go/src/runtime/netpoll.go:234
#	0x4ca171	internal/poll.(*pollDesc).wait+0x31		/usr/local/go/src/internal/poll/fd_poll_runtime.go:84
#	0x4cd4cb	internal/poll.(*pollDesc).waitRead+0x22b	/usr/local/go/src/internal/poll/fd_poll_runtime.go:89
#	0x4cd4b8	internal/poll.(*FD).Accept+0x218		/usr/local/go/src/internal/poll/fd_unix.go:402
#	0x58c454	net.(*netFD).accept+0x34			/usr/local/go/src/net/fd_unix.go:173
#	0x5a09e7	net.(*TCPListener).accept+0x27			/usr/local/go/src/net/tcpsock_posix.go:140
#	0x59fbbc	net.(*TCPListener).Accept+0x3c			/usr/local/go/src/net/tcpsock.go:262
#	0x66e133	net/http.(*Server).Serve+0x393			/usr/local/go/src/net/http/server.go:3002
#	0x66dd5c	net/http.(*Server).ListenAndServe+0x7c		/usr/local/go/src/net/http/server.go:2931
#	0x82a668	repolite2/internal/api.Run+0x828		/build/internal/api/api.go:101
#	0x834884	main.main+0x1e4					/build/main.go:46
#	0x439766	runtime.main+0x226				/usr/local/go/src/runtime/proc.go:255

1 @ 0x464ac5 0x6bdfd5 0x6bdded 0x6baf6b 0x6c9b7a 0x6ca72e 0x66a18f 0x66bf49 0x82a88c 0x66a18f 0x66dbbb 0x669268 0x46a4e1
#	0x464ac4	runtime/pprof.runtime_goroutineProfileWithLabels+0x24	/usr/local/go/src/runtime/mprof.go:746
#	0x6bdfd4	runtime/pprof.writeRuntimeProfile+0xb4			/usr/local/go/src/runtime/pprof/pprof.go:724
#	0x6bddec	runtime/pprof.writeGoroutine+0x4c			/usr/local/go/src/runtime/pprof/pprof.go:684
#	0x6baf6a	runtime/pprof.(*Profile).WriteTo+0x14a			/usr/local/go/src/runtime/pprof/pprof.go:331
#	0x6c9b79	net/http/pprof.handler.ServeHTTP+0x499			/usr/local/go/src/net/http/pprof/pprof.go:253
#	0x6ca72d	net/http/pprof.Index+0x12d				/usr/local/go/src/net/http/pprof/pprof.go:371
#	0x66a18e	net/http.HandlerFunc.ServeHTTP+0x2e			/usr/local/go/src/net/http/server.go:2047
#	0x66bf48	net/http.(*ServeMux).ServeHTTP+0x148			/usr/local/go/src/net/http/server.go:2425
#	0x82a88b	repolite2/internal/api.loggingMiddleware.func1+0x6b	/build/internal/api/api.go:111
#	0x66a18e	net/http.HandlerFunc.ServeHTTP+0x2e			/usr/local/go/src/net/http/server.go:2047
#	0x66dbba	net/http.serverHandler.ServeHTTP+0x43a			/usr/local/go/src/net/http/server.go:2879
#	0x669267	net/http.(*conn).serve+0xb07				/usr/local/go/src/net/http/server.go:1930

1 @ 0x4b2c25 0x4b096d 0x4cb505 0x4cb4ed 0x4cb2a5 0x58aca9 0x597965 0x6633bf 0x46a4e1
#	0x4b2c24	syscall.Syscall+0x4				/usr/local/go/src/syscall/asm_linux_amd64.s:20
#	0x4b096c	syscall.read+0x4c				/usr/local/go/src/syscall/zsyscall_linux_amd64.go:687
#	0x4cb504	syscall.Read+0x284				/usr/local/go/src/syscall/syscall_unix.go:189
#	0x4cb4ec	internal/poll.ignoringEINTRIO+0x26c		/usr/local/go/src/internal/poll/fd_unix.go:582
#	0x4cb2a4	internal/poll.(*FD).Read+0x24			/usr/local/go/src/internal/poll/fd_unix.go:163
#	0x58aca8	net.(*netFD).Read+0x28				/usr/local/go/src/net/fd_posix.go:56
#	0x597964	net.(*conn).Read+0x44				/usr/local/go/src/net/net.go:183
#	0x6633be	net/http.(*connReader).backgroundRead+0x3e	/usr/local/go/src/net/http/server.go:672

My pprof inuse_space report:
go tool pprof -png http://<host>/debug/pprof/heap

I'm able to run top inside the container:

I've added another handler with debug.FreeOSMemory():
func freeHandler() http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		debug.FreeOSMemory()

		w.WriteHeader(http.StatusOK)
		w.Header().Set("Content-Type", "application/text")
		w.Write([]byte("memfree"))

	})
}

But triggering the handler makes no effect. I have been waiting for several days, but my memory wasn't returned to OS.
I'm using Go 1.17.8
still wondering which entry consumes 1.5 Gb of memory..
Appreciate any suggestion how to release the memory :)


Show usage when docker swarm update has no flags

- What I did
This fix tries to address the issue raised in #24352. Previously, when docker swarm update has no flags, the output is
Swarm updated.

even though nothing was updated.
This could be misleading for users.
- How I did it
This fix tries to address the issue by adding a PreRunE function in the command so that in case no flag is provided (cmd.Flags().NFlag() == 0), the usage will be outputed instead.
- How to verify it
An integration has been added to cover the changes.
- Description for the changelog
- A picture of a cute animal (not mandatory but encouraged)

This fix fixes #24352.
Signed-off-by: Yong Tang<EMAIL_ADDRESS>

cc @aaronlehmann

LGTM

Needs a rebase.

Thanks @aaronlehmann. The PR has been rebased.


[bundle list] add --without-group and --only-group

Listing gems according to groups, either excluding a particular or from a specific group.
Usage:
bundle list --without-group dev

bundle list --only-group dev

Addresses #6564

Thank you for looking into adding the feature but I don't like the idea of bringing the same --with and --without options from bundle install. My concern is that these options are for performing an operation on a different unit (ie: this command lists gems, not groups) and those options really don't convey the intent very well without having to lookup the documentation. Maybe instead just have --filter-by-group? i'm not really sure what would be better in this context
My other concern is how complex this PR is now making the list command. Having 2 different ways to filter the output seems over complicated and unnecessary.

Maybe instead just have --filter-by-group? i'm not really sure what would be better in this context

How about --without-group and --only-group maybe?

@colby-swandale r?

@bundler r+

@bundlerbot r+


Capture Android app traffic

Hi, I have a question regarding the suitability of Hoverfly to capture requests from an Android app. I would like to capture the traffic of an Android app not owned by me running on my mobile phone, so I don't know exactly what would be the best setup for this purpose.
I guess I would need to add a proxy on my phone to redirect the traffic of my phone to Hoverfly and from there record all ongoing and outgoing requests.
Another inconvenient is that the traffic is encrypted using SSL with SSL pinning. I don't know how Hoverfly it's able to record and simulate encrypted data. In the case of simple SSL maybe it's not too complicated but with SSL pinning I guess i'ts much more complicated.
What is the better way to do that?
Thanks!

Hi @pjmartorell, you can try configuring the Android system proxy to use a Hoverfly listening on the network where the phone can route to.
Not sure about SSL pinning, does it mean the app only trust a specific certificate from the server or the other way around?

@pjmartorell Are neither the app or server in your control?

Thanks for your help. No, I don't own neither the app nor the server. For simple SSL I guess Hoverfly is able to record the responses using a self-signed certificate. Adding a proxy would be enough? or should I add the self-signed certificate in my device?
In the case of SSL pinning, as far as I know, the server checks that the app certificate matches with the server certificate, avoiding MITM attacks. One option I am trying is to decompile the app and get rid of the SSL pinning, as I have seen the app is not obfuscated. I don't see any other alternatives.

Your device needs to trust the hoverfly self-signed certificate. You can find more details about SSL configuration here: https://hoverfly.readthedocs.io/en/latest/pages/tutorials/advanced/configuressl/configuressl.html


Milestone 10 - Incremental generation of recipes for latest Melodic (2019-12-10)

Depends on https://github.com/ros/meta-ros/pull/655

Merged in:
d85431585..6d4554a24  thud-draft  -> ros/thud-draft


Stop-Deallocate to resize Data Disk

There isn't clarity on whether the VM should be stopped-deallocated when you resize the data disk of a VM, from the portal it appears that if the disk is allocated to a VM that VM should be deallocated before you change the size.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: ccc58c74-7913-f1f6-8681-1823bbd8f050
Version Independent ID: c11bab28-a82a-29a1-84bf-fb3dfb9a5cb2
Content: Expand the OS drive of a Windows VM in an Azure
Content Source: articles/virtual-machines/windows/expand-os-disk.md
Service: virtual-machines-windows
Sub-service: disks
GitHub Login: @kirpasingh
Microsoft Alias: kirpas

Thanks for the feedback! We are currently investigating and will update you shortly.

The best practice is to deallocate the VM before resizing so that no data is lost.
Thanks for the feedback! I have assigned the issue to the content author to investigate further and update the document as appropriate.

@sandeepraichura each of the scripts that apply to the OS disk have commands to stop the VM before resizing. The part that does not include a stop is related to the Data Disks which you don't need to stop the VM.
At this point in time we are planning on leaving the doc as is. However if others find it confusing they can comment and we can always revisit in the future.


Azure Marketplace and AppSource publishing guide

I see the detail in this article regarding Marketplace but nothing regarding the requirements and benefits of Microsoft AppSource. Can you provide more detail?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 7733e992-48c0-0f31-d9c4-8303064d63cc
Version Independent ID: 609d9b6e-03d6-a8d8-3b86-6ca7021dba44
Content: Azure Marketplace and AppSource publishing guide
Content Source: articles/marketplace/marketplace-publishers-guide.md
Service: marketplace
GitHub Login: @ellacroi
Microsoft Alias: ellacroi

@mikeper12 Thanks for the feedback. We are actively investigating and will get back to you soon.

@ellacroi please review the comments for potential documentation enhancements.

You claim it rakes 2 to 3 business days to get back to us when we submit detail of app to be publiahed . But you never get back to us after weeks of filling the app publish request form , this is rather unprofessional

Still nothing:  nothing regarding the requirements and benefits of Microsoft AppSource. Can you provide more detail? - how do you publish on AppSource ?

@jfrandol you can get started on AppSource here: https://appsource.microsoft.com/en-us/partners/signup
The registration and publishing tools are the same for both storefronts.
@qianw211 could you please read through the main articles and make sure that AppSource is called out specifically?

@ellacroi Hi Lizzie, I'm looking through this article here, and the feedbacks are correct that this article talks only about Marketplace benefits, nothing is said about AppSource.  Would you be the SME for information about why people need to publish on AppSource?  I think we need this content added to the opening page for Publishing Guide.  Thanks!

Just to let you know, I did enter my information in that form at :
https://appsource.microsoft.com/en-us/partners/signup
And got a response only for Azure marketplace,  so perhaps they "decide for you " based on your entries, but I thought since my app was on premises, they would decide appsource for me, but I never got any info about it.
Find the right app | Microsoft AppSourcehttps://appsource.microsoft.com/en-us/partners/signup
Tell us about your company. Interested in publishing on Azure Marketplace or AppSource?Please share your information and we will follow up with you within 2 business days. If you have questions, you can review our Publisher Guide or open a support request under Problem Type, Marketplace Onboarding.. If you already are a publisher, you can sign into your account on Cloud Partner Portal to publish.
appsource.microsoft.com
Sent from Outlookhttp://aka.ms/weboutlook

From: Qian Wen<EMAIL_ADDRESS>Sent: Tuesday, December 11, 2018 11:55 AM
To: MicrosoftDocs/azure-docs
Cc: jfrandol; Mention
Subject: Re: [MicrosoftDocs/azure-docs] Azure Marketplace and AppSource publishing guide (#13556)
@ellacroihttps://github.com/ellacroi Hi Lizzie, I'm looking through this article here, and the feedbacks are correct that this article talks only about Marketplace benefits, nothing is said about AppSource. Would you be the SME for information about why people need to publish on AppSource? I think we need this content added to the opening page for Publishing Guide. Thanks!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/MicrosoftDocs/azure-docs/issues/13556#issuecomment-446277160, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALyHmZx36YKalso3qDapoGrL6o_sqlkUks5u3-OdgaJpZM4WAt-D.

@jfrandol This feedback system is for improving our documentation on Microsoft Docs.  The best way for you to get support is to file a support ticket by the following link:
https://support.microsoft.com/en-us/getsupport?wf=0&tenant=classiccommercial&oaspworkflow=start_<IP_ADDRESS>&locale=en-us&supportregion=en-us&pesid=16230&forceorigin=esmc&ccsid=636595105151894820
Hope this helps!

#please-close
Content has been updated, and partner will need to file a support ticket to get their particular question answered.


Add an option to print shell errors that do not make it into the log files

Is your feature request related to a problem? Please describe.
I'm running a test in a github action using snakemake/snakemake-github-action@v1.  The one test that's failing has no error output.  It's stderr log file is empty (as reported in the github log when the --show-failed-logs option is supplied), so I assume that the error is at the shell level.  I have noted, when working with snakemake locally on the command line, if there's no error logged, if I supply --printshellcmds and then try the command manually, there are errors reported by the shell (e.g. "command not found") that do not get into the log.
In my specific case however, the test runs fine locally.  Here's the setup for the test in the yaml:
  temporary-summits-delete-after-59-merged:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Codebase
        uses: actions/checkout@v3
      - name: Test summits steps only
        uses: snakemake/snakemake-github-action@v1
        with:
          directory: '.tests/test_6_summits'
          snakefile: 'workflow/Snakefile'
          args: '--use-conda -c 2 --show-failed-logs -p all_atac_summits'

If I run the command locally, there is no error.  Everything succeeds:
snakemake --directory .tests/test_6_summits --use-conda --cores 2 --printshellcmds all_atac_summits

I suspect the problem has to do with 1 output where I use pipe(), in this rule, but note that it's not a simple command.  I use the {output} twice, because I want to be able to handle the case where there's only 1 input file, and the merge tool will not work in that case:
rule bigwigs_to_summed_bedgraph:
    input:
        expand(
            "results/bigwig/{dataset}.bw",
            dataset=DATASETS,
        ),
    output:
        pipe("results/bigwig/all_summed.bedGraph"),
    params:
        nargs=len(DATASETS),
    log:
        std="results/bigwig/logs/bigwigs_to_summed_bedgraph.stdout",
        err="results/bigwig/logs/bigwigs_to_summed_bedgraph.stderr",
    conda:
        "../envs/bigwig_tools.yml"
    shell:
        # bigwigmerge requires a minimum of 2 files, but the original R script
        # supports 1, so to support 1 file here, we use bigwigtobedgraph
        """
        if [ {params.nargs:q} -eq 1 ]; then \
            bigwigtobedgraph {input:q} {output:q} 1> {log.std:q} 2> {log.err:q}; \
        else \
            bigwigmerge {input:q} {output:q} 1> {log.std:q} 2> {log.err:q}; \
        fi
        """

I assume that there's some issue with the shell commands involving the named pipe when run as a github action, but since I don't have an error to read, I'm kind of stuck.
Describe the solution you'd like
So note, I am not asking for a solution to my current problem.  What I would like is simply a way to make snakemake print any shell-related errors.  As I have described above, there are certain errors that can happen that do not get into the log, because they're from the parent shell process.  Since I cannot do what I usually do on a local command line (using --printshellcmds) when the error is exclusively encountered in a github action, the only way to view these errors would be a new feature to allow snakemake to print them.
Describe alternatives you've considered
I'm not terribly familiar with all the niche options that snakemake provides, but I have keyword searched the usage/help output and cannot find anything that can report these errors.  I cannot more the entire snakemake log.  Maybe I could rule out the pipe hunch I have by changing it to temp(), but that doesn't solve the problem - it only works around it.
The one thing I thought maybe I could do is insert some cats & echos & ls's in the command so I can see if the named pipe exists.  Or maybe I could put the code in a separate shell script to get some debug output... But all of those options would be work-arounds.
Additional context
Note that my first attempt to run with -c 1 resulted in a resource error.  I learned from that output that pipe requires at least 2 cores:
WorkflowError:
Error grouping resources in group 'db63fca3-7b3a-43f1-a76a-d92491e9e345': Not enough resources were provided. This error is typically caused by a Pipe group requiring too many resources. Note that resources are summed across every member of the pipe group, except for ['runtime'], which is calculated via max(). Excess Resources:
	_cores: 2/1

The error I see in the github action log is:
Error in group 8c80e530-b72c-4cd2-ba80-00b10c34aa9e:
    jobs:
        rule sort_summed_bedgraph:
            jobid: 5
            output: results/bigwig/all_sorted.bedGraph
            log: results/bigwig/logs/sort_summed_bedgraph.stderr (check log file(s) for error details)
        rule bigwigs_to_summed_bedgraph:
            jobid: 6
            output: results/bigwig/all_summed.bedGraph (pipe)
            log: results/bigwig/logs/bigwigs_to_summed_bedgraph.stdout, results/bigwig/logs/bigwigs_to_summed_bedgraph.stderr (check log file(s) for error details)
Logfile results/bigwig/logs/sort_summed_bedgraph.stderr: empty file

Removing output files of failed job sort_summed_bedgraph since they might be corrupted:
results/bigwig/all_sorted.bedGraph
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

So it turns out that there are multiple things going on here...  The factors involved are:

macOS file system's case insensitivity
snakemake's --show-failed-logs only prints 1 of multiple logs
In a group created by pipe(), the wrong log is printed (the error was from the rule that created the pipe, but the, but the log that was printed was from the one that consumed the pipe.

The problem in my specific case was that the commands are actually bedGraphToBigWig and bigWigMerge, but I had bigwigtobedgraph and bigwigmerge.  Since macOS is case insensitive, it worked on my machine.  In the github action, I had no clue what the issue was becasue it doesn't show me the relevant log and I could not reproduce the error locally.
IMHO, --show-failed-logs should show all logs from the rule with the error, not just a single arbitrary log from the rule without the error.


Doesn't work on macOS Big Sur (11.0)

The app launches but never connects to discord or just disappears from the Menu Bar.

Same issue here. Doesn't work on the public release version of Big Sur (11.0.1) either unfortunately.

@joel-tonuri The issue appears to be related to AppleScript. Apple must have broken it or changed how it functions breaking most apps that fetch music using AppleScript. I’m looking into another solution that would work but would require a rewrite in Swift. I’ll keep you updated on any progress here.

@joel-tonuri After a few hours of work, I figured out a new solution. It's not perfect but it is a lot better then not having AppleScript work. You can download the beta version at https://github.com/A-da-m/AppleMusicPresence/releases/tag/1.0.0-dev. This version is macOS 11 only, previous versions can use the 0.0.7 for AMP.

@joel-tonuri After a few hours of work, I figured out a new solution. It's not perfect but it is a lot better then not having AppleScript work. You can download the beta version at https://github.com/A-da-m/AppleMusicPresence/releases/tag/1.0.0-dev. This version is macOS 11 only, previous versions can use the 0.0.7 for AMP.

Thanks heaps! I've been using it for the past few days -- really solid -- I appreciate you taking the time to update this project. I'll let you know if I run into any unexpected issues.