Move custom validations to a library validator

We are using validate :check_timezone in at least two different classes. We should put our custom validators in a library validation class and then use validates. See
http://guides.rubyonrails.org/active_record_validations_callbacks.html#performing-custom-validations

I will work on this one.

I have a problem on moving the validations.
According to the link, I believe we should take the 2nd way to add custom validator (with ActiveModel::EachValidator and then use 'validates' in models).
However, there has to be a naming match between the attribute to be validated and the validator name. Taking the example in the link, for the code
validates :email, :presence => true, :email => true,
the corresponding validator class must be named as 'EmailValidator' in order to be invoked.
Our case is, although "check_timezone" is used twice, it validates different attributes in two different models. In assignment.rb, it validates the attribute "due_date", In grade_entry_form.rb, it validates the attribute "date".
If we want to validate different attributes using the same validator, how should we name the validator? How can we invoke the same validator for different model attributes?
(I did a lot of search but haven't found out)


Fix SectionList example

SectionList Example 1 is displayed incorrectly.

Oops, good catch!
Looks like the tabulations have been overridden, I've changed the tab code blocks to proper markdown and also apply this to versioned docs of 0.55.


Move /metrics from the query port to internal (health-check)

Problem - what in Jaeger blocks you from solving the requirement?
Jaeger currently exposes the metrics via the main query endpoint, which might be exposed to the public internet. Ideally, the /metrics endpoint would be directed to the internal, admin-only port that is used by the health-check handler.
$ docker run -p 16686:16686 --rm jaegertracing/all-in-one:1.9
$ curl localhost:16686/metrics

+1, although support for the endpoint on the existing port would need to be deprecated so not to break existing deployments.

support for the endpoint on the existing port would need to be deprecated so not to break existing deployments

This is quite messy to do, I prefer we require a clean cut over to the new port (#1442).


Fix: Remove np.asarray() from keras numeric check

Description
Switches the is_numeric to check the tf.dtypes.DType builtin functions, or dtype starts with int or float if a string instead of converting to numpy.
DType doc:
https://www.tensorflow.org/api_docs/python/tf/dtypes/DType
Testing
Using functional tests and existing repo tests
Release Notes
Below, please enter user-facing release notes as one or more bullet points.
If your change is not user-visible, write NO RELEASE NOTES instead, with no bullet points.
------------- BEGIN RELEASE NOTES ------------------
NO RELEASE NOTES
------------- END RELEASE NOTES --------------------

Do you think there is a need to add tests for coverage?


[Pre-SVE] Test TC-SC-5.2 Update of onoff status and nodelabel does not work.

SHA: 1e3f12041013dd71a53d035dbf2f6d142fc06c4e
platform: nRF52840
all clusters app
I cant update onoff status and nodelabel using group communication.
First I did test 5.1 with commands:
./chip-tool accesscontrol write acl '[{"fabricIndex": 1, "privilege": 5, "authMode": 2, "subjects": [112233], "targets": [{"cluster": null, "endpoint": 0, "deviceType": null}]}, {"fabricIndex": 1, "privilege": 3, "authMode": 3, "subjects": null, "targets": [{"cluster": null, "endpoint": 1, "deviceType": null}]}]' 1 0
./chip-tool groupkeymanagement key-set-write '{"groupKeySetID": 42,
"groupKeySecurityPolicy": 0, "epochKey0":
"d0d1d2d3d4d5d6d7d8d9dadbdcdddedf", "epochStartTime0": 2220000,"epochKey1":
"d1d1d2d3d4d5d6d7d8d9dadbdcdddedf", "epochStartTime1": 2220001,"epochKey2":
"d2d1d2d3d4d5d6d7d8d9dadbdcdddedf", "epochStartTime2": 2220002 }' 1 0
./chip-tool groups add-group 0x0001 grp1 1 0
./chip-tool groupkeymanagement write group-key-map '[{"groupId": 1, "groupKeySetID": 42, "fabricIndex": 1}]' 1 0
./chip-tool groups view-group 0x0001 1 0
./chip-tool groupkeymanagement key-set-read 42 1 0
./chip-tool groupkeymanagement read group-key-map 1 0
./chip-tool groupkeymanagement read group-table 1 0
./chip-tool groupsettings add-group grp1 0x0001
./chip-tool groupsettings add-keysets 0x0042 0 0x000000000021dfe0 hex:d0d1d2d3d4d5d6d7d8d9dadbdcdddedf
./chip-tool groupsettings bind-keyset 0x0001 0x0042
./chip-tool groupsettings show-groups
And when I try to run test 5.2
after command
./chip-tool basic write node-label newnode 0xffffffffffff0001 0
command:
./chip-tool basic read node-label 1 0
returns empty label
also in following steps command
./chip-tool onoff read on-off 1 1
returns false, and its not changed by
./chip-tool onoff on 0xffffffffffff0001 1
same case (at least with label) occurs when I run
./chip-tool tests TestGroupMessaging --nodeId 1
instead
Side note: ACLs in first step are bad and don't give you permissions establish case session on endpoint 1.
All these commands are from excel with verification steps.

@HubertHugo See this comment for steps that work for groups.
https://github.com/project-chip/connectedhomeip/issues/19608#issuecomment-1156931777
and replace the acl command by thise one
https://github.com/project-chip/connectedhomeip/issues/19801#issuecomment-1163223895

./chip-tool accesscontrol write acl '[{"fabricIndex": 1, "privilege": 5, "authMode": 2, "subjects": [112233], "targets": [{"cluster": null, "endpoint": 0, "deviceType": null}]}, {"fabricIndex": 1, "privilege": 3, "authMode": 3, "subjects": null, "targets": [{"cluster": null, "endpoint": 1, "deviceType": null}]}]' 1 0

Allows any group access to endpoint 1.

./chip-tool groups add-group 0x0001 grp1 1 0

Adds endpoint 0 to group 0x0001.

./chip-tool basic write node-label newnode 0xffffffffffff0001 0

Will fail ACL checks because Basic Information is on endpoint 0 (which is in the group), but we only have ACLs for endpoint 1.

./chip-tool onoff on 0xffffffffffff0001 1

Will do nothing because only endpoint 0 is in the group and there is no On/Off cluster on that endpoint.

All these commands are from excel with verification steps.

Looks like those steps need fixing.  I filed https://github.com/CHIP-Specifications/chip-test-plans/issues/1561

@bzbarsky-apple  This is a duplicate of
#19801  and #19608

Yep, but finally someone explicitly said that it's the verification spreadsheet that's confused.... :)


feat: add --select flag data

Description
stg_dbt__run_results show models args as selected_models column, but since moving to dbt1, the models flag has been deprecated and the select flag is being used instead. It would be great to have the information from the select flag available in the model.
Issue Link
https://github.com/brooklyn-data/dbt_artifacts/issues/91

Thank you @LukasS29 🙌


which PX4  firmware should I use?

Hello, I have made a six-rotor with a wheelbase of 550mm, which can adjust the tilt Angle around the arm. May I ask if your pX4 firmware can be used for control?Is "px4_fmu_v2_multicopter"?The other question is how many degrees can you achieve with a normal tilt Angle?

Hello, I have made a six-rotor with a wheelbase of 550mm, which can adjust the tilt Angle around the arm. May I ask if your pX4 firmware can be used for control?Is "px4_fmu_v2_multicopter"?The other question is how many degrees can you achieve with a normal tilt Angle?

Yes, you should be able to control your multirotor using this firmware. You can't use the pre-compiled binaries and should compile the firmware for your pixhawk/autopilot. I will post a detailed guide tomorrow.

Hello, I have made a six-rotor with a wheelbase of 550mm, which can adjust the tilt Angle around the arm. May I ask if your pX4 firmware can be used for control?Is "px4_fmu_v2_multicopter"?The other question is how many degrees can you achieve with a normal tilt Angle?

The readme file is now updated with a guide on how to use the firmware. Please feel free to reach us if you have any questions.

  I'm really sorry for that.Today I had a few problems during the compilation of your firmware, I clone your code in the ubuntu18.04 "git clone https://github.com/castacks/PX4-fully-actuated.git -- recursive", but I "make px4_fmu_v3_default" or use other compiler command, an error is as follows:

At 2020-11-22 05:38:05, "Azarakhsh Keipour"<EMAIL_ADDRESS>wrote:
Hello, I have made a six-rotor with a wheelbase of 550mm, which can adjust the tilt Angle around the arm. May I ask if your pX4 firmware can be used for control?Is "px4_fmu_v2_multicopter"?The other question is how many degrees can you achieve with a normal tilt Angle?
The readme file is now updated with a guide on how to use the firmware. Please feel free to reach us if you have any questions.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

I cannot see the error message. Would you please post the error message again?

Hello,when I "make px4_fmu-v3_default",the error is:
"FAILED: src/lib/version/build_git_version.h
cd /home/cg/wort_src/PX4-fully-actuated && /usr/bin/python3 /home/cg/wort_src/PX4-fully-actuated/src/lib/version/px_update_git_header.py /home/cg/wort_src/PX4-fully-actuated/build/px4_fmu-v3_default/src/lib/version/build_git_version.h --validate
Error: the git tag 'f33b118dfb' does not match the expected format.
The expected format is 'v[-]'
: v..[-rc|-beta]
: ..
Examples:
v1.9.0rc3 (deprecated)
v1.9.0-rc3 (preferred)
v1.9.0-beta1
v1.9.0-1.0.0
See also https://dev.px4.io/master/en/setup/building_px4.html#firmware_version
[14/1274] Performing configure step for 'px4io_firmware'
-- PX4 version: f33b118dfb
-- PX4 config file: /home/cg/wort_src/PX4-fully-actuated/boards/px4/io-v2/default.cmake
-- PX4 config: px4_io-v2_default
-- PX4 platform: nuttx
-- cmake build type: MinSizeRel
-- The CXX compiler identification is GNU 7.2.1
-- The C compiler identification is GNU 7.2.1
-- The ASM compiler identification is GNU
-- Found assembler: /home/cg/tools/gcc-arm-none-eabi-7-2017-q4-major/bin/arm-none-eabi-gcc
-- Check for working CXX compiler: /home/cg/tools/gcc-arm-none-eabi-7-2017-q4-major/bin/arm-none-eabi-g++
-- Check for working CXX compiler: /home/cg/tools/gcc-arm-none-eabi-7-2017-q4-major/bin/arm-none-eabi-g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working C compiler: /home/cg/tools/gcc-arm-none-eabi-7-2017-q4-major/bin/arm-none-eabi-gcc
-- Check for working C compiler: /home/cg/tools/gcc-arm-none-eabi-7-2017-q4-major/bin/arm-none-eabi-gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- ccache enabled (export CCACHE_DISABLE=1 to disable)
-- Found PythonInterp: /usr/bin/python3 (found suitable version "3.6.9", minimum required is "3")
-- build type is MinSizeRel
-- PX4 ECL: Very lightweight Estimation & Control Library v1.9.0-rc1-182-g29cf788
-- Configuring done
-- Generating done
-- Build files have been written to: /home/cg/wort_src/PX4-fully-actuated/build/px4_fmu-v3_default/external/Build/px4io_firmware
ninja: build stopped: subcommand failed.

At 2020-11-23 23:42:58, "Azarakhsh Keipour"<EMAIL_ADDRESS>wrote:
I cannot see the error message. Would you please post the error message again?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

     Hi, I am very interested in your recent paper"Attitude and Thrust Strategies for Fully Actuated Multirotors:The Fast-Track to Real-World Applications"。I have some questions I hope to get your answers
    how do you view The last two experiments and their data in your paper ?In The paper have such a words: "The figure illustrates The Angle staying almost constant while The multimotor performs aggressive motions", my question is tilt Angle is not what we fixed in advance?How to understand "The tilt Angle Staying almost Constant "?The final question is what does the data in Figure 10 illustrate compared to the data in Figure 9?

I sincerely hope to get your reply and answer. Thank you very much！
At 2020-11-23 23:42:58, "Azarakhsh Keipour"<EMAIL_ADDRESS>wrote:
I cannot see the error message. Would you please post the error message again?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Hello,when I "make px4_fmu-v3_default",the error is: "FAILED: src/lib/version/build_git_version.h cd /home/cg/wort_src/PX4-fully-actuated && /usr/bin/python3 /home/cg/wort_src/PX4-fully-actuated/src/lib/version/px_update_git_header.py /home/cg/wort_src/PX4-fully-actuated/build/px4_fmu-v3_default/src/lib/version/build_git_version.h --validate Error: the git tag 'f33b118dfb' does not match the expected format. The expected format is 'v[-]' : v..[-rc|-beta] : .. Examples: v1.9.0rc3 (deprecated) v1.9.0-rc3 (preferred) v1.9.0-beta1 v1.9.0-1.0.0 See also https://dev.px4.io/master/en/setup/building_px4.html#firmware_version [14/1274] Performing configure step for 'px4io_firmware'
…
-- PX4 version: f33b118
-- PX4 config file: /home/cg/wort_src/PX4-fully-actuated/boards/px4/io-v2/default.cmake
-- PX4 config: px4_io-v2_default
-- PX4 platform: nuttx
-- cmake build type: MinSizeRel
-- The CXX compiler identification is GNU 7.2.1
-- The C compiler identification is GNU 7.2.1
-- The ASM compiler identification is GNU
-- Found assembler: /home/cg/tools/gcc-arm-none-eabi-7-2017-q4-major/bin/arm-none-eabi-gcc
-- Check for working CXX compiler: /home/cg/tools/gcc-arm-none-eabi-7-2017-q4-major/bin/arm-none-eabi-g++
-- Check for working CXX compiler: /home/cg/tools/gcc-arm-none-eabi-7-2017-q4-major/bin/arm-none-eabi-g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working C compiler: /home/cg/tools/gcc-arm-none-eabi-7-2017-q4-major/bin/arm-none-eabi-gcc
-- Check for working C compiler: /home/cg/tools/gcc-arm-none-eabi-7-2017-q4-major/bin/arm-none-eabi-gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- ccache enabled (export CCACHE_DISABLE=1 to disable)
-- Found PythonInterp: /usr/bin/python3 (found suitable version "3.6.9", minimum required is "3")
-- build type is MinSizeRel
-- PX4 ECL: Very lightweight Estimation & Control Library v1.9.0-rc1-182-g29cf788
-- Configuring done
-- Generating done
-- Build files have been written to: /home/cg/wort_src/PX4-fully-actuated/build/px4_fmu-v3_default/external/Build/px4io_firmware ninja: build stopped: subcommand failed.  At 2020-11-23 23:42:58, "Azarakhsh Keipour"<EMAIL_ADDRESS>wrote: I cannot see the error message. Would you please post the error message again? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

Sorry for the delay in responding. The issue was due to the tag not being uploaded to the repo. Please git pull --tags to update the tags on your computer and try building the code again. Just note that since you are compiling for fmu-v3, you probably need to comment some of the geometries in the CMakeFiles before your compilation to free some flash space. Please refer to README to see how you can do that.

Hi, I am very interested in your recent paper"Attitude and Thrust Strategies for Fully Actuated Multirotors:The Fast-Track to Real-World Applications"。I have some questions I hope to get your answers how do you view The last two experiments and their data in your paper ?In The paper have such a words: "The figure illustrates The Angle staying almost constant while The multimotor performs aggressive motions", my question is tilt Angle is not what we fixed in advance?How to understand "The tilt Angle Staying almost Constant "?

You set the tilt angle in advance. In the specific case of Figure 10, the tilt angle and direction were estimated from the wind, but you can set it using the PX4 parameters instead (see README for more information about the parameters). The figure shows that while the UAV is flying around (not just hovering), the tilt angle can stay the same without much error.
Note that we are flying outdoors and our test site is pretty windy and has significant gusts, so keeping the angle error too low is hard. Nevertheless, we are working on further improving it and will update the repo regularly with improvements.

The final question is what does the data in Figure 10 illustrate compared to the data in Figure 9? I sincerely hope to get your reply and answer. Thank you very much！

Figures 9 and 10 are two different attitude strategies. Figure 9 shows a flight with the zero-tilt attitude strategy, while Figure 10 shows a flight with the fixed-tilt attitude strategy.


How to push the tags created on the vscode menu to the remote?

Does this issue occur when all extensions are disabled?: Yes/No

VS Code Version: 1.63.0
OS Version: windows_nt x64 10.0.17763

Steps to Reproduce:
1.I  created a tag by vscode menu

2.Ctrl + Shift + P and then push tags. Is there a shortcut click on the menu?
QS: What the complete steps from creation to submission to the remote?

There are global commands available:

Feel free to submit a PR, adding them to that menu.


Restore backward after each batch for grad accum

The fix to normalize CE loss by total number of tokens in a step moved our backward call from per-batch to per-step.  We should be able to call backward on the unnormalized values for each batch then manually scale the gradients just before optimizer step.
This is easy enough for our single-device recipes, but for our distributed recipes it's slightly more work. In fact it cannot be done in a way that is both (a) correct and (b) backwards compatible. If you want to know why this is the case, see the "Long digression" section below.
TLDR of the changes:

Gradient accumulation logic changes: We now (a) call backward on every batch's unnormalized loss, (b) accumulate num_tokens over every batch, (c) (distributed only) all_reduce num_tokens on the last batch prior to stepping, then (d) manually scale gradients prior to stepping with the optimizer. This will revert the memory regression introduced by #1875 but keep the correctness of the gradient accumulation logic.
Logging changes: If we are stepping based on total_loss / total_tokens (where total is over all batches and over all ranks), that's what should show up in the logs. Similarly, we can now use the number of tokens over all ranks instead of just rank 0 (though we still normalize tokens/sec to per GPU)
Optimizer in backward: This shouldn't be supported when gradient accumulation is enabled. Previously we didn't raise an explicit error about this (also it actually worked when we only call .backward() once per step, but we don't wanna do that). So now we'll raise an error (also I disabled a couple test cases that snuck in during this time period)
Minor changes to KD and QAT recipes: Integrate utils.batch_to_device into QAT recipe. This was causing num_tokens to show up on CPU instead of GPU because labels weren't moved until later.

Test plan
All the single-device recipe tests succeed without any changes. However, for our distributed recipe tests some of the expected values need to change.
Why are you changing the expected values?!
Again, see the "long digression" section below, but we have been normalizing our loss by the local number of tokens, not the number of tokens seen over all ranks. Is this a huge deal? Honestly probably not, but technically it's not correct. How do I know this version is correct? I've run the following command both on main and on this PR:
pytest -m integration_test tests/recipes/test_full_finetune_distributed.py -k 'test_loss[False-llama2/7B_full-llama2-hf-1-4]'

In both cases I added logging of # of tokens and grad for the token embeddings weight. On main I also commented out the loss normalization so that we can get the raw values. Diff for my changes on main (changes on this PR are just the identical logging).
Note that after the first iteration the results differ due to the difference in how the gradients are calculated, so looking at the logs from just the first iteration:
On main
rank: 1, idx: 0, num_tokens: 12
rank: 0, idx: 0, num_tokens: 59
rank: 0, idx: 1, num_tokens: 60
rank: 1, idx: 1, num_tokens: 268
rank: 0, idx: 2, num_tokens: 14
rank: 1, idx: 2, num_tokens: 133
rank: 0, idx: 3, num_tokens: 11
rank: 1, idx: 3, num_tokens: 6
rank: 0, unnormalized grad: DTensor(local_tensor=-13.8838529586792, device_mesh=DeviceMesh('cuda', [0, 1]), placements=(Partial(sum),))
rank: 1, unnormalized grad: DTensor(local_tensor=1.0207021236419678, device_mesh=DeviceMesh('cuda', [0, 1]), placements=(Partial(sum),))

On this PR
rank: 1, idx: 0, num_tokens: 12
rank: 0, idx: 0, num_tokens: 59
rank: 0, idx: 1, num_tokens: 60
rank: 1, idx: 1, num_tokens: 268
rank: 0, idx: 2, num_tokens: 14
rank: 1, idx: 2, num_tokens: 133
rank: 0, idx: 3, num_tokens: 11
rank: 1, idx: 3, num_tokens: 6
rank: 0, grad: DTensor(local_tensor=-0.02466048300266266, device_mesh=DeviceMesh('cuda', [0, 1]), placements=(Partial(sum),))
rank: 1, grad: DTensor(local_tensor=0.0018129688687622547 device_mesh=DeviceMesh('cuda', [0, 1]), placements=(Partial(sum),))

But what does it all mean?
In both the preceding snippets, the first step sees 12 + 59 + 60 + 268 + 14 + 133 + 11 + 6 = 563 tokens.
The grad value logged is just the sum of the elements in the tensor. On main this is -13.8838529586792 on rank 0 and 1.0207021236419678 on rank 1. On this PR it is -0.02466048300266266 on rank 0 and 0.0018129688687622547 on rank 1. In both cases, the value on this PR == the unnormalized value on main / 563, which is what we would expect
End-to-end testing
Llama3 8B full finetune on 4 devices
Repro:
tune run --nproc_per_node 4 full_finetune_distributed --config llama3/8B_full \
max_steps_per_epoch=500 gradient_accumulation_steps=4

Peak allocated memory drops substantially after this PR (back to where it was before the first gradient accumulation PR, but with actual correct loss calculation now).

Llama 3.2 1B full finetune on 2 devices
tune run --nproc_per_node full_finetune_distributed --config llama3_2/1B_full max_steps_per_epoch=500

We can see that (a) peak allocated memory drops back down to where it was before #1875, (b) tokens per second is the same as #1875 (and faster than before), and (c) the loss curves look similar.

Long digression: data parallel and gradient accumulation aren't so different
First: on a single device
Let's consider a simple example of how to calculate the correctly-normalized CE loss on a single device (i.e. no data parallel) with two gradient accumulation steps. Say our unnormalized cross-entropy loss for the first batch is L1 and our (similarly unnormalized) loss for the second batch is L2, and we have n1 tokens in the first batch and n2 tokens in the second batch. Then the properly-normalized cross-entropy loss to step with will be (L1 + L2) / (n1 + n2). The naive approach taken in #1875 is to just accumulate a running sum of both loss and number of tokens, then call backward on the ratio running_loss / running_num_tokens just before optimizer step. The problem with this is that we only call backward once, so our activations stick around and blow up our memory. What can we do instead?
Sticking with the single device case, it's not too hard to fix this. Repeated calls to .backward() accumulate gradients, so e.g. the following two are numerically equivalent:
# Single backward call
loss_1 = model(batch_1)
loss_2 = model(batch_2)
summed_loss = loss_1 + loss_2
summed_loss.backward()

# Multiple backward calls
loss_1 = model(batch_1)
loss_1.backward()
loss_2 = model(batch_2)
loss_2.backward()

Then for the single device case we can actually take the second approach with L1 and L2. But remember that we still want to normalize by n1 + n2. This isn't too hard.. we can just manually scale the gradients ourselves just before the optimizer step (since grad(c*X) = c*grad(X) for any constant c this is equivalent to scaling the loss).
Adding data parallel
This is where things get slightly messier. Let's extend the example to two devices: using Lij to refer to the unnormalized cross-entropy loss for rank i in its jth batch, and similarly for nij with number of tokens.
So in the first batch the model will see:
On rank 1: loss L11 based on n11 tokens
On rank 2: loss L21 based on  n21 tokens
In the second batch, it will see:
On rank 1: loss L12 based on n12 tokens
On rank 2: loss L22 based on  n22 tokens
Similarly to the single-device case, the total number of tokens seen across all batches and all ranks will be n11 + n21 + n12 + n22. This means that the properly-normalized loss should be given by (L11 + L21 + L12 + L22) / (n11 + n21 + n12 + n22).
What are we doing today?
Currently we take a similar approach to the single-device case described previously: we accumulate the losses and tokens, then call .backward() on the ratio running_loss / running_num_tokens just before stepping with the optimizer. See the below code: https://github.com/pytorch/torchtune/blob/a1bcb971cf2f506c10e07433303e1eb90613efbe/recipes/full_finetune_distributed.py#L724-L732
What's wrong with this?
During data parallel training, the loss on a given rank is based only on the subset of data seen by that rank. Similarly, our calculation of running_num_tokens is based only on the tokens from that rank. This means that when we normalize we are normalizing only over iterations, not over ranks. Put another way, the line loss = running_loss / num_tokens in the above snippet will yield (L11 + L12) / (n11 + n12) on rank 1 and (L21 + L22) / (n21 + n22) on rank 2. Finally, we call .backward(), which calculates local grads before firing a hook to sync by reducing over all ranks. The upshot is that our loss winds up as [(L11 + L12) / (n11 + n12)] + [(L21 + L22) / (n21 + n22)], which is definitely not (L11 + L21 + L12 + L22) / (n11 + n21 + n12 + n22).
How can we fix it?
Aside from the correctness issue described in the previous section, we also need an approach that still calls .backward() on each batch to free the activation memory. How can we do this? Actually it's not so bad.. we just need to reduce the number of tokens before our final gradient normalization and let data parallel backward hooks take care of the rest. More explicitly:
First batch:
On rank 1: calculate loss L11 based on n11 tokens
On rank 2: calculate loss L21 based on n21 tokens
Call backward -> this triggers a grad sync and each rank now has the (local) grads for L11 + L21
Second batch:
On rank 1: calculate loss L12 based on n12 tokens
On rank 2: calculate loss L22 based on n22 tokens
Call backward -> this accumulates the grads from L11 + L21 locally, then triggers another sync so that we now have L11 + L21 + L12 + L22
Then we just need n11 + n21 + n12 + n22. Fortunately this is just an all-reduce on running_num_tokens. Then we manually scale the gradients just like in the single-device case.
In summary, we wind up with the following process:
For each batch:

Call backward on the unnormalized losses
Keep a running tally of the number of tokens seen (per rank)

Before optimizer step:

[data parallel only] Reduce the running tally of number of tokens over all ranks
Scale gradients by 1 / total_num_tokens

cc @andrewor14 for review of the QAT recipe changes

also cc @lindawangg for the KD recipe changes

lgtm

The only thing that is not crystal clear to me is why TPS would increase. I guess that before we would also count padded tokens, which should make TPS looks higher in the older version.

The only thing that is not crystal clear to me is why TPS would increase. I guess that before we would also count padded tokens, which should make TPS looks higher in the older version.

Yeahhhhhhh, seems like the reduce would actually slow down training no?

@andrewor14 if it's all the same to you I may keep the QAT changes in. Mainly because this change will actually impact the memory usage and loss curves in a nontrivial way, so I'd rather not have any divergence between our recipes there, even for a short time. Lmk if that sounds alright to you though

@felipemello1 @joecummings discussed offline but the WPS is not measured in exactly the same way before vs after these changes. In the figures we have "old-version", "main", and "this-pr". In "old-version" I hacked it so that we are counting non-padding tokens only (this way it lines up with the other two). But both "old-version" and "main" are using only the tokens seen on rank 0. On "this-pr" we now have the tokens seen by all ranks, which we then normalize by the number of ranks. So this PR's version should be less noisy, but there will be some slight differences when compared to the other two versions.
Regarding the question of all_reduce slowing down training: I can do a more detailed comparison, but we are only reducing a single float value. When compared against various FSDP comms on large tensors happening in forward and backward this should be pretty minimal (but will confirm).


Using selfsim_fc layer for label propagation

@ajabri By chance, have you tried using the layer from selfsim_fc head for label propagation?  But what about selfsim_fc? It is located even closer to the loss function, does it perform even worse than res4?
Thanks!

I haven't tried this, but note one detail; selfsim_fc parameters are learned at training time on top of pooled patch embeddings (since we train with patches). So it makes less sense to apply that layer on elements of the dense feature map at test time.

I haven't tried this, but note one detail; selfsim_fc parameters are learned at training time on top of pooled patch embeddings (since we train with patches). So it makes less sense to apply that layer on elements of the dense feature map at test time.

Should it make more sense to first apply AvgPool(8, 8) and then appl selfsim_fc?

Yes, I think this would make more sense than applying the selfsim_fc on the full feature map, but I haven't had a chance to test this.


Bratseth/persist metrics

I confirm that this contribution is made under the terms of the license found in the root directory of this repository's source tree and that I have the authority necessary to make this contribution on behalf of its copyright owner.

I'll rebase


Speed up btp-manager-build job

Description
Now it takes 3min 20sec. Maybe we can apply some caching strategy.
AC

[ ] 1) check which step takes so long and try to make it faster

Timebox: Size/S


After sending POST request, it gives GET https://localhost:9000/mailServer 404 (Not found)

This is my code.
const express = require('express');
const app = express();
const bodyParser = require('body-parser')

const port = 9000;

app.use(bodyParser.urlencoded({
    extended: true
}));
app.use(bodyParser.json());

app.post('mailServer', (req, res) =>{
res.send(req.body)
res.end();
})

app.listen(port, ()=>console.log(`Server listen on ${port}`)

@MirzadVa could you please also add the http request you're making? We'll need a reproducible example. When you say after sending post request, are you saying that your post is successful?
This works for me (notice you need the slash at the beginning of /mailServer:
const express = require('express');
const app = express();
const bodyParser = require('body-parser')

const port = 9000;

app.use(bodyParser.urlencoded({
    extended: true
}));

app.use(bodyParser.json());

app.post('/mailServer', (req, res) =>{
  res.send(req.body)
  res.end();
})

app.listen(port, ()=>console.log(`Server listen on ${port}`))

@ryhinchey I'm prety much new at all of this. I'm trying to send post with postman. But even before I send it, it gives me this error. As soon as I type node server.js, in console I get 404

are you trying to go to https://localhost:9000/mailServer 404 in a browser?  When I run this code, I don't get the error you receive.  Can you create a reproducible repo I can look at that will demonstrate this issue happening?
I'm going to go ahead and close this issue as it's not something wrong with express. Feel free to keep commenting on the closed issue and we can hopefully be of help.  Also - Stack Overflow and the express gitter are two great resources on your express journey :)

@ryhinchey Yes, that's right, that's the problem and I don't see why it's happening.
This is the repo. It's nothing, I'm just starting with express, but this is my concern :D
https://github.com/MirzadVa/express-post

to be clear, are you trying to access localhost:9000/insert in the browser and it's not working or are you trying to make a POST request to localhost:9000/insert?

Actually, I'm trying to access it. I'm guessing if I can't access, I can't make post request either?

post requests are different from get requests. when you go to a webpage in the browser, the browser makes a get request for the site. you don't have a get method set up in your app.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods
Add something like this to get your app to respond to get requests:
app.get('/mailServer', (req, res) =>{
  res.send('ok')
})

If you're looking to learn more about express and web development with node, this is a solid resource:
https://developer.mozilla.org/en-US/docs/Learn/Server-side/Express_Nodejs


App icon should include an request indicator

Task: App icon should include an indicator showing we have incoming friend request(s). This should add to existing global notification count.
Example:

Hogan tackled this in the Notifications utils which are ported over now, so this ticket just needs to implement that and intelligently display notifications in a "combined" view. We may want to use the UI state.

Added in Matts PR, going to create any issues with it into a ticket since the scope of this was just to implement it in


More performance from smaller  ?

@mathjeff commented on Tue Aug 28 2018
I made the interesting observation on my computer (having 56 cores) that a clean build of AndroidX using --max-workers=16 runs much faster than a clean build without any specification of --max-workers.
I've also discovered that according to pstree -p ${gradlePid} | wc -l, Gradle has 8000-10000 child processes in existence at a time.
I've also found that even with only "--max-workers=16", according to htop, Gradle generally uses more than 99% of all of the CPUs
My guess is that what's going on is that Gradle (specifically, probably one or more Gradle plugins) is spawning many more processes than is needed, and that that's resulting in excess context switching.
I'm not sure whether this is something that Gradle can reasonably be expected to dynamically detect and compensate for, but if it is then that would be handy.
I'm still hoping to dig deeper into the details eventually to figure out more specifically why so many processes are spawning.
I enabled build scans for two representative builds, which can be found at:
Build scan from max-workers 16         : https://scans.gradle.com/s/fknlr375evb4o
Build scan from max-workers unspecified: https://scans.gradle.com/s/mmkeof2zrjxcy
To reproduce this behavior:

download the androidx-master-dev branch of AndroidX via repo as described in https://android.googlesource.com/platform/frameworks/support/+/refs/heads/androidx-master-dev/README.md
run cd frameworks/support && rm -rf ../../out && GRADLE_USER_HOME../../out ./gradlew buildOnServer

Some additional build durations for other values of --max-workers :
--max-workers=8 buildOnServer
9.37user 11.74system 12:42.06elapsed 2%CPU (0avgtext+0avgdata 383932maxresident)k
32inputs+319152outputs (0major+98306minor)pagefaults 0swaps
--max-workers=16 buildOnServer
9.02user 10.86system 11:22.92elapsed 2%CPU (0avgtext+0avgdata 409428maxresident)k
0inputs+319088outputs (0major+103593minor)pagefaults 0swaps
--max-workers=24 buildOnServer
9.28user 11.56system 12:06.81elapsed 2%CPU (0avgtext+0avgdata 499748maxresident)k
0inputs+319192outputs (0major+128482minor)pagefaults 0swaps
--max-workers=32 buildOnServer
11.05user 12.31system 13:57.19elapsed 2%CPU (0avgtext+0avgdata 579620maxresident)k
0inputs+319256outputs (0major+152748minor)pagefaults 0swaps
--max-workers=40 buildOnServer
12.42user 12.18system 15:00.71elapsed 2%CPU (0avgtext+0avgdata 576896maxresident)k
0inputs+319280outputs (0major+150236minor)pagefaults 0swaps
--max-workers=48 buildOnServer
11.77user 11.93system 17:40.43elapsed 2%CPU (0avgtext+0avgdata 616720maxresident)k
0inputs+319408outputs (0major+157018minor)pagefaults 0swaps
--max-workers=56 buildOnServer
12.91user 12.34system 23:53.76elapsed 1%CPU (0avgtext+0avgdata 631672maxresident)k
0inputs+319552outputs (0major+160298minor)pagefaults 0swaps
--max-workers=64 buildOnServer
14.16user 13.41system 26:04.43elapsed 1%CPU (0avgtext+0avgdata 653612maxresident)k
0inputs+319776outputs (0major+165561minor)pagefaults 0swaps
Anyway, I'm not necessarily expecting a specific solution from Gradle ; mostly just trying to raise awareness
Thanks

@oehme commented on Wed Aug 29 2018
Having thousands of child processes is definitely not normal. Gradle itself will only allow --max-workers "things" to run at any given time. Have you checked what some of these processes are doing? Maybe we can narrow down the culprit that way.

@mathjeff commented on Fri Sep 14 2018
Slight clarification/correction to my previous statement, it turns out that the >8000 number that I computed from pstree -p ${gradlePid} | wc -l was actually threads, not processes.
Now it turns out that based on jstack -l ${gradlePid} | grep 'ForkJoinPool-' | wc -l, it looks like there are 5445 ForkJoinPool workers. Most of the blocks of output from jstack look like:
"ForkJoinPool-10488-worker-56" #20122 daemon prio=5 os_prio=0 tid=0x00007f351c055000 nid=0x15718 waiting on condition [0x00007f2d22dec000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000006d0a73b50> (a java.util.concurrent.ForkJoinPool)
at java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Locked ownable synchronizers:
- None
I don't see very many instances of the text "ForkJoinPool" in the source code of Gradle but I do see a bunch of instances in the source code of the Android Gradle Plugin so I guess that's probably where I'll look next

@oehme commented on Fri Sep 14 2018
Make sure you're using the latest Gradle version. There was a Thread leak that we fixed in 4.10.

@mathjeff commented on Mon Sep 17 2018
Thanks - we're now upgrading from 4.9 to 4.10 .
Some more evidence that seems consistent with a thread leak is the fact that the number of threads (reported by pstree -l -a $gradlePid) seems to grow slowly from roughly 200 to >8000.
The high thread count does appear to be happening with Gradle 4.10 (in fact, with Gradle 4.10 it seems to max out at a higher number of threads: 13199) ; hopefully I'll be able to look closer eventually.  Not sure if this is a bug in the worker API or a bug in AAPT.

Comments about findings are on the original issue.


machineType.id is a number. machineType.value is a string like e2-micro.

message: "Invalid value for field 'resource.machineType': 'projects/kaholo-alpha/zones/asia-southeast1-b/machineTypes/334002'. Machine type with name '334002' does not exist in zone 'asia-southeast1-b'."
machineType: { id: '334002', value: 'e2-micro' }


Default value for when default_permissions argument is not provided

Hey @codemation  I do not know what is wrong with the linter github action. Do you have any suggestion ?
Thanks!

Hey @Rictus , good catch with isort, looks like you need to setup precommit locally and try to apply the change it found here via a new commit: https://github.com/codemation/easyauth/actions/runs/4058334348/jobs/6997735511#step:5:44

@codemation  Should look good now 👍


Add more detail to commands.md documentation

The documentation located at https://github.com/Microsoft/vsts-tasks/blob/master/docs/authoring/commands.md needs more detail added to it, here are some things that I am still unsure of after trying to puzzle it out for a while
All the issues I have found mainly have been with the documentation of ##vso[task.logdetail]

Does parentid need to be passed with every request or is it a write once field like name or order?
What format should time be in for starttime and finishtime? If it is a format without timezone should it be local time or UTC?
What format should the GUIDs be in?
What should we be putting in for type? The two examples that you show use build as the value passed in, should we always pass in build or are there other valid values?
What format does progress need to be in, can it accept decimals, can it accept commas?
Is there well defined behavior when two objects have the same value for order?
Examples are not actual examples: ##vso[task.logdetail id=new guid;name=project1;type=build;order=1]create new timeline record. would fail because id=new guid is not a valid value.

Also, it is not clear where "current operation" is used, or how doing parent id's provides any benefit. Neither is visible.
For example in the below image G7x Build is the parent of all other items in the list, also the last thing that happened to it should have set the current operation to "Handle SeclectedProjects", however there is 0 indication of any kind of hierarchy and the only place I could see "Handle SeclectedProjects" was in the agent console (and even then it did not include then name, it just wrote ": Handle SelectProjects")

I'm also struggling with the documentation. Can I use the type parameter to log Test results? Is test a valid record type? I'd like to write a reporter for Mocha test progress.

Can we get an update on this? I can't find any other documentation on these commands.
I can't get anything using task.logdetail working...

I too would like an update here. There appear to be a lot of useful vso commands we can utilize, but as the OP stated, the documentation is really lacking. The few commands I currently utilize I got working via trial and error.

What is the status of this?

Yes I agree this needs to added to the docs. For now you can peek into the code and answer some of the questions:
https://github.com/Microsoft/vsts-agent/blob/59b25208aa1c17a30c1dddc4828044fd89513599/src/Agent.Worker/TaskCommandExtension.cs#L84-L237

Does parentid need to be passed with every request or is it a write once field like name or order?

No it can be left out. If it is given it needs to be equal to the first usage.

What format should time be in for starttime and finishtime? If it is a format without timezone should it be local time or UTC?

DateTime.TryParse(dateTimeText, CultureInfo.CurrentCulture, DateTimeStyles.AdjustToUniversal, out dateTime) is used:
https://github.com/Microsoft/vsts-agent/blob/59b25208aa1c17a30c1dddc4828044fd89513599/src/Agent.Worker/TaskCommandExtension.cs#L611-L620

What format should the GUIDs be in?

new Guid(...) is used.

What should we be putting in for type? The two examples that you show use build as the value passed in, should we always pass in build or are there other valid values?

It looks like a custom string.

What format does progress need to be in, can it accept decimals, can it accept commas?

Int32.TryParse(percentCompleteValue, out progress) is used.

I don't see any sort of sub-timeline information anywhere in the latest version of the Build output UX.
In the newest UX for builds, where does the output of logdetail go? Is this even relevant information any longer?

I just ran into this myself, wanting to provide some detail about the state of some long running tasks.
It appears this issue noted that when the build log display changed, the timeline with this info disappeared. The author of the issue was directed to take it up with the Developer Community website which isn't great as far as explaining why this doesn't work as expected... or really what to expect.
Effectively the same question has been asked on the docs repo to see if the docs could add clarification around what the expected outcome of task.logdetail usage is... but again, the author was punted to ask on StackOverflow or the Developer Community website.
The only answered StackOverflow question I can find is here, which basically says this isn't available in the "new build output." There's a nearly identical answer on this question which wasn't accepted.
In all of these cases, the "answer" seems to be "someone should suggest that get implemented," but I think it's valid to have - somewhere in the documentation - that with the "new build output" view the timeline is no longer a thing that should be expected. Whether it should be re-implemented, sure, may be something that should be voted on, but man, it'd be nice to know that it's not just me unable to find where my cheese was moved.

As a follow up, apparently this was suggested on the Developer Community site and was closed due to 90 days of inactivity. It was reported a second time and summarily closed with the Microsoft representative pretty clearly not understanding what was being asked.
I'm not sure how we might be expected to make any traction on getting docs updated or getting the view fixed when it doesn't seem anyone's actually listening to the issue.

This issue still exists as more-or-less as originally reported. It should not be marked as stale. The next action needs to be taken by the Azure Pipelines Task team to clarify the format and semantics here. This is not something that we, the users of Azure Pipelines, can definitively answer ourselves.

Hi, the outstanding need has not been addressed. The bot should not close this issue without a response from the team which sets some expectations.

The following information could be added for example. I could not find this anywhere in the official documentation.
Thanks @ https://developercommunity.visualstudio.com/t/write-host-foreground-color-with-powershell-task-i/440605
##[error] : Highlight the line in the log red
##[warning] : Highlight the line in the log orange
##[section] : Highlight the line in the log green
##[command] : Highlight the line in the log blue
##[debug] : Highlight the line in the log purple
For example for Powershell you would use the following in your script:

Write-Host "##[section] This is colored green!"
Write-Host "##[command] This is colored blue!"
Write-Host "##[debug] This is colored purple!"
Write-Host "##[warning] This is colored orange!"
Write-Host "##vso[task.logissue type=warning;]This is colored orange!"
Write-Host "##[error] This is colored red!"
Write-Host "##vso[task.logissue type=error;]This is colored red!"

@snodecoder, information regarding logging commands and text highlighting already provided on the related page in the ms docs.

@snodecoder, information regarding logging commands and text highlighting already provided on the related page in the ms docs.

My bad ;)
Thanks!

Again:

Hi, the outstanding need has not been addressed. The bot should not close this issue without a response from the team which sets some expectations.

Again:

Hi, the outstanding need has not been addressed. The bot should not close this issue without a response from the team which sets some expectations.

This issue is not stale: we're still waiting on the Azure DevOps Pipeline team to address the gaps in the documentation.
The bot logic needs to be updated to not mark things as stale when the pending action is on the Pipelines teams itself.

Who needs to be pinged in order to get a response?

This issue is not stale: we're still waiting on the Azure DevOps Pipeline team to address the gaps in the documentation.
The bot logic needs to be updated to not mark things as stale when the pending action is on the Pipelines teams itself.

Hi @chwarr! We have documentation about the task.logdetail command here:
https://learn.microsoft.com/azure/devops/pipelines/scripts/logging-commands?view=azure-devops&tabs=bash#logdetail-create-or-update-a-timeline-record-for-a-task

The documentation at https://learn.microsoft.com/azure/devops/pipelines/scripts/logging-commands?view=azure-devops&tabs=bash#logdetail-create-or-update-a-timeline-record-for-a-task has the same issues as in the original report. This is a request to provide reference-level documentation for everything that logging can do.


Cant extract partial URL

Im trying to extract a partial url but am having trouble.
The source code looks like this

--
  |
  |

This is my website agent.
{
  "url": "https://somewebsite.com",
  "mode": "on_change",
  "type": "html",
  "expected_update_period_in_days": "10",
  "extract": {
    "url": {
      "css": ".posttitle div",
      "value": "href"
    }
  },
  "template": {
    "url": "{{ url | to_uri: _response_.url }}"
  }
}

unfortunately this does not work. How cant Iextract that partial url and then put the whole thing together ?

Try changing "css": ".posttitle div", to "css": ".posttitle",.

Try changing "css": ".posttitle div", to "css": ".posttitle",.

I tried that before.  I can post the site but i have to warn you guys its adult in nature.

Pretty sure it will, the Agent extracted empty values so the match for the .posttitle must have worked.

Dsander i think you might have been right. Something weird is going on where if i try to extract too many fields some dont work. Thank you for everything


Comprehensive numerical simulations

[x] let us compare with more benchmarks, and add dcov and hhg using existing R packages.
[x] show FH test power at best k, MGC power at best scale and best t, plus dcov and hhg power at best t.
[x] for given model, plot the power wrt increasing sample size, for example from 25 to 300 at intervals of 25. The range should be meaningful, i.e., if all powers become 1 after 100, add some noise to your model or reduce the range.
[x] Do above for all models you would like to include in paper, just one parameter setting for each model.
[x] use simulations to illustrate we are similar to FH test under their model, but can be better out of their model and in real data.
[x] Find out what ADJ1 and ADJ2 means in your old simulation. Maybe for adj1 you directly use adjacency for distance, and for adj2 you take Euclidean distance of adjacency matrix? I strongly suggest that for all the above simulations / real data, include MGC on adj2 for benchmark too.
(may not for inclusion in paper, but we really want to see the performance of all benchmarks to further our understanding! )

@jovo above is what I think for the numerical section, where we shall systematically illustrate our advantage over Fosick-H test for network dependency. What do you think? Anything to add?

@jove Thanks Cencheng. Actually we might need one or two real data example. With C.elegans data, both Fosdick/MNT pvalues are so low :(

seems like a good start to me!
for dMRI data, we could test independence of topology with vertex 3d location?

@jovo you mean you have relational dMRI data? Due to the context I assume in a paper, undirected and unweighted network (graph) would be most preferable.

i don't know what "relational dMRI data" means.  i have undirected graphs.
we can unweight them if you want.
On Fri, Aug 26, 2016 at 9:37 AM, Youjin Lee<EMAIL_ADDRESS>wrote:

@jovo https://github.com/jovo you mean you have relational dMRI data?
Due to the context I assume in a paper, undirected and unweighted network
(graph) would be most preferable.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/neurodata/Multiscale-Network-Test/issues/2#issuecomment-242737398,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AACjctD8PAgdF_Ix_Y2J8DZ5VBGlZsk1ks5qjuwFgaJpZM4Jton8
.

--
the glass is all full:  half water, half air.
neurodata.io, jovo calendar
https://calendar.google.com/calendar/embed?src=joshuav%40gmail.com&ctz=America/New_York

for theoretical purpose and simulations, yes we should use undirected /
unweighted graph;
but in real data we do not have to limit us to the assumptions :-)
On Fri, Aug 26, 2016 at 9:37 AM, Youjin Lee<EMAIL_ADDRESS>wrote:

@jovo https://github.com/jovo you mean you have relational dMRI data?
Due to the context I assume in a paper, undirected and unweighted network
(graph) would be most preferable.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/neurodata/Multiscale-Network-Test/issues/2#issuecomment-242737398,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALX0y4p9EPskchaIHbf__F2eSWAPV2ttks5qjuwFgaJpZM4Jton8
.

@jovo great! Can you please send me a link to such dMRI with relevant attributes?

@gkiar can tell you the best one to start with.
On Fri, Aug 26, 2016 at 12:08 PM, Youjin Lee<EMAIL_ADDRESS>wrote:

@jovo https://github.com/jovo great! Can you please send me a link to
such dMRI with relevant attributes?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/neurodata/Multiscale-Network-Test/issues/2#issuecomment-242778256,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AACjcslbPKpD_0t71aIrpqLdl9wQxJ5yks5qjw9jgaJpZM4Jton8
.

--
the glass is all full:  half water, half air.
neurodata.io, jovo calendar
https://calendar.google.com/calendar/embed?src=joshuav%40gmail.com&ctz=America/New_York

We're generally recommending the following voxelwise graphs: http://openconnecto.me/data/public/MR/m2g_v1_1_1/KKI2009/derivatives/bg/
Each node has an attribute called "spatial_id" which is a Morton Z-order representation of their physical coordinate in space. Depending on what language you're using, we've written a couple tools to help you turn these into (X,Y,Z) triples. These scripts can be found here (Python), and here (R).

@gkiar Thanks, I will take a look at it.

thanks!
On Mon, Aug 29, 2016 at 10:03 AM, Greg Kiar<EMAIL_ADDRESS>wrote:

We're generally recommending the following voxelwise graphs:
http://openconnecto.me/data/public/MR/m2g_v1_1_1/KKI2009/derivatives/bg/
Each node has an attribute called "spatial_id" which is a Morton Z-order
representation of their physical coordinate in space. Depending on what
language you're using, we've written a couple tools to help you turn these
into (X,Y,Z) triples. These scripts can be found here (Python
https://github.com/neurodata/ndgrutedb/blob/master/MR-OCP/mrcap/zindex.pyx#L86),
and here (R
https://github.com/neurodata/ndmg/blob/master/analysis/MortonXYZ.R).
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/neurodata/Multiscale-Network-Test/issues/2#issuecomment-243132964,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AACjcmwB3izgzjL-SubGx_-MNBftS3v-ks5qkuangaJpZM4Jton8
.

--
the glass is all full:  half water, half air.
neurodata.io, jovo calendar
https://calendar.google.com/calendar/embed?src=joshuav%40gmail.com&ctz=America/New_York

@jove @gkiar Triple (x,y,z) location is way too much correlated with network topology! No matter what scale and what statistics I use we get the least p-value.

that's great!
On Tuesday, August 30, 2016, Youjin Lee<EMAIL_ADDRESS>wrote:

@jove https://github.com/jove @gkiar https://github.com/gkiar Triple
(x,y,z) location is way too much correlated with network topology! No
matter what scale and what statistics I use we get the least p-value.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/neurodata/Multiscale-Network-Test/issues/2#issuecomment-243529364,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AACjcrmLmAYlkWAg1C0LDa5IEke54472ks5qlHJ2gaJpZM4Jton8
.

--
the glass is all full:  half water, half air.
neurodata.io, jovo calendar
https://calendar.google.com/calendar/embed?src=joshuav%40gmail.com&ctz=America/New_York

Very cool!

Youjin,
that is great for jovo and greg, probably because so their tools did the
right thing :-)
I have  quite a number of real networks plus labels, that I used to try
classifications on.
I can send them for you to try different tests.
On Tue, Aug 30, 2016 at 4:52 PM, Greg Kiar<EMAIL_ADDRESS>wrote:

Very cool!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/neurodata/Multiscale-Network-Test/issues/2#issuecomment-243577047,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALX0y8WHc97YZPta__q0x-IpYJ57hqUsks5qlJgmgaJpZM4Jton8
.

Youjin,
I tried all graph data I have, by testing between:
Euclidean distance of the adjacency matrix vs distance of the labels (+1 to
off-diagonal distances).
MGC equals the global correlation for all of them, which yields very
significant p-values.
So it may be too easy again...But I didn't try FH test, so maybe it worth a
try.
Attached graph data are all in matlab format, but you can use the matlab.R
package to get the data, or just copy paste the adjacency and the label
from Matlab.
Best,
Cencheng
On Tue, Aug 30, 2016 at 5:01 PM, Cencheng Shen<EMAIL_ADDRESS>wrote:

Youjin,
that is great for jovo and greg, probably because so their tools did the
right thing :-)
I have  quite a number of real networks plus labels, that I used to try
classifications on.
I can send them for you to try different tests.
On Tue, Aug 30, 2016 at 4:52 PM, Greg Kiar<EMAIL_ADDRESS>wrote:

Very cool!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/neurodata/Multiscale-Network-Test/issues/2#issuecomment-243577047,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALX0y8WHc97YZPta__q0x-IpYJ57hqUsks5qlJgmgaJpZM4Jton8
.

probably all done except last adjacency one


Fix mapping and analysis types

Description
Fix mapping and analysis types
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Do we have test coverage for the types being modified? Otherwise add the missing tests?

Are you referring to the code/validator changes or the API types?
The majority of the API type changes are just being more specific about the number variety. Otherwise they're difficult to test without trying to create an index with every mapper, analyzer, tokenizer & token filter with all relevant properties set.

Are you referring to the code/validator changes or the API types?

I merged this, but generally when I see changes in the schema itself I want to see tests that exercise that schema. If it's something unsupported it makes sense that there are no tests though :)


OSASINFRA-3402: Use Gophercloud v2.0.0

This is a no-op change as the v2.0.0 tag was cut from the same commit as v2.0.0-rc.3.

/retest

/cc mandre

/lgtm
/approve


SCMOD-11574: Explicitly specify base image

https://portal.digitalsafe.net/browse/SCMOD-11754
This change really should have been made as part of the SCMOD-5156 changes.  Clearly cases where no base image was specified and the docker-maven-plugin defaulted to busybox:latest were overlooked.

A developer build has not yet been created for this branch. Click here to go ahead and create the build...

CI Build Link:
http://sou-jenkins2.swinfra.net:8080/job/CAFDataProcessing/job/CAFDataProcessing~boilerplate-service~SCMOD-11574~CI


Fix release and build tasks within gruntfile

Several problems I've found trying to publish a new version of CartoDB.js.

grunt release task (specified in the releasing notes) was publishing dist files to S3.
grunt set_current_version task was ignoring the version selected in the prompt, if it was specified.
grunt build task working properly and ready to be used for CartoDB on-premises stuff (cc @azamorano).

cc @CartoDB/frontend

:+1: Thanks @xavijam!

Cool!

don't forget to remove branches! cc @xavijam

Thanks @javisantana


Added More jokes

Short description of what this resolves:
Added three more lines of jokes to the jokes.json file. As I am familiarising myself with contributing on github

Thank you for taking the time to contribute! This is sincerely appreciated. 😇
(Powered by GitMate.io)


Cannot submit test results

{"qid": "anet_val384", "pred_relevant_windows": [[0, 46]], "ans": "C"}
{"qid": "qvh_val521", "pred_relevant_windows": [[20, 56]], "ans": "C"}
{"qid": "anet_val1017", "pred_relevant_windows": [[16, 45]], "ans": "A"}

When I upload a zip file, a new error shows
Traceback (most recent call last):
  File "/tmp/codalab/tmpkZpq2s/run/program/evaluate.py", line 289, in
    acc, gqa, mIoU, r13, r15 = evaluate(truth_filename, submission_answer_filename)
  File "/tmp/codalab/tmpkZpq2s/run/program/evaluate.py", line 249, in evaluate
    submission = load_jsonl(submission_answer_filename)
  File "/tmp/codalab/tmpkZpq2s/run/program/utils.py", line 11, in load_jsonl
    with open(filename, "r") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/codalab/tmpkZpq2s/run/input/res/pred.jsonl'


Laravel 8 make default namespace App\Models instead of App

it annoying having to this every time I make a new model

Thanks to @danrovito for fixing it so fast, I will push up a new version

In the future, you could use a config as well by copying this file into your config folder and setting 'namespace' to whatever you need it to be. This is something that needs to be documented in this repo but overall is a general practice in a lot of Laravel projects and on new installs this config file should show up in your config folder automatically.


fix: recursive function not working

when a function call is executed, the code is not generating a new memory space for the function and making when the recursion is done, the previous variable is marked as undefined

The compiler supports the recursive function, after finishing most feature of the compiler, the Interpreter will get recursive function

solved on 0.0.1 version of the compiler


get an error  ' count(): Parameter must be an array or an object that implements'

Hello,
I'm using php 7. When I called my class method , I have hot an error below;
Warning:  count(): Parameter must be an array or an object that implements Countable in C:\xampp\htdocs\piksel-ng\api\vendor\marcj\php-rest-service\RestService\Server.php on line 836

disable PHP warnings, or insert:
else $regexArguments = [];
around line 813 in Server.php

The bug has been resolved in dev-master.
But it seem haven't been pushed to packagist.
To resolve it, just need to switch "*" to "dev-master" until the author update.


Feature: Binding emit for room messages

Out of all emit possibilities only socket based ones are supported
// sending to the client
socket.emit('hello', 'can you hear me?', 1, 2, 'abc');
// sending to all clients except sender
socket.broadcast.emit('broadcast', 'hello friends!');
// sending to all clients in 'game' room except sender
socket.to('game').emit('nice game', "let's play a game");
// sending to all clients in 'game1' and/or in 'game2' room, except sender
socket.to('game1').to('game2').emit('nice game', "let's play a game (too)");

Please add support to io:
// sending to all clients in 'game' room, including sender
io.in('game').emit('big-announcement', 'the game will start soon');
// sending to all clients in namespace 'myNamespace', including sender
io.of('myNamespace').emit('bigger-announcement', 'the tournament will start soon');
// sending to a specific room in a specific namespace, including sender
io.of('myNamespace').to('room').emit('event', 'message');
// sending to individual socketid (private message)
io.to(socketId).emit('hey', 'I just met you');

It can be done in the exact same way as the socket ones. If i was to write an more generic function for hooking onto emits I would get something like:
private emitHook(
    originalEmit: (event: string, ...data: unknown[]) => boolean,
    emitter: NodeJS.EventEmitter,
    labels: prom.LabelValues<string>
) {
    return (event: string, ...data: unknown[]) => {
        if (this.blacklisted_events.has(event))
            return originalEmit.apply(emitter, [event, ...data]);

        const labelsWEvent = {event: event, ...labels};
        this.metrics.bytesTransmitted.inc(labelsWEvent, SocketIOMetrics.dataToBytes(data));
        this.metrics.eventsSentTotal.inc(labelsWEvent);

        return originalEmit.apply(emitter, [event, ...data]);
    };
}

Which can be used to hook onto socket events
        // Hook into emit (outgoing event)
        socket.emit = this.emitHook(socket.emit, socket, labels);

And io events
    // Hook into emit (outgoing event)
    server.emit = this.emitHook(server.emit, server, labels);

Thanks for the suggestion. This will be a good candidate for a feature for version 2 of this project. It can be found on the v2 branch but it is very much a work in progress.
Feel free to contribute if you like!


Tests and others update

Updated lots of things to be 2.1 ready.

Closing this as many of these changes are unnecessary, for example manifests need to use the stix media-type.


soundfileLib: adding buffer to player and recorder

I often have clicks when recording audio from Max using jcom.soundfile.recorder???.
To prevent this, soundfilePlayer and soundfileRecorder should be equipped with an internal buffer - similar to sfplay~ and sfrecord~.
see more on the original redmine issue http://redmine.jamoma.org/issues/727

TTSoundfileLib is being quarantined for 1.0 release. If this issue is addressed, it will be done for 1.1 or in jamoma2.


Logging: Changing Future -> ApiFuture

Progress on https://github.com/GoogleCloudPlatform/google-cloud-java/issues/1580 .

Coverage increased (+0.004%) to 81.441% when pulling 319b783cdb21764ea431cb9ea8eaa91d3dd9108b on garrettjonesgoogle:master into 5cbbbae1ac80aecfa742972097c705eb8871930c on GoogleCloudPlatform:master.


heroku rename adds extra .git to remote

A user reported that when renaming, the git remote is getting .git added to it twice, so you end up with a remote like git@heroku.com:app.git.git.

Thanks, pretty sure I must have introduced that bug when I switched from returning the data straight from the api for the git url and the old way (creating the git url string based on the app name).


Considerations for Week 4

By Tuesday evening

[X]  Initialize app.R @reikookamoto

By Wednesday evening

[ ] Add bar plots to app.R @clsu22

By the end of Thursday's lab

[ ] Add heat maps to app.R @moniquewong
Attempt deploying app (from milestone4 branch) on Heroku in class

By Friday evening

[ ] reflection.md

By Saturday 6 pm

[ ] Submit links to milestone4 branch and Heroku app
[ ] Individually, complete peer review and teamwork assignments

New file is now available. Refer to #8

Things to consider when making further changes:
Use double quotes as they're preferred in R
Use lower case and underscores for variable names (storeRadioItems --> store_radio_items)
Let's try to re-use variable/function names from app.py for consistency
Dash Bootstrap Component doesn't exist for R

See #24 for final submission links


Fix/slider broken anims

If animations result in errors during update, animation loop will stop ticking, forever. Even after a context reset. Fix that by first requesting a next frame, then doing logic.
Such broken animations could be triggered by calling SliderComponent.animateToValue(); so adding guards against weird slider values. (There is can be a loop in between knop position change -> update value -> change knob position, normally broken by value == lastvalue, but !(NaN == NaN))

Is it safe to call requestAnimationFrame before doing update? Can't this result in two ticks happening before the update has taken place, causing race conditions?

Not unless after update() it can result in two ticks ... I mean usually update() is practically a no-op when the lists are empty.
It is defined as called before the next repaint. So not the current repaint.

But what if our update() takes 2 repaints to complete? Usually we would drop a frame, but now we already requested to be called again causing a second call to update() while it is still not finished with the previous pass, right?


Action Bench Build Failing

https://github.com/hanabi1224/Programming-Language-Benchmarks/actions
The scheduled bench build has been failing for the past four months.

The specific error is, ./.github/lua.sh: line 26: luajit: command not found
Link to log - https://github.com/hanabi1224/Programming-Language-Benchmarks/actions/runs/7054359292/job/19203092827#step:15:197


Adding Media Browser

Hi,
Is it possible to add Media Browser in the apps that HTPC manage ?
There is XBMC and Plex, will look nice to get Media Browser (OS alternative to Plex, at least the Server is OS)

I would love to see this as well


error: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version while accessing

Operating system and version:
MacBook Ai

nvm debug output:

nvm ls output:
error: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version while accessing

How did you install nvm? (e.g. install script in readme, Homebrew):
I have godday shared hosting , I am connecting with ssh and at home directory I ran
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.34.0/install.sh | bash

What steps did you perform?

What happened?
I am getting tthe following error :
error: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version while accessing

What did you expect to happen?
Install nvm , in order to install nodejs

Is there anything in any of your profile files (.bashrc, .bash_profile, .zshrc, etc) that modifies the PATH?
no

If you are having installation issues, or getting "N/A", what does curl -I --compressed -v https://nodejs.org/dist/ print out?

Please stop filing duplicate issues for the same thing. Closing in favor of #2031 and #2040.

fatal: unable to access 'https://github.com/ChrissaLouvrou/lesson0.git/': error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version
This is the message i am getting when trying to connect with my repository.

@ChrissaLouvrou that implies that your computer's SSL certificates, and/or system clock, are wrong/out of date.


Fallback to default value if setting is not provided by client

This PR tries to provided a more generic and robust solution to https://github.com/ansible/ansible-language-server/issues/391
The solution from @yaegassy is good but IMHO has 2 drawbacks:

the fallback value is hardcoded and can differ from value defined in defaultSettingsWithDescription (if this one changes in the future).
it only handles a subset of the settings. For example, the server output a similar error if the client does not provided values for ansible.ansibleLint settings.

PS: I changed the get function signature to be able to use an await call in the function but I don't know all the consequences of this change. At least, it works as expected with my manual tests.

The PR from @yaegassy was important. And it is essential to set a default value because a few clients do not support this setting. For things to work well in both cases, it was necessary.
FYI, this was the reason for failure in the auto-completion feature in the extension's v0.11.11 :)

English is not my mother tongue. So maybe my message was not well understood: the PR from @yaegassy is good and correctly fix the issue.
But IMHO, it does not take the problem at its root and does not prevent similar issue with other settings (current and futur). This is why I suggest this PR which ensure that the get function will always return a structure with all the known settings even if a setting is unknown by a client. It is done by merging default settings with settings provided by the client. Settings from client take precedence over default settings (see https://www.geeksforgeeks.org/lodash-_-merge-method/ for more information).
An implementation of Perl Language Server use a similar approach (with a simple merge and not a recursively merge that is needed by ALS): https://github.com/bscan/PerlNavigator/blob/main/server/src/server.ts#L178

I agree, and I am absolutely in favor of this PR. All I was saying was that @yaegassy's PR helped us resolve the broken issue.
Thanks for this PR.
In addition, it would be awesome if you could add a few tests for this :)

I try to add tests to check if the merge of settings is correctly done. But it's not a trivial task. If I understand well how it works, I need to simulate a client to answer workspace/configuration requests from the test framework. I started to look for a way to do it but without success. I need to read more documentation. So it will take some time before I can add tests.
@ssbarnea: presently the typescript transpilation failed if a setting is not defined in defaultSettingsWithDescription or if it is defined with no default value. It is not enough?

@fredericgiquel, you can take a look at the doCompletion and doCompletionResolve tests. The helper file has all the necessary connection and workspace configurations to dummy the client.


Published 0.20.0

Updated dependencies

Seems to be a compiler bug in beta https://github.com/PistonDevelopers/image/issues/645. Merging.


Get coordinates/position of button form field.

I am using pdf-lib in our NodeJs project. We are modifying our existing pdfs to place an image on button form field. I want to get the exact  coordinates of the button field.
Can you please help me with this.
Thanks.

Up! Same question here!

Up! Same question here!

@MorganWaya I then did it manually by using acroField of PDF lib. It required some research but it eventually resulted as desired.

If placing an image on a button, not sure if this helps you: https://pdf-lib.js.org/docs/api/classes/pdfbutton#setimage
As long as you know the name of the button.
https://github.com/Hopding/pdf-lib/issues/602#issuecomment-693010472
Maybe that helps?

@ArifNawaz36 seems to have the right idea. It can be done, but you'll have to use the lower-level widgets API. You can find some examples in other issues.


chore(icons): update iconography guidelines

Remove reference to sprite sheet on the iconography guidelines.
Changelog
New
Changed

src/content/guidelines/iconography/usage.mdx

Removed

The dev usage is a bit over my head. Is there adequate usage guidance following this deletion?

Closing in favor of: https://github.com/carbon-design-system/carbon-website/issues/1613


Add PropTypes dependency

Hey there, thank you for the awesome package. However, issues are with react 16 ;(
Add PropTypes in order to avoid React 16 error

Okay, I had to update <3


Basic Zone - change init sequence

ISSUE TYPE

Improvement Request

COMPONENT NAME

UI, Zone

CLOUDSTACK VERSION

4.15.1

CONFIGURATION

Single Server deployment
Basic Zone
LocalStorage enabled
OS / ENVIRONMENT

Ubuntu 20.04.2
SUMMARY

Creation of a Basic Zone after a clean CloudStack setup fails with the error: TypeError: h.prefillContent.secondaryStorageProvider is undefined
The host can't be added BEFORE secondary storage is created. GUI wizard asks to setup Secondary Storage AFTER host setup, which causes this problem
So, need to change the init sequence.Init Secondary Storage right after creating a Zone (when secondary storage is zone-wide) or after creating a Cluster (when secondary storage is cluster-wide)
STEPS TO REPRODUCE

Change init/create sequence for Basic Zone so that Secondary Storage is created BEFORE adding the host

EXPECTED RESULTS

Basic Zone is created without errors

ACTUAL RESULTS

Error creating a Basic Zone

Fixed in https://github.com/apache/cloudstack/pull/5303


Improve documentation colors

I think the links in the documentation are too dark to distinguish from the rest of the text, so this PR makes them a brighter purple.
I've also fixed the color of the logo in the header, and added instructions to the README for running the documentation server locally.
Before:

After:

Is there interest in this PR @kristiankielhofner? Anything I can do to get it merged?

Is there interest in this PR @kristiankielhofner? Anything I can do to get it merged?

Hey @brianpeiris - thanks for your PR! I agree the contrast is an issue. I've been slammed with other tasks but I will review and likely merge this shortly.
Thanks again!

Thanks for the merge!

Thanks for the contribution! Sorry again it took so long.


No of the supported Kotlin-Plugins (multiplatform, js, native, jvm) has be applied to the project

@comahe-de @Shabinder When re-building gradle on the project with:
plugins {
    id("de.comahe.i18n4k") version "0.1.3"
}

it thows: 'No of the supported Kotlin-Plugins (multiplatform, js, native, jvm) has be applied to the project'

@ArnauKokoro Do you have an example project that throws this error?
This error can be avoid when setting the generationTargetPlatform field in the plugin config (see also GenerationTargetPlatform)
e.g.
i18n4k {
    generationTargetPlatform = de.comahe.i18n4k.generator.GenerationTargetPlatform.MULTI_PLATFORM
}

@comahe-de Is it possible to add org.jetbrains.kotlin.android to the plugin checks in I18n4kPlugin ? So that I can apply this plugin in an android project ?

@zeromod I have added Android support for the generation plugin in version 0.5.0. Will be available within the next day. May you can have a try...


Auto import component does not work

Auto import component does not work anymore typing < and pressing ctrl+space

I noticed the same. I figured out that the last working version of the VSCode extension is 1.0.8. The auto import of components broke with 1.0.9. I suspect the following commit as the cause since that where the volar.completion.autoImportComponent setting got changed/removed.
https://github.com/johnsoncodehk/volar/commit/0779ae2d5c2bf17d0909cbf58e8d4abe61bbd2b8#diff-e38db040908417b4291b2bfad8d46d0d9f972cfa9bdf79997624aa414b372dee

As a workaround you can input <Vue to trigger it for now.

A somewhat related bug is that even using the workaround, the behaviour for components structured like ComponentName/index.vue the resulting import is incorrect.
import Index from<EMAIL_ADDRESS>

for me it autofills import AutocompletePersonsOfSpace from 'components/APIComponents/AutocompletePersonsOfSpace.vue';
but I need import AutocompletePersonsOfSpace from<EMAIL_ADDRESS>Does anybody know how to do this?

Closing this one as many things have changed since 2.x. Feel free to create a new issue with a minimal reproduction if you're still having issues, thanks!


No command history/Arrow Up in Python interactive window

VSCode Version: 1.47.3
OS Version: Ubuntu 18.04.4 LTS

Steps to Reproduce:

Install Python Extension
Create Python-Script
Send selection to interative window
Type some code in interactive window
try arrow-up

Does this issue occur when all extensions are disabled?: Yes
I tried different versions of python and different versions of the python-extension. There seems to be no command-history in the interactive window.
Arrow up works when I type something in the interactive window and open ipython in the integrated terminal. Then thera are even the commands I executed in the interactive window.

/extPython


WIP: SDK-1275: Change ActivityDetails errstrings to provide something more meaningful

Additions

Add error types in errors.go and requests/errors.go which can be used for flow control by client code.

Changes

Change errStrings []string in GetActivityDetails to err error
Use HTTPError to describe HTTP-Request based error messages rather than using strings

Hi @emmas-yoti just adding the definition of temp.Error that we came up with the other day.
package temp

// Error is a temporary error. Trying the same operation again with the same
// parameters might change the result. This error is useful when the result of
// an operation depends on external factors that might change over time.
// E.G Trying to access an external API through a network connection could
// result in a temporary error if the service is down. Retrying again with the
// same parameters could work if the external service gets fixed.
type Error struct {
	Err error
}

func (e *Error) Error() string {
	return e.Err.Error()
}

func (e *Error) Unwrap() error {
	return e.Err
}

To mark an error as temp you simply do
err = &temp.Error{err}


Simplify E2Lib Encode/Decode Table Generation

Saw this and thought it was unnecessarily complex and a little hard to read. Hopefully a performance, or at least memory, improvement.
Only functional difference is that bit.tohex uses lowercase hex. There are no instances of the output of E2Lib.encode being used in a way that this would cause an issue. Namely, this is only used when sending/receiving E2 chip files and E2 file extension files.

Can't we just swap the functions to use util.Base64Encode / util.Base64Decode?
Or better yet, just not use the function at all. It looks like it's just used in place of net.ReadData and net.WriteData.

Can always count on you to think outside the box.

Is this going to break old e2 dupes?

Is this going to break old e2 dupes?

I striked it out, but its output is never serialized to a file, only used for network serialization. It might cause issues if a third-party extension uses it for some reason.


Problems with Dataset

Hello,
I am having difficulty aligning the URL from the "curlie_filtered.csv.gz" file, which contains the columns "url," "uid," "label," and "lang," with the HTML from the "html_content.json.gz" file, which contains the columns "uid" and "HTML."
Unfortunately, it appears to me that the "uids" between the two files do not correspond.
I hope that you will be able to provide guidance on this matter.

Dear XnibyH,
We fixed the indexing issue in the dataset and published a new version on Figshare. Indeed, some files were based on two different indexing, making it impossible to align data. Now you should be able to correctly associate URLs with their HTML content. Thanks for spotting that!

Hello Sylvain,
Thank you for addressing the indexing issue and publishing a new dataset version on Figshare. I appreciate your prompt attention to this matter and look forward to further exploring this dataset.


Jakarta-ize SOAP with Attachments API Document

This change should:

Change javax package mentions to jakarta
Update copyrights to Eclipse Foundation
Replace mentions and links to old Java EE specifications to Jakarta EE specifications.

Feedbacks are welcome.

I'd change also the following sections

"Status of This Document" mentioning that the document now is developed under the Jakarta EE Specification Process

Should I remove the previous Statuses and just mention the Jakarta EE Specification Process now?

Should I remove the previous Statuses and just mention the Jakarta EE Specification Process now?

no, not yet, there are more important things to fix first...

Sorry about the delay on fix the comments, I was in a process of starting to work in a new company. :)


Upgrade Debian from buster -> bullseye (10 -> 11) in our Dockerfile

Debian Bullseye release notes: https://www.debian.org/News/2021/20210814

Done


How to start a debugging session via API ?

I am trying to integrate in https://github.com/scalameta/metals-sublime with this package.
It is a helper package for Metals and already takes care of the installation. The debug workflow get started via code lenses, so as far as I understand there isn't a need to implement the adapter class.
I already have a host, port and the other debug parameters but I couldn't find a way to pass this info to the debugger and start it.
Can you point me to which API command with its argument I should use ?

This package is not designed to be used externally like LSP there is no API for other packages to consume right. All of the adapters live here https://github.com/daveleroy/sublime_debugger/tree/master/modules/adapters
The AdapterConfiguration is how every debug adapter is registered with the system there is no way to do what you want which is basically to give this package an already running debug adapter and a configuration to use with it.
This is the flow I think you should start with https://scalameta.org/metals/docs/editors/vscode/#via-a-launchjson-configuration
AdapterConfiguration does basically two main things

It handles installing an adapter  (install(...), installed_status(...), installed_version(...))
Starting a debug adapter (start(...)) when a user tries to run a configuration for that adapter (specified via the type field)

For 1. You can probably check that https://github.com/scalameta/metals-sublime is installed or something.
For 2. You probably need to ask the metals LSP to start the debug adapter because I guess it is part of the LSP client and it gives you a host/port which you would return with SocketTransport. This seems to be the LSP command https://github.com/scalameta/metals-languageclient/blob/87425244724013a92ac5f442fc8e199eb89212d1/src/interfaces/ServerCommands.ts#L64
After that is working code lenses probably just give a configuration to run so something like the following command could be added
`window.run_command('debugger', {
'action': 'start',
'configuration': {
...
}
})
That command would work just like if the user has that configuration in their .sublime-project

The debugger now supports starting via a configuration with both start and open_and_start commands. Still need an adapter to handle the configuration type.
window.run_command('debugger', {
	'action': 'open_and_start'/'start',
	'configuration': {
		...
	}
})```

@daveleroy thanks for the update, I am giving this issue another try and so far I managed to:

Capture the code lens click event
call  Metals debug-adapter-start with the code lens arguments
get back an object like this {'uri': 'tcp://<IP_ADDRESS>:65264', 'name': 'example.Hello'}
run window.run_command('debugger', {  'action': 'open_and_start'/'start',  'configuration': {  	...  } })
Notice the error: Unable to find debug adapter with the type name "None"

As long as I create an instance of AdapterConfiguration for Metals in this repo, is this the right approach ?
Also what are the values that I am supposed to put in the configuration object ?

cc @prassee

@daveleroy thanks for the update, I am giving this issue another try and so far I managed to:

Capture the code lens click event
call  Metals debug-adapter-start with the code lens arguments
get back an object like this {'uri': 'tcp://<IP_ADDRESS>:65264', 'name': 'example.Hello'}
run window.run_command('debugger', { 'action': 'open_and_start'/'start', 'configuration': { ... } })
Notice the error: Unable to find debug adapter with the type name "None"

As long as I create an instance of AdapterConfiguration for Metals in this repo, is this the right approach ?
Also what are the values that I am supposed to put in the configuration object ?

cc @prassee

https://github.com/scalameta/metals-vscode#via-a-launchjson-configuration that is the standard way to setup debugging through a configuration and I would start by trying to get this setup and working first and then adding support for the code lens stuff.
Yes you need an AdapterConfiguration and your configuration needs a type field which is used to route it to a specific AdapterConfiguration so it can decide how to setup the transport for the debug adapter protocol. A request field of either launch or attach to specify what debug adapter request to make and a name field. Other configuration fields are adapter specific and get sent to the adapter during the launch/attach request.
They seem to just pass in a dummy configuration for vscode for code lenses here https://github.com/scalameta/metals-vscode/blob/821c44598870d9f6cabc7f28b788401eb7e3e0a7/src/debugger/scalaDebugger.ts#L136
I believe the the code lens is the configuration just missing a type, request and name so presumably you could add those to the configuration and the AdapterConfiguration starts the adapter with the lsp debug-adapter-start request so it actually supports normal configurations not just code lens stuff.
I believe you probably want something like the following on the Debugger side

class Scala(dap.AdapterConfiguration):
	type = 'scala'
	docs = 'https://github.com/scalameta/metals-vscode#readme'

	installer = util.GitInstaller(
		type='scala',
		repo='scalameta/metals-vscode'
	)

	async def start(self, log, configuration):
		# Make sure LSP and LSP-metals are installed
		util.require_package('LSP-metals')
		util.require_package('LSP')

		response = await util.lsp.request('metals', 'workspace/executeCommand', {
			'command': 'debug-adapter-start',
			'arguments': configuration,
		})

		uri: str = response['uri']
		name: str = response['name']

		address, port = uri.split('tcp://')[1].split(':')

		configuration.name = name

		return dap.SocketTransport(log, address, int(port))


Carousel Auto play

It is possible to make the autoplay not start over, but continue. The customer does not like it to come back at the end of the slides.
Thank you

Any answer?

Good afternoon. In this case I would have to do it completely in javascript, I can't think of a form using pure css and that would take away the main attraction of the slider.


[REQ] Add overload type hints for from_openapi_data_

Is your feature request related to a problem? Please describe.
Add overload type hints for from_openapi_data_oapg
Describe the solution you'd like

Describe alternatives you've considered

Additional context

resolved by 3.0.0 release


Using Memcached Provider helper and allow for persistant_id

This will fix issue:
https://github.com/php-cache/issues/issues/59
https://github.com/php-cache/issues/issues/60
Ping @angelsk, Does this work for you?

[Current coverage][cc-pull] is 24.09%

Merging [#38][cc-pull] into [master][cc-base-branch] will decrease coverage by 0.08%

@@             master        #38   diff @@
==========================================
  Files            34         34
  Lines           575        577     +2
  Methods          77         77
  Messages          0          0
  Branches          0          0
==========================================
  Hits            139        139
- Misses          436        438     +2
  Partials          0          0

Powered by Codecov. Last updated by [b4fc0e6...121727e][cc-compare]
[cc-base-branch]: https://codecov.io/gh/php-cache/adapter-bundle/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/php-cache/adapter-bundle/compare/b4fc0e6e8413de9e8e61048439137cfeb08baef4...121727ed05b0b80b44e06cbffc315aa3340624be?src=pr
[cc-pull]: https://codecov.io/gh/php-cache/adapter-bundle/pull/38?src=pr

Could we get a new tag please @Nyholm :) Then I can pull it into my project.  Much obliged.

I just tagged 0.3.4


Tweak spec for special case when split_dim = concat_dim in all_to_all

This was pointed out by Yuyou on the StableHLO Discord: https://discord.com/channels/999073994483433573/999074539138990131/1246199149905449060

Thanks for pointing this.

Thanks for the review!
I actually learned about this from someone on Discord! Thanks for the reminder, added proper credit to the PR description.

Hi! This is  Yuyou, and glad I did help on this project 0w0.
I am a PhD student at University of Utah, and working on compiler testing and verification so I am pretty sensitive to formalized constraints.
Besides, I am really surprised SHLO has a so nice document for operations and their semantics as well as constraints, appreciate your effort.

Hi! This is Yuyou, and glad I did help on this project 0w0. I am a PhD student at University of Utah, and working on compiler testing and verification so I am pretty sensitive to formalized constraints. Besides, I am really surprised SHLO has a so nice document for operations and their semantics as well as constraints, appreciate your effort.

Hi there!
Thank you for taking the time to look at the spec and for sharing a bit about yourself and your interests. I'm happy to hear that the StableHLO spec is useful to you! If you find more gaps in the spec (or any other area of the project), please feel free to open an issue or a pull request, or simply to ping us on Discord again.
Thanks :)


How to set keywords highlight for each language ?

VSCode Version: 1.0.0
OS Version: windows 10

I downloaded a theme from "http://tmtheme-editor.herokuapp.com/#!/editor/theme/Eclipse", but it seems that some setting have no effect ?

Same problem on editplus theme.

The differences that you are seeing in JS and HTML are because we have different tokenizers than http://tmtheme-editor.herokuapp.com.
The scopes that tokenenizers should emit are only loosely specified, so there's no easy right or wrong. But if you have a specific request, please make a request with:

code example showing a specific token
the scope you are expecting for that token


Refactor Sprite classes

Thoughts:

There's lot of repetition going around. :)

PR #12 just merged. Closing.


UX upgrades

https://www.notion.so/Website-MVP2-Feedback-4cdb42545a5640e284232735a323155e#10d9bcb033d880d29e3ecd4e5cf14aa4

footer has this weird line as header for right most column
edit profile should be full width not half width


capping doesn't work

I love the plugin but for last 15 days plugin shows ads after I close the app and doesn't respect capping number that I set on AdMob. what may be the cause for this?

How are you setting the frequency capping, and where?

And what do you mean it shows ads after you close the app. What kind of ads? Interstitial?

Same problem here, with phonegap. The FC doesnt work evertime on iOS with an interstitial banner. The interstitial should be show 1 time per day, but sometimes you see the interstitial after every klick. The IP adress doesnt change in this time.
autoShow is false in the config
We call the interstitial with window.plugins.AdMob.showInterstitialAd(); everytime we change the page.
I expect, that I can call the interstitial everytime and the FC is managed by the ad-network and not by the app.

The frequency cap is managed by the admob servers, not our app. What I would suggest is that you set a cookie on the phone that determines the date and if an interstitial was shown yet, and if it was shown, do not call showInterstitialAd().