[Feature]

Description
Chinese Zodiac calculator that allows users to input their birth year and receive their corresponding Chinese Zodiac sign. The feature should also provide a brief description of the characteristics associated with each Zodiac sign.
Screenshots
No response
Checklist

[X] I have checked the existing issues.
[X] I follow Contributing Guidelines of this project.
[X] I am a GSSOC'24 contributor.
[ ] I am a IWOC'24 contributor.
[ ] I am a JWOC'24 contributor.
[ ] I am a SWOC'24 contributor.

Code of Conduct

[X] I agree to follow this project's "Code of Conduct".

@Rakesh9100 please assign it to me!

Please go ahead and create the PR, Rakesh will start assigning the issues and merging PRs soon.
Please keep the UI similar to the zodiac sign calculator to have some harmony.


[3pt] CatFIM duplicate records bug

In the most recent run of CatFIM, we have been finding duplicate lids in the HUC_messages.txt file which has translated to duplication of those LIDs in the CatFIM outputs.
Add a section of code to catch and mitigate duplicate LIDs in the HUC messages file if there are two and one of them starts with '---', because that is more of a warning than a message. This will help prevent LID duplication in the outputs.
Duplicate LIDs and HUCs that could be used for testing:
gaka2    19020101
gaka2    19020101
gcka2    19020101
gkna2    19020101
jcta2    19020101
mena2    19020101
mnta2    19020101
paxk    19020101
pxka2    19020101
scca2    19020101
slna2    19020101
ccga2    19020102
crta2    19020102
dada2    19020102
egha2    19020102
enka2    19020102
ghya2    19020102
glna2    19020102
gnna2    19020102
gpoa2    19020102
gpoa2    19020102
grsa2    19020102
Code snippet to adjust:
  # TODO: Aug 2024: This is now identical to the way flow handles messages
  # but the system should probably be changed to somethign more elegant but good enough
  # for now. At least is is MP safe.
  # Write all_messages to huc-specific file.
  # MP_LOG.lprint(f'Writing message file for {huc}')
  huc_messages_txt_file = os.path.join(huc_messages_dir, str(huc) + '_messages.txt')
with open(huc_messages_txt_file, 'w') as f:
  for item in all_messages:
  item = item.strip()
  # f.write("%s\n" % item)```

This issue was fixed with the code added in Commit https://github.com/NOAA-OWP/inundation-mapping/commit/c589232812a268dbdffe380f3ad3f841c2c07e2d .
# Check for duplicate sites in the messages
lids  = [msg.split(':')[0] for msg in all_messages]
duplicate_lids = set([x for x in lids if lids.count(x) > 1])
# If there are duplicate sites and one of the lines has '---', drop that line
filtered_messages = [
     msg for msg in all_messages
     if not (msg.split(':')[0] in duplicate_lids and ':---' in msg)
]


Could not find a package configuration file provided by "grid_map_ros"

dockerfile add :
RUN sudo apt-get -y install ros-indigo-grid-map

This issue has been fixed in the following.
https://github.com/CPFL/Autoware/pull/661
But it has not been merged in the master branch yet. I don't know when it will be merged.

It has already been merged in the master branch.
https://github.com/CPFL/Autoware/commit/e05cf7b7d6c065fa3f4d7abe92ba0f3f9a4f753c#diff-ebacf6f6ae4ee68078bb16454b23247d
Thanks,

Could not find the required component 'gps_common'. The following CMake error indicates that you either need to install the package with the same name or change your environment so that it can be found.
CMake Error at /opt/ros/indigo/share/catkin/cmake/catkinConfig.cmake:83 (find_package):
Could not find a package configuration file provided by "gps_common" with
any of the following names:
gps_commonConfig.cmake
gps_common-config.cmake

Add the installation prefix of "gps_common" to CMAKE_PREFIX_PATH or set
"gps_common_DIR" to a directory containing one of the above files.  If
"gps_common" provides a separate development package or SDK, be sure it has
been installed.
Call Stack (most recent call first):
sensing/drivers/imu/packages/xsens/src/xsens_driver/CMakeLists.txt:7 (find_package)
add below from readme:
sudo apt-get install ros-indigo-desktop-full ros-indigo-nmea-msgs ros-indigo-nmea-navsat-driver ros-indigo-sound-play ros-indigo-jsk-visualization ros-indigo-grid-map ros-indigo-gps-common
sudo apt-get install ros-indigo-controller-manager ros-indigo-ros-control ros-indigo-ros-controllers ros-indigo-gazebo-ros-control ros-indigo-sicktoolbox ros-indigo-sicktoolbox-wrapper ros-indigo-joystick-drivers ros-indigo-novatel-span-driver
sudo apt-get install libnlopt-dev freeglut3-dev qtbase5-dev libqt5opengl5-dev libssh2-1-dev libarmadillo-dev libpcap-dev gksu libgl1-mesa-dev libglew-dev

I tested the master branch again, then  the error occurred.
I am going to try to fix it.

ubuntu@ip-172-30-0-188:~/github/Autoware/docker$ sudo sh build.sh
nvidia-docker | 2017/06/06 10:39:23 Error: unsupported Docker version
change docker readme：
use latest nvidia docker:
wget -P /tmp https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.1/nvidia-docker-1.0.1-1.x86_64.rpm
sudo rpm -i /tmp/nvidia-docker*.rpm && rm /tmp/nvidia-docker*.rpm

I have made a pull request .
https://github.com/CPFL/Autoware/pull/680

I have made a next pull request to fix README.md.
https://github.com/CPFL/Autoware/pull/683

They was merged into master branch.


M03-LAB08: Update Lab module name

Related Issue
M03-LAB08: Update Lab module name #341
Checklist
Mark completed with "x" between brackets, "[x]", or checking the box once the PR is created:

[x] Has related GitHub Issue 💥 #341
[x] Tested it
[x] Read the PR collaboration guide 👓 Collaboration Guide 📝

Changes proposed in this pull request:

Update Lab module name to "Create and manage containers using Docker and Kubernetes"

great find, thanks @fimdim :)

@LuizMacedo will wait for your approval

Thank you so much @fimdim!


multi-level or nested grouping

Hi,
Just wondering is there any plan to add  multi-level or nested grouping features in any of upcoming releases?.

Hi @nabeelahm2, I'm not really sure how useful the grouping functionality is anyway as currently implemented.
Take for example the demo: http://ng-table.com/#/demo/4-1
For me it doesn't make sense that the list of user's are split between page 1 and 2 of the grid.
Instead of using the grouping feature of ngtable, I would probably shape the data to be displayed in the grid already grouped. For example:
{
  name: "Christian",
  children: [{ name: "xx" }, { name: "yy" }]
}

In the html for my table I would then use a nested table to render the children. That nested table would not require the ng-table directive.
Note: I haven't tried the above so I might be talking rubbish!


Support extracting Private DICOM tags using the png-extraction

Currently, the png extraction module supports extracting only public DICOM attributes. Extending it to support private tags can significantly help research works that depend on those.

Hello, I am working on this issue.

ok, thanks for the update.

Hi @pradeeban, in some DICOM files there would be some private tags which the user does not want to extract such as histogram tag shown  in the picture below. What I would like to do is extract only the tags whose value length is less than certain threshold.  Is there any other way to handle this better?

This is a good question. So let me answer in detail.
If you check https://github.com/Emory-HITI/Niffler/blob/dev/modules/png-extraction/ImageExtractor.py, you will see the below lines:
# dicom images should not have more than 300 dicom tags
if len(kv)>300:
So, we are ignoring images with more than 300 attributes. Similarly, you could use the first/easy approach you mentioned (extract only the tags whose value length is less than a certain threshold).
You likely will need to add a property in https://github.com/Emory-HITI/Niffler/blob/dev/modules/png-extraction/config.json
"PublicHeadersOnly": true,

The above default will ensure by default only public headers will be extracted. When you set that to false, you will also extract the private headers. This will ensure that we are not bombarding users with private tags all the times (most users do not need those, and that is why we did not have this implemented for long).
Your second option is the ideal scenario. That is how the meta-extraction module handles its extraction. It uses a featureset. See https://github.com/Emory-HITI/Niffler/blob/dev/modules/meta-extraction/conf/featureset.txt
Ideally, we should have both options for the png-extraction. When the featureset is present, get only those listed fields. Otherwise, get everything (as it is now, but without the private tags for now).

Thank you for the answer. I will modify the code such that both the options are available for the user.

Hy @pradeeban , I have solved this issue.  Please check it.

@Nitesh639 I have requested changes to your pull request.

@pradeeban I make changes. Check now.

Fixed by @Nitesh639 in Niffler-0.8.5.


feat: reformat human readable output for test results

What does this PR do?
Adds a verbose concise parameter to the HumanReporter format function. If true, then display all test results (current behavior). If false, only display failing tests. I have left the default value as true false to preserve existing behavior, with the plan to add a --verbose --concise flag to the sf apex run test command to control this behavior.
I have also moved the summary section for test results to the bottom of the output. This makes it much easier to tell at a glance whether your tests ran successfully or not. Even with the removal of passing tests in the output, it can still be cumbersome to scroll through the results to see the summary.
I primarily have the sf cli in mind but I don't know if this is used anywhere else or if there are other considerations I'm unaware of.
What issues does this PR fix or reference?

https://github.com/forcedotcom/cli/discussions/2872
https://github.com/forcedotcom/salesforcedx-apex/issues/243

Functionality Before
Before, the summary was at the top of the terminal output and it displayed the results of ALL test methods, which required users to scroll a lot.
Functionality After
Now, the summary is at the bottom and can optionally only display failing tests, making it easier to see when read by a human.

@k-capehart We have taken a look at the PR and we really like the notion of displaying the summary last for the human reporter. We would like to move forward with this change with the following recommendations.

make the same change to humanFormatTransform.ts - we have recently added streaming reporters to better handle large test/coverage results.
change the new verbose parameter to concise
to protect backward compatibility concise should default false
human output with concise equal to true will produce in this order

test failures only
no code coverage
summary

human output with concise equal to false will produce in this order

all test results
code coverage
summary

Let us know if you have any questions and we do appreciate the contribution.

@peternhale That's great news! I'll definitely make those changes when I get the time and let you know.
There were some concerns raised here about functionality within the CLI and making the same changes for JSON output. Is that still something I should be considering? https://github.com/forcedotcom/salesforcedx-apex/issues/243#issuecomment-2165342033

@peternhale That's great news! I'll definitely make those changes when I get the time and let you know.
There were some concerns raised here about functionality within the CLI and making the same changes for JSON output. Is that still something I should be considering? #243 (comment)

For this feature, let's restrict the changes to just the human format reporters. All other formatters, include JSON should remain as is.

@peternhale Hello! I made the changes as requested. I didn't see any existing tests for HumanFormatTransform but let me know if I should add some.

@peternhale Hello! I made the changes as requested. I didn't see any existing tests for HumanFormatTransform but let me know if I should add some.

Hmm, my bad. I would be nice to have tests ;), thanks in advance!

@peternhale Will do 👍
Also, I think I missed something anyways. Your message mentions that if concise = true then code coverage shouldn't display. Is that the case even if detailedCoverage = true as well? Which flag takes priority?

@k-capehart detail-coverage flag asks that test results have code coverage applied from the per class codecoverage.
So I would say these two flags are independent, concise only shows failed tests and suppresses "=== Apex Code Coverage by Class." The flag detailed-coverage augments each test with its associated coverage.

@peternhale Thank you, that helps. So just to confirm, using the --concise flag with your last output should look like this:
sf apex run test -c -w5 --detailed-coverage --concise
=== Apex Code Coverage for Test Run<PHONE_NUMBER>sfPP3
TEST NAME                                                 CLASS BEING TESTED    OUTCOME  PERCENT  MESSAGE                                                                            RUNTIME (MS)
────────────────────────────────────────────────────────  ────────────────────  ───────  ───────  ─────────────────────────────────────────────────────────────────────────────────  ────────────
GeocodingServiceTest.successResponse                      GeocodingService      Fail     86%      System.AssertException: Assertion Failed: Assertion failed: I broke it on purpose
                                                                                External entry point
                                                                                Class.GeocodingServiceTest.successResponse: line 35, column 1                      4

=== Test Summary
NAME                 VALUE
───────────────────  ─────────────────────────────
Outcome              Failed
Tests Ran            11
Pass Rate            91%
Fail Rate            9%
Skip Rate            0%
Test Run Id<PHONE_NUMBER>sfPP3
Test Execution Time  2313 ms
Org Id               00D53000001GiuYEAS
Username<EMAIL_ADDRESS>Org Wide Coverage    93%

@peternhale Ok, I think things should be good now. Looks like you already saw my comments but let me know if I missed anything.

@k-capehart Thanks for the updates and we appreciate the contribution. I will try to get to a full review and my testing by COB tomorrow.

Thanks for the help!

@k-capehart I found some time to review and test these changes.
I did find one case that is producing undesired results in HumanReporter. It occurs when running with concise true and no test failures. The output contains undefined in the text results. I am attaching a zip that contains a runTest.ts file along with the text of two runs, with test failures and without test failures.
pr_testing.zip

@peternhale Thanks for catching that. It should be fixed now. I ran your tests and no longer see 'undefined'.

@k-capehart I approved the PR and I will merge soon. Thanks again!
plugin-apex should pick up the change automatically in the coming weeks
FYI @mshanemc

@peternhale Awesome, thanks so much! If you need someone to work on the update for plugin-apex I'll be happy to submit a PR there as well once this update rolls out to it. I'll keep an eye on it.

If you want it sooner than later then all it should require is picking up the version of apex-node.
I am trying to figure out why windows tests are failing for HumanFormatTransform test should format test results and not display coverage results if concise is true, so might be tomorrow sometime for the merge.

@k-capehart I am going to close this PR. I had to fix a formatting error in human transform (that your test found, thanks again). That change and your changes from this PR have been merged in PR #383.
Thanks again for this contribution.

@peternhale Awesome! I'm happy to contribute, thanks for you assistance and support in getting it out.


We should move all molecule- tests to user our runners

Short Description
Save minutes use
Evidence/Steps to Reproduce
No response
Acceptance Criteria/Expected Behaviour
No response
Details
No response
Resolution
No response

@drmatthews should we close this with explanation?

Closing - this repo is public so we would not be able to use the pull_request trigger if we use a self hosted runner.


Add to https://langserver.org/

There's a list here of different language servers, it would be nice to be visible in that list once we're ready.
You can ask to be added by submitting a PR/issue to https://github.com/langserver/langserver.github.io

Sumbitted:

https://github.com/langserver/langserver.github.io/pull/279
https://github.com/microsoft/language-server-protocol/pull/1451

Both are merged now, so closing this.


Can't use :filename: directive after any code

I need to set an environment variable before I include using exec_code. The following works just fine:
.. exec_code::
   :filename: /tmp/file.py

However, any code included before the :filename: directive causes errors:
.. exec_code::

   print('here')
   :filename: /tmp/file.py

ERROR:    print('here')
ERROR:    :filename: /tmp/file.py
ERROR:
ERROR:
ERROR:
ERROR:   File "", line 3
ERROR:     :filename: /tmp/file.py
ERROR:     ^
ERROR: SyntaxError: invalid syntax
Extension error:
Could not execute code!
make: *** [html] Error 2
If I leave out the blank line, the :filename: include runs, but the print('here') line is ignored.
Is there anyway to execute some code before including a file?

It's either file or sphinx code but you can use code Markers to achieve everything.


Modified CMake to use newer directives.

This PR updates the CMakeLists.txt to use newer target_ directives for includes and compile options, which removes some of the cruft associated with inheriting these properties.
The required version of CMake has been updated to 2.8.12 to make use of these newer features. This should be available on all targeted platforms (and specifically, Ubuntu 14.04).
Closes #141

Thanks @jslee02!  Good suggestions.  I think I addressed all your comments.


How to sample latent codes from the W space?

Hi, as mentioned in here and here, we need to randomly sample w vectors in order to learn the interfacegan_direction.
However, I have no idea how to randomly sample the w vectors. Would you kindly tell me how to do it (using some function in this repo probably?)
Thank you very much!

Hi @falloncandra!
You can check out the first 2 lines in the sample_real_and_fake_latents function, which sample a Z vector and pass it into the mapping network to obtain a W space latent code (denoted there as real_w): https://github.com/omertov/encoder4editing/blob/2795aa93714e03ce4d8b70d4d803e4718f5c63d4/training/coach.py#L426
Best,
Omer

Hi, thank you very much for your help! :)

Glad it helped! closing the issue :)


Entities are not usable from event handlers

I have an example application where I define an entity in models. Using from an endpoint works fine, but when I attempt to use it from an event handler, the following error appears:
Unhandled exception: Error: Promise rejected without a handler: Error: unable to construct QueryPlan from an unknown entity name `TopOfBook`
    at file:///worker.js:6:15
    at Object.[Symbol.asyncIterator] (file:///chisel.ts:63:31)
    at AsyncGenerator.next (<anonymous>)
    at Function.findOne (file:///chisel.ts:574:26)
    at Object.book_updates_default [as /dev/book-updates] (file:///dev/events/book-updates:18:31)
    at callEventHandlerImpl (file:///worker.js:194:28)
    at async rollback_on_failure (file:///worker.js:85:16)
    at async handleMsg (file:///worker.js:23:17)
Unhandled exception: Error: Promise rejected without a handler: Error: unable to construct QueryPlan from an unknown entity name `TopOfBook`
    at file:///worker.js:6:15
    at Object.book_updates_default [as /dev/book-updates] (file:///dev/events/book-updates:38:1)

As @honzasp points out, we need to set the apiVersion in requestContext:
diff --git a/api/src/worker.ts b/api/src/worker.ts
index 82a8b681..b720bc92 100644
--- a/api/src/worker.ts
+++ b/api/src/worker.ts
@@ -297,6 +297,8 @@ async function callEventHandlerImpl(
     key: ArrayBuffer,
     value: ArrayBuffer,
 ) {
+    requestContext.apiVersion = apiVersion;
+
     await Deno.core.opAsync("op_chisel_start_event_handler");

     await Deno.core.opAsync("op_chisel_create_transaction");

However, that doesn't fully fix the issue. Now a findOne() call fails with:
Unhandled exception: Error: Promise rejected without a handler: Interrupted: operation canceled
    at file:///worker.js:6:15
    at Object.book_updates_default [as /dev/book-updates] (file:///dev/events/book-updates:22:1)


Tiny refactor on desugar.rs

Changes proposed in this PR:

impl FromIterator<Expr>
make some doubling codes more succinct in desugar.rs

@mtshiba

Thanks!


Blazor async based polling with a timer

Describe the solution you'd like
So I have the following service interface:
interface NewsService {  Task<NewsUpdate> GetNews(DateTime lastUpdate); }
NewsUpdate contains a timestamp and an array of new News to be shown. I want to poll this service frequently for updates with a timer. This works as long you don't do It asynchronously, however System.Threading.Timer doesn't support async/await. So I would like some Timer which support async/await properly.
Additional context
Using async void can cause unwanted behavior because the timer would not wait until one pass finished but instead spawn multiple tasks which can result in concurrency issues.

Thanks for contacting us, @JonasZ95.
We have a sample showing how to do something similar here: https://docs.microsoft.com/en-us/aspnet/core/blazor/components?view=aspnetcore-3.0#invoke-component-methods-externally-to-update-state

@mkArtakMSFT I don't see how this can solve the concurrency problems which is caused by the nature of the state machines which are generated for async functions.
Just look at this example:
@using System.Threading
@implements IDisposable

Counter

Current count: @_currentCount

@code {

    Timer _updateTimer;
    int _currentCount = 0;

    public void Dispose()
    {
        _updateTimer.Dispose();
    }

    protected override async Task OnInitializedAsync()
    {
        _updateTimer = new Timer(state => { InvokeAsync(UpdateCounter); }, null, 0, 100);
    }

    async Task UpdateCounter()
    {
        _currentCount = 0;
        await Task.Delay(500);
        _currentCount++;
        await Task.Delay(500);
        _currentCount++;
        StateHasChanged();
    }
}

The key problem is that a new task is created while the previous task is stil running.

Just because it's created doesn't mean it has run. Calling InvokeAsync essentially queue's a task to be run on a specific thread, in this case Blazor's rendering thread. Meaning, any render operation will occur in the order in which it is enqueued.
If you're worried about not awaiting your InvokeAsync you can mark your timer method as async (yes it'll be async void); events marked as async void are a supported pattern. That being said you'll still have to worry about your timer firing twice in parallel. That can be protected against in many different ways 😄

@NTaylorMullen thanks for clarifying for the way how InvokeAsync works internally sadly there was close to no information regarding this. However, I already assumed that a Queue was used like this and I created the example under this assumption. Actually I think we don't need to discuss about InvokeAsync vs async void as long as the task plans to update the state.
Could you show one of the 'many different ways' to protect the timer firing twice? I tried this and I think It's kinda difficult to do this correctly.  Also, using the term 'parallel' is completely wrong here.


12/hardcode frameworks

Hardcoding embedded frameworks to facilitate building the starter kit.

@glennrfisher ah my bad.... fixed!


ReferenceError: document is not defined

The first string is being executed immediately (on script load) and it throws and error on SSR mode:
ReferenceError: document is not defined

I have a similar problem in Nuxt 3. Every element that has the v-click-away directive needs to be wrapped in a <ClientOnly> tag, otherwise it breaks on SSR. The directive should probably check if window/document is available.


Bring in MbedTls certificate verification

Add verification of x509 certificate chain.
This verification lacks CRL processing.

Motivation

Current dependencies on/for this PR:

main

PR #83

PR #84

PR #85

PR #86

PR #87

PR #48

PR #89

PR #90

@nick-mobilecoin started a stack merge that includes this pull request via Graphite.

@nick-mobilecoin merged this pull request with Graphite.


QEMU crashes with assertion when running macOS guest with Intel Hypervisor

When attempting to run a macOS Monterey Beta VM in UTM on an Intel Mac, QEMU crashes with the message:
Assertion failed: (vector < nr_vectors), function msi_notify, file ../hw/pci/msi.c, line 317.
After searching online I can not find anyone with this error before. I have tried to change my configuration to use only one core, at no avail (maybe vectors has to do with the number of cores).
Configuration

UTM Version: 2.1.2 (1)
Compiled myself from 77d6ca1
OS Version: macOS 11.3.1 (20E241)
Intel or Apple Silicon? Intel

Debug log
debug.log
VM Config
config.plist.zip
More info:
As you can see, the VM uses custom EFI BIOS and OpenCore to get macOS to boot. Here's a screenshot of the verbose macOS boot right before the crash (only QEMU crashes, not UTM):

Update: I have discovered a workaround for this issue. By changing the configuration to disable the HD Audio and USB Redirect features, the error goes away.

The issue seems to be the USB 3 controller (added by using USB passthrough of a USB 3 device) and its MSI setting.
I added this to my QEMU command line to disable MSI and the assert went away, and my USB 3 card reader functions correctly:
-global nec-usb-xhci.msi=off

@thenickdude Thanks! Do you know if it is safe to add this MSI setting as a default in UTM?

I think for non-macOS guests this will cause a performance reduction for USB 3 devices (as non-MSI interrupts have higher overhead)
For macOS guests this should be perfect

I think this is a bug in QEMU and should be fixed in QEMU rather than hacking in a config to disable MSI.

It could, just with a performance hit to USB passthrough, according to @thenickdude. What do you think about a "quirks" tab instead of adding this as a default? https://github.com/utmapp/UTM/pull/3031#discussion_r696593486

Often virtual machine software rolls quirks like this into a set that is implicitly activated by telling it what guest OS you're running (e.g. Workstation, VirtualBox and Proxmox all do this).

@conath So for a while we’ve used qemu-xhci which has MSI=0 by default. However I think that caused issues with Windows (#3194) so I changed it back to nec-usb-xhci. However I’m wondering if maybe QEMU 6.2.0 has fixed this issue so can you re-test after 8fb5b5d2fa7c04f137eff7a4ad42d879772b7cc3?

-global nec-usb-xhci.msi=off

I think this is a bug in QEMU and should be fixed in QEMU rather than hacking in a config to disable MSI. Has this been reported to QEMU?
In the meantime, -global nec-usb-xhci.msi=off can be added to custom arguments as a workaround right?

Thanks.This tweak works in pve 8.1.3 with hacintosh sonoma


Error on VirtualBox image

When running :
d2vm build -p MyP4Ssw0rd -f ubuntu.Dockerfile -o ubuntu.vdi .

Building docker image from ubuntu.Dockerfile
Inspecting image d2vm-4dca598b-8f4d-4c6e-b6c2-c36d145dcb2a
No network manager specified, using distribution defaults: netplan
Docker image based on Ubuntu 22.04.1 LTS (Jammy Jellyfish)
Building kernel enabled image
Creating vm image
Creating raw image
Mounting raw image
Build failed
Unmounting raw image
Failed to unmount
Error: losetup --show -f /tmp/d2vm/834629c7-f718-4d96-bbea-c7890c7a1a09/disk0.d2vm.raw: stdout:  stderr: losetup: cannot find an unused loop device
 error: exit status 1

I am running this inside an Ubuntu VirtualBox with a Windows host

The relevant error is:
losetup: cannot find an unused loop device

Can you list the loop devices inside your vm ?
sudo losetup -a

If there is none, can you ensure the kernel module is available ?
lsmod | grep loop

and enable it:
modprobe loop

Ah... you have to use sudo to build the image...

Ah... you have to use sudo to build the image...

Ahh right!
I didn't because when using sudo: Command 'd2vm' not found
I installed it through  brew install linka-cloud/tap/d2vm

I always forget... since ubuntu 20.04 the user's $PATH is not preserved by sudo, so you need to run with the -E flag:
sudo -E d2vm ...

hey yeah I tried but same message
sudo: d2vm: command not found

what is the output of which d2vm ?

what is the output of which d2vm ?

/home/linuxbrew/.linuxbrew/bin/d2vm

can you show me your $PATH ?
echo $PATH

/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin

Can you try sudo env "PATH=$PATH" d2vm ... ?

That worked and the build finished successfully, thank you!

You're welcome !


2024 10 07

julia v1.11 upgrades, 2dcal completing

format suggestions is tripping?

This is an issue with the formatter on Korg as well.  I think it might have something to do with normalizing line endings.  When I fix it in Korg I'll fix it here.


(ios): Document WKUrlSchemeTask

This mentions just the method we added in 6.2.0 to hook into the WKWebView loading for plugin developers. Plugin developers might find it and come up with intersting solutions.
Do we want to document this more? Do we want examples? Is it worth mentioning at all?
Closes #1149

Do we want examples?

I think a small code snippet to serve as an example would be beneficial.


Unable to retrieve projects with tags

How to reproduce the behaviour

I'm unable to retrieve any projects that have tags associated using doccano-client. To reproduce:

create a Doccano project
Select the project in the Doccano UI and go to Settings via the lefthand menu
add any number of tags
Take note of the project's ID in the project's URL
Execute the following code

from doccano_client import DoccanoClient

# Replace below w/ your details
host = "https://doccano.instance.url"
user = "username"
pswd = "password"
project_id = 1

client = DoccanoClient(host)
client.login(user, pswd)

# Getting by ID produces error
client.find_project_by_id(project_id)

# Getting all projects also produces error
list(client.list_projects())

Traceback:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3444, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-10-56bd588657fc>", line 1, in <module>
    for project in projects:
  File "/usr/local/lib/python3.9/site-packages/doccano_client/client.py", line 289, in list_projects
    yield from self.project.list()
  File "/usr/local/lib/python3.9/site-packages/doccano_client/usecase/project.py", line 40, in list
    yield from self._repository.list()
  File "/usr/local/lib/python3.9/site-packages/doccano_client/repositories/project.py", line 38, in list
    yield Project.parse_obj(project)
  File "pydantic/main.py", line 526, in pydantic.main.BaseModel.parse_obj
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for Project
tags -> 0
  str type expected (type=type_error.str)

Your Environment

Operating System: macOS Ventura 13.3.1
Python Version: 3.9.6
Package Version: 1.2.7

Quick update - upon digging I believe I've found the core issue. The Project class lists the tags property type as a List[str] but the response JSON retrieved from the API call made under the hood has the value of the tags key as a list of dicts with str keys and values that are either int or str, for example:
[{'id': 4, 'project': 1, 'text': 'test1'},
  {'id': 5, 'project': 1, 'text': 'test2'},
  {'id': 6, 'project': 1, 'text': 'test3'}]

This causes an issue during the call to Project.parse_obj() happening in both ProjectRepository.list() and ProjectRepository.find_by_id() methods. If the tags key in the response JSON is updated to just a list of values extracted from the text field of the tags list of dicts (e.g. project_json['tags'] = [tag.get("text") for tag in project_json.get("tags")]), then the Project.parse_obj() call is able to succeed without error. I'm happy to put in a PR for this change if desired, but am unsure if this would be adequate or if a change to the API itself to return tags in the expected format would be more desirable - let me know. Thanks!

Update: the create_project() method is also impacted and I'm unable to create a project with tags. Documentation states tags should be List[str] but attempting to create a project this way returns an error. To reproduce:
from doccano_client import DoccanoClient

# Replace below w/ your details
host = "https://doccano.instance.url"
user = "username"
pswd = "password"
project_id = 1

client = DoccanoClient(host)
client.login(user, pswd)

client.create_project("test create with tags", project_type="ImageClassification", description="test create project with tags", tags=["testTag1", "testTag2"])

This results in the following traceback:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/doccano_client/repositories/base.py", line 24, in verbose_raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://doccano.instance.url/v1/projects
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3444, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-39-3f970c629f42>", line 1, in <module>
    client.create_project("test create with tags", project_type="ImageClassification", description="test create project with tags", tags=["testTag1", "testTag2"])
  File "/usr/local/lib/python3.9/site-packages/doccano_client/client.py", line 383, in create_project
    return self.project.create(
  File "/usr/local/lib/python3.9/site-packages/doccano_client/usecase/project.py", line 87, in create
    return self._repository.create(project)
  File "/usr/local/lib/python3.9/site-packages/doccano_client/repositories/project.py", line 54, in create
    response = self._client.post("projects", json=project.dict(exclude={"id"}))
  File "/usr/local/lib/python3.9/site-packages/doccano_client/repositories/base.py", line 124, in post
    verbose_raise_for_status(response)
  File "/usr/local/lib/python3.9/site-packages/doccano_client/repositories/base.py", line 26, in verbose_raise_for_status
    raise DoccanoAPIError(str(err), err.response)
doccano_client.exceptions.DoccanoAPIError: {'tags': [{'non_field_errors': ['Invalid data. Expected a dictionary, but got str.']}, {'non_field_errors': ['Invalid data. Expected a dictionary, but got str.']}]}

This error response suggests that the create API is expecting a List[Dict[str, Any] like is returned by the API rather than List[str] as documentation and the properties of the Project class suggest. Running w/ this assumption, I tried creating this time w/ tags as List[Dict[str, Any]:
client.create_project("test create with tags", project_type="ImageClassification", description="test create project with tags", tags=[{"text": "testTag1"}, {"text": "testTag2"}])

but this brings us back to the same pydantic.error_wrappers.ValidationError, this time caused by ProjectUseCase.create() due to the attempt to pass the List[Dict[str, Any]] type in for tags in the Project constructor, which expects List[str], and this happens before the API call is actually made. Traceback:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3444, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-40-52ed3f58c976>", line 1, in <module>
    client.create_project("test create with tags", project_type="ImageClassification", description="test create project with tags", tags=[{"text": "testTag1"}, {"text": "testTag2"}])
  File "/usr/local/lib/python3.9/site-packages/doccano_client/client.py", line 383, in create_project
    return self.project.create(
  File "/usr/local/lib/python3.9/site-packages/doccano_client/usecase/project.py", line 74, in create
    project = Project(
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 2 validation errors for Project
tags -> 0
  str type expected (type=type_error.str)
tags -> 1
  str type expected (type=type_error.str)

I've confirmed passing a List[Dict[str, Any]] would work at the API level by doing the following:
import requests

# Replace below w/ your details
host = "https://doccano.instance.url"
user = "username"
pswd = "password"

# setting session as in BaseRepository (https://github.com/doccano/doccano-client/blob/master/doccano_client/repositories/base.py#L30)
host = host.rstrip("/")
_session = requests.Session()
headers = {
    "content-type": "application/json",
    "accept": "application/json",
    "referer": host,
}
_session.headers.update(headers)
api_url = host + "/v1"
_session.verify = True
# authenticate
_session.post(api_url + "/auth/login/", json={"username": user, "password": pswd})
_session.headers.update({"X-CSRFToken": _session.cookies.get("csrftoken")})

project_properties = {
    "name": "test create with tags",
    "project_type": "ImageClassification",
    "description": "test create project with tags",
    "tags": [{"text": "testTag1"}, {"text": "testTag2"}],  # List[Dict[str, str]]
    'resourcetype': 'ImageClassificationProject'
}
c_api_resp = _session.post(api_url + "/projects", json=project_properties)

This works and the project is created as expected, confirming that the issue is a disconnect between what the API expects/returns and the class definition of a Project in the doccano-client

Thank you for your detailed analysis.


fix data transfer bottlenecks

fix enough bottlenecks for async dispatch to be a bottleneck.
Start server with profiler
$ nim c --debugger:native --threads:off -d:danger -d:useMalloc --mm:refc -o:bin/localServer examples/localServer.nim && valgrind --tool=callgrind -v ./bin/localServer

then send data to a single stream
nim c -r -d:release examples/dataStream.nim

current speed is 10MB/s
removing the control flow improves things 10x, speeds goes up to 100MB/s
replacing the add copy (search XXX x_x) by memCopy makes speed up to 150MB/s
then it's probably creating a single frame per stream and reusing it that may improve something. There is a los of mem set and mem copys at this point.

profiling showed SSL stuff as the next bottleneck. I tried removing all SSL related code, and the next bottleneck is the add copy, I changed that to use moveMem that speeds up to 370MB/s.

I tried yasync and got a 2x speed up when running h2load with a high number of streams.
changes are here https://github.com/nitely/nim-hyperx/compare/master...futurevar
load test command ./h2load -n100000 -c10 -m1000 -t2 https://<IP_ADDRESS>:4443

I got asyncdispatch on ORC to reach 430MB/s by increasing the socket buffer from 8KB to 64KB.
On refc it reaches +500MB/s.
https://github.com/nitely/nim-hyperx/compare/asyncsockssl?expand=1

I removed the queue for writes, and the send lock. Checking the code it seems safe to make concurrent send calls, and at least POSIX allows it at the OS level. It's 2-3x faster on h2load bench.
I think a queue will block user code less as long as it's not full, but thinking about it, if the user code running between sends is really fast it'll eventually fill the queue and block, and if it's too slow then it does not matter much. The more streams the more likely this is to occur. There may be a bench that shows code that runs exactly the right amount for a queue to help, but it seems artificial.

I added flow-control for streams back (#12), so data transfer is back to 10MB/s. Increasing the window size to 256KB seems to increase it to ~200MB/s. But cannot do that until flow-control for the connection is implemented.

bench for the yasync branch on my machine with 4 server instances:
$ ./h2load -n1000000 -c10 -m10 -ph2c http://<IP_ADDRESS>:8783
finished in 5.45s, 183437.80 req/s, 1.75MB/s
                     min         max         mean         sd        +/- sd
time for request:       91us      4.54ms       364us       161us    79.21%

$ oha-linux-amd64 -z 10s -c 10 -p 10 --latency-correction --disable-keepalive --http2 https://<IP_ADDRESS>:8783

Show output
Summary:
  Success rate:	100.00%
  Total:	10.0004 secs
  Slowest:	0.0068 secs
  Fastest:	0.0001 secs
  Average:	0.0009 secs
  Requests/sec:<PHONE_NUMBER>

  Total data:	0 B
  Size/request:	0 B
  Size/sec:	0 B

Response time histogram:
  0.000 [1]      |
  0.001 [410525] |■■■■■■■■■■■■■■■■■■■■■
  0.001 [623506] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.002 [73436]  |■■■
  0.003 [5542]   |
  0.003 [661]    |
  0.004 [281]    |
  0.005 [85]     |
  0.005 [24]     |
  0.006 [13]     |
  0.007 [3]      |

Server: refc, ssl = false, 4 instances, no data
  Success rate:	100.00%
  Total:	10.0025 secs
  Slowest:	0.0111 secs
  Fastest:	0.0001 secs
  Average:	0.0012 secs
  Requests/sec:	81802.0800

  Total data:	9.36 MiB
  Size/request:	12 B
  Size/sec:	958.50 KiB

Response time histogram:
  0.000 [1]      |
  0.001 [520063] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.002 [266810] |■■■■■■■■■■■■■■■■
  0.003 [29471]  |■
  0.005 [1350]   |
  0.006 [283]    |
  0.007 [69]     |
  0.008 [29]     |
  0.009 [43]     |
  0.010 [8]      |
  0.011 [1]      |

same but send "hello world" data


added code to html file

This PR adds all of the HTML we need for the prework study guide application.

Completed an HTML file.
Created a Git feature branch.
Opened a pull request in GitHub.
Merged a pull request in GitHub.


tracker,quorum: use native Sort

The insertion sort is no longer needed, since Go 1.21 introduced a generic slices.Sort algorithm which achieves the same effect: it doesn't incur allocations, and uses insertion sort for slices up to 12 elements.

Note: This PR will require the applications, which depend on raft, to use a go version >= 1.21. It shouldn't be a problem, because go1.20 is already out of support. But we should document the requirement in readme.

go.mod already states 1.21, plus some other 1.21+ things are already used (e.g. #135). Seems like this PR isn't the best place to update documentation.

go.mod already states 1.21

It doesn't technically prevent applications, which depend on raft, from using a old go version.

some other 1.21+ things are already used (e.g. #135). Seems like this PR isn't the best place to update documentation.

OK. We can take care of the document separately.


VM compiler's range analysis needs to be adjusted for int64 ints

Current range analysis does not account for overflows. Given a >= 0 && b >= 0 it would infer that a + b >= 0.
I could not immediately make it crash or generate wrong code (due to some phase ordering issue - I think), but you can see the problem if you look at the graph for a function like:
const NeverInline = 'NeverInline';

@NeverInline
bool foofoo(int b) {
  for (int i = 0x7ffffffffffffffc; i <= b; i += 2) {
    if (i < 0) {
      return true;
    }
  }
  return false;
}

main () {
  for (var i = 0; i < 10000; i++) {
    foofoo(0x7fffffffffffffff);
  }
  print(foofoo(0x7fffffffffffffff));
}

this yields:
 12: B7[join]:38 pred(B1, B6) {
      v4 <- phi(v3, v28 T{Type: class '_int64@0150898'}) alive [9223372036854775804,<PHONE_NUMBER>854775807] T{Type: class '_int64@0150898'}
}

This range is obviously wrong - but luckily we don't optimize away anything based on this range.
/cc @alexmarkov @mkustermann
[It is a known problem but we still somehow forgot about it.]

The meeting today inspired me to make a reproduction - the boxing is affected by range information, so if we compute incorrect range we compile boxing incorrectly:
 const NeverInline = 'NeverInline';

 @NeverInline
 int foofoo(int b) {
   for (int i = 0x7ffffffffffffffc; i <= b; i += 2) {
     if (i < 0) {
       return i - 0x4000000000000000;
     }
   }
   return 0;
 }

 main () {
   for (var i = 0; i < 10000; i++) {
     foofoo(0x7fffffffffffffff);
   }
   print(foofoo(0x7fffffffffffffff));
 }

╭─ ~/src/dart/sdk ‹suppress-error-about-abstract-super›
╰─$ pkg/vm/tool/dart2 --optimization-counter-threshold=-1 --no-background-compilation --enable-inlining-annotations t/range.dart
4611686018427387904
╭─ ~/src/dart/sdk ‹suppress-error-about-abstract-super›
╰─$ pkg/vm/tool/dart2 --no-background-compilation --enable-inlining-annotations t/range.dart
-4611686018427387904

Guess this issue can be closed now.


Include all loader errors when failing

Closes #2736.
Include all errors from the loader transcript when looking for missing DLLs. This will help users to analyze the problem, as this information can include missing symbols, etc.

Codecov Report

Merging #3509 (e7217a9) into main (c7a9827) will decrease coverage by 8.52%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #3509      +/-   ##
==========================================
- Coverage   39.47%   30.96%   -8.52%
==========================================
  Files         299      135     -164
  Lines       36314    14602   -21712
==========================================
- Hits        14336     4521    -9815
+ Misses      21978    10081   -11897

see 164 files with indirect coverage changes


Drop hash fragment when downloading from registry

Summary
When pip does an HTTP download, it turns out that it clears the fragment: https://github.com/pypa/pip/blob/0ad4c94be74cc24874c6feb5bb3c2152c398a18e/src/pip/_internal/network/download.py#L115C5-L115C23. Including the fragment may be breaking some custom indexes.
Closes https://github.com/astral-sh/uv/issues/2099

@hmc-cs-mdrissi -- are you interested in testing this branch?

@pradyunsg -- sorry to bother, it was pointed out to me that pip strips fragments when making HTTP requests. Do you know if this is encoded in any spec?

Yes I am happy to. First time building rust codebase and fairly smooth experience.
It sadly failed though,
Verbose Logs
     Running `target/debug/uv --verbose pip install '--index-url=https://AUTH@index/python/virtual/' internal-lib`
 uv::requirements::from_source source=internal-lib
    0.001651s DEBUG uv_interpreter::python_environment Found a virtualenv through VIRTUAL_ENV at: /Users/pa-loaner/Snapchat/Dev/.venvs/bento_uv2
    0.002148s DEBUG uv_interpreter::interpreter Cached interpreter info for Python 3.9.16, skipping probing: /Users/pa-loaner/Snapchat/Dev/.venvs/bento_uv2/bin/python
    0.002237s DEBUG uv::commands::pip_install Using Python 3.9.16 environment at /Users/pa-loaner/Snapchat/Dev/.venvs/bento_uv2/bin/python
    0.010349s DEBUG uv_client::registry_client Using registry request timeout of 300s
 uv_client::flat_index::from_entries
 uv_resolver::resolver::solve
      0.306961s   0ms DEBUG uv_resolver::resolver Solving with target Python version 3.9.16
   uv_resolver::resolver::choose_version package=root
   uv_resolver::resolver::get_dependencies package=root, version=0a0.dev0
        0.307338s   0ms DEBUG uv_resolver::resolver Adding direct dependency: internal-lib*
   uv_resolver::resolver::choose_version package=internal-lib
     uv_resolver::resolver::package_wait package_name=internal-lib
 uv_resolver::resolver::process_request request=Versions internal-lib
   uv_client::registry_client::simple_api package=internal-lib
     uv_client::cached_client::get_cacheable
       uv_client::cached_client::read_and_parse_cache file=/Users/pa-loaner/Library/Caches/uv/simple-v3/28d1ce4468d5ce6b/internal-lib.rkyv
 uv_resolver::resolver::process_request request=Prefetch internal-lib *
 uv_client::cached_client::from_path_sync path="/Users/pa-loaner/Library/Caches/uv/simple-v3/28d1ce4468d5ce6b/internal-lib.rkyv"
          0.325625s  17ms DEBUG uv_client::cached_client Found fresh response for: https://index/python/virtual/internal-lib/
   uv_resolver::version_map::from_metadata
   uv_distribution::distribution_database::get_or_build_wheel_metadata dist=internal-lib==0.1.602402
     uv_client::registry_client::wheel_metadata built_dist=internal-lib==0.1.602402
       uv_client::cached_client::get_serde
         uv_client::cached_client::get_cacheable
           uv_client::cached_client::read_and_parse_cache file=/Users/pa-loaner/Library/Caches/uv/wheels-v0/index/28d1ce4468d5ce6b/internal-lib/internal-lib-0.1.602402-py3-none-any.msgpack
        0.618904s 311ms DEBUG uv_resolver::resolver Searching for a compatible version of internal-lib (*)
 uv_client::cached_client::from_path_sync path="/Users/pa-loaner/Library/Caches/uv/wheels-v0/index/28d1ce4468d5ce6b/pb-internal-lib/internal-lib-0.1.602402-py3-none-any.msgpack"
        0.619005s 311ms DEBUG uv_resolver::resolver Selecting: internal-lib==0.1.602402 (internal-lib-0.1.602402-py3-none-any.whl)
   uv_resolver::resolver::get_dependencies package=internal-lib, version=0.1.602402
     uv_resolver::resolver::distributions_wait package_id=internal-lib-0.1.602402
              0.619291s   0ms DEBUG uv_client::cached_client No cache entry for: index/python/virtual/internal-lib/0.1.602402/internal-lib-0.1.602402-py3-none-any.whl
           uv_client::cached_client::fresh_request url="index/python/virtual/internal-lib/0.1.602402/internal-lib-0.1.602402-py3-none-any.whl"
error: Failed to download: internal-lib==0.1.602402
  Caused by: HTTP status client error (404 Not Found) for url (index/python/virtual/internal-lib/0.1.602402/internal-lib-0.1.602402-py3-none-any.whl)

cargo run -- --verbose pip install --index-url=$INDEX_URL internal-lib being exact command I used, while pip install --index-url=$INDEX_URL works fine. Unsure what remaining differences if any exist in request uv sends vs pip sends especially as uv is able to get index page successfully so I know authentication works for some requests.

Thank you! That's a bummer. So it's an identical URL to what pip is requesting, right? My only remaining guess then is that we're losing the auth somewhere, and the server is responding with a 404.

Yes the url is identical to pip based on breakpointing. Auth headers is my best guess and the headers being used for some requests (index fetch vs wheel fetch) are inconsistent. I see pip has PipSession object that manages auth and the wheel fetch does store the needed authentication tokens.

If we could log the headers in error message used along with url I can compare exact headers pip uses vs uv.

Are the two URLs (for index fetch vs. wheel fetch) on different domains, or do they use different schemes, or anything like that?

The first url is,
https://registry.company-dev.net/python/virtual/lib-name/
The second url is,
https://registry.company-dev.net/python/virtual/lib-name/0.1.602402/lib-name-0.1.602402-py3-none-any.whl
So the url is exact same. I may take a look tomorrow at uv closer, just not confident as it'll be my first time reading rust code and figuring out how to use a rust debugger.

Okay, thanks. And how do you typically provide the credentials? (We have this safe_copy_url_auth method that copies the credentials from one URL to the next if they are on the same "realm", which is why I asked about the domains.)

The credentials are contained within the index-url. Not sure you saw the edit I made to show the index url.

Okay cool, yeah, I would expect this to work. I can push a branch that adds more logging for the headers and such tomorrow if you'd like. Would love to get this working for you since if you're running into this issue, I'm sure you won't be the only one.

A few other things is I am able to successfully curl the wheel fetch url with a simple command like,
curl -v -H "authorization: basic `echo -n "LCAv1:AUTH_TOKEN" | base64` https://registry.company-dev.net/python/virtual/lib-name/0.1.602402/lib-name-0.1.602402-py3-none-any.whl

So I think the only important thing is header being used and that index fetching successfully finding latest version feels like a sign that uv has the right authorization header for first fetch. The pip authorization header looks same as one in that curl command.

@hmc-cs-mdrissi - What kind of logging would be most useful? Like, all headers?

Yes headers for each request sent. Most important two requests being index fetch + wheel fetch.
Alternatively if you could point me to the code where these requests are sent I could try to do some print debugging

Do you know if this is encoded in any spec?

I just saw this.
I don't think it's written in any packaging spec but I'm pretty sure the hash fragment has to be dropped when querying the server since that's how URLs work? The fragment part is usually meant to reference something within the resource, eg anchors on HTML pages, and isn't sent to the server.
From https://url.spec.whatwg.org/#concept-url-fragment:

A URL’s fragment is either null or an ASCII string that can be used for further processing on the resource the URL’s other components identify.

I don't think I've ever seen someone send an HTTP request with # fragment in the path segment, and I'm pretty sure it'll fail for most servers.
I'm actually surprised if PyPI actually responds to a malformed request like that, if that is indeed what y'all are doing.

Honestly very possible that reqwest is dropping the fragments already.

Ah, yea, that makes sense! It would be a bit of a surprise if it wasn't. 😅


No GovCloud region ARNs present in worker nodes Yaml

There are no Gov cloud regions mentioned.

https://github.com/aws-samples/opensource-4g-core-eks/blob/56ad4ac606d108223911f73c2b28f61b45f29eb0/x86-Architecture/cfn-templates/open5gs-worker-x86.yaml#L11-L41

Also see an error with the SAM:
Transform AWS::Serverless-2016-10-31 failed with: Invalid Serverless Application Specification document. Number of errors found: 1. Resource with id [ConfigMapUpdate] is invalid. User: arn:aws-us-gov:iam::xxxxxxxxxx:user/<Redacted> is not authorized to perform: serverlessrepo:CreateCloudFormationTemplate on resource: arn:aws:serverlessrepo:us-east-1:903779448426:applications/eks-auth-update-hook. Rollback requested by user.

This can be because of the following reason
[] AWS Serverless Application Repository - How AWS Serverless Application Repository Differs for AWS GovCloud (US) - https://docs.aws.amazon.com/govcloud-us/latest/UserGuide/govcloud-sar.html#govcloud-sar-diffs
> Applications that are publicly shared in other AWS Regions are not automatically available in AWS GovCloud (US) Regions. To make applications available in AWS GovCloud (US) Regions, you must publish and share them independently of other AWS Regions.
https://github.com/aws-samples/opensource-4g-core-eks/blob/56ad4ac606d108223911f73c2b28f61b45f29eb0/x86-Architecture/cfn-templates/open5gs-worker-x86.yaml#L770-L788

Hi @kakarotbyte, simple fix of this issue that removing out ConfigMapUpdate custom resource from the CloudFormation. It is purely optional steps to make aws-auth-cm.yaml update to be done automatically. As mentioned in the blogpost (of below) and also illustrated in below UG, you can update manually aws-auth-cm.yaml so that wordernode group can be added to the cluster w/o this SAR application.
https://docs.aws.amazon.com/eks/latest/userguide/launch-workers.html
...
Run the CloudFormation for worker node group creation (open5gs-worker.yaml)
...
Optional: If worker node groups don’t get joined to the EKS cluster, then manually update aws-auth-cm.yaml ConfigMap so that the EKS cluster control plane can register the worker nodes. (This step is usually not required if a step of Bastion Host Configuration-4 has been done properly.)


The properties language uses a Java icon in the Seti theme

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.70.0
OS Version: Any

Generic properties files are assigned the Java icon in the file explorer, when they should have a generic cogwheel icon.
The wrong behavior can be observed in VS Code by manually setting the file language for a plain text file to "properties":

Steps to Reproduce:

Disable all extensions
Open any file
Invoke the "Change Language Mode" command
Find "Properties" in the list
Expecting the cogwheel used for files like .editorconfig, but finds the Java icon, see screenshot.

Thanks for creating this issue! It looks like you may be using an old version of VS Code, the latest stable release is 1.70.2. Please try upgrading to the latest version and checking whether this issue remains.
Happy Coding!


[1.10.2] Protection duplication bug.

Explained it with images here: http://imgur.com/a/N5Rq3
Original SF Issue: #773

This bug is totally gamebreaking, any news @kashike?

@LexManos with the latest changes, did anything change?

Well, I'm Not lex, but will give it a try and report back in a few minutes.

No, keeps happening.


Limit key range of layers generate by compaction to 16k relations to separate rel/non-rel and sys/user relation entries

refer #2948

Also https://github.com/neondatabase/neon/pull/2563 doesn't have this problem, as an alternative solution

Ok so we might get one small L1 per batch (in the non-rel part of the key space), but that doesn't seem like a problem on average. Seems worth it.
Have you checked this code has the desired effect? I thought we were changing L1 layer bounds, but I see changes to repartitioning code, which is only used in image layer generation (i think). https://github.com/neondatabase/neon/blob/87c3e55449320f2816bdbfafe18b4e5643152ece/pageserver/src/tenant/timeline.rs#L2529

Have you checked this code has the desired effect? I thought we were changing L1 layer bounds, but I see changes to repartitioning code, which is only used in image layer generation (i think).

As far as I understand image generation for the whole database happens because of layers with larger key range generated by recompaction. Only htis layers are taken in account when we check whether to perform image generation or not.
Assume that we just append one table. But such append also affect system tables and SLRUs.
This is why instead of producing small L1 layer which will contain only range of the appended blocks [first-appended-block..last-appended-block], we will give two "wide" delta layers: [MIN_KEY...first-appeded-block], [last_appended_block...MAX_KEY].
Repeat it 3 times and we get three delta layers covering most of database key space, which force image layer generation for the whole database and not only for [first-appended-block..last-appended-block].

Yes that's how it works. You misunderstood my question: how does changing KeySpace::partition affect L1 generation? The function is not called at all from compact_level0.
Have you tried running this on pgbench init 10GB and drawing the layers to see that it works? (or any other test)

how does changing KeySpace::partition affect L1 generation? The function is not called at all from compact_level0.

Yeh, you are right. Layer key dimension should be limited not in  KeySpace::partition  but in  compact_level0 functions. Redo it now.

Timeline after pgbench 10gb python test. Doesn't seem fixed. Maybe there are factors other than non-rel files that contribute to extremely sparse L1s.
To make your own graph run ls test_output/test_pgbench\[neon-45-684\]/repo/tenants/b81c45370fa23e38d4f71e5ccd4f1bf8/timelines/6576e861feaaf82aeaae21bee6dd8220 | grep "__" | cargo run --release --bin draw_timeline_dir > out.svg and then firefox out.svg

NOTE: When reading these pictures remember that both the x and y axis are compressed. I only preserve the relative order of any rectangle endpoints without preserving size (otherwise we won't be able to see anything). So when I say a layer is "sparse", i mean "i see that later we have 10 layers in the same key space, so it's at least 10x sparser than those layers"

Sorry, can you explain how to interpret this graph?
What are this empty rectangles represent? L1 lasers?
And "dotted" bold line?

empty rectangle = delta. We have 2 L0s at the top, and below 4 batches of L1 layers.
 We often reimage a big range at once so we get a dotted line of images. They're mostly the same width, but some look wider because of coordinate compression (I only preserve the order of rectangle endpoints, not relative size).
Images sometimes overlap in lsn range with an L1 delta layer because we take images at latest LSN, which includes WAL from L0 and inmem layers, which later disappear.
All layers are taken with ls from the timeline directory directly, so I guess there's no gc kicking in yet.
So the story in this picture is:

Pgbench init creates 3 batches of L1s. For unknown reasons batch 1 and 3 cover an aggressive amount of key space.
All 3 batches cover the middle-ish part of the key range, so that get's covered by images
Simple update phase adds a 4th batch of L1s which (for good reason) covers the entire keyspace
Now the rest of the keyspace gets image layers because L1 batches 1 3 and 4 together cover the entire keyspace

I think the problem is larger and this approach only goes so far. Nonrel pages are one obvious cause of the problem, but even without them the problem exists.
Take for example an average OLTP workload. Let's squint and model this hypothetical workload as follows: Write random pages, with a certain part of the keyspace being a hotspot, and the hotspot moving every few minutes to a different part.
Not only will this workload produce enough sparse deltas to trigger the problem, but actually most deltas will be very sparse and cover the entire keyspace. Only the hotspot will have deltas worth reimaging over, and by definition most of the keyspace is not in a hotspot. If we add layer barriers like you do in this PR, that would only create tiny gaps between the L1 layers and eliminate less than 1% of the images.
We actually need to address the fact that currently the definition of L1 is very broad. We either need to introduce L2, and reimage only after a certain number of L2s (I haven't thought about this, and I'm not sure if just L2 will be enough), or go with my approach (classify layers based on density, as defined by "number of partitions inside the layer"), or something else.

Sorry, I still do not understand how coordinate compression is performed.
So if I have delta images
00_00000000_00000000_00004001_00_000000000 .. 00_00000000_00000000_00004001_00_000000100
00_00000000_00000000_00004002_00_000000000 .. 00_00000000_00000000_00004004_00_000000100
what will be the ratio of length of correspondent rectangles?
If we perform random updates, then not wonder that image layers need to be generated for the whole database range even if percent of updated pages is relatively small. But let's estimate. So L0 layer contains 256Mb wal records.
For simplicity lets assume hat it is FPIs - 8kb (if wa record is smaller, it will only increase my estimation.
So L0 layer with random updates modifies. 32k random pages. Lets multiply it on image layer size: 128MB.
It is 4TB! It means that only for databases with size larger than 4TB delta layers produced by compact_level0 cause generation of
"useless" image layers - image layers which page were not actually changed.
The problem I am trying to address is not related with random/sparse updates - here it is really har to improve something.
But assume that we just append one very larger table (i.e.with 1 TB size). So we add 256Mb of data to this table and it produce 256Mb L0 layers.
The we reshuffle it and ... get delta layer something like:
00_00000000_00000000_00000000_00_000000000 .. 00_00000000_00000000_00004001_00_08008000
So it actually covers all this huge table. There such L1 layers and we will generate 1Tb image layers, while we need actually image layers for 256*3 =  768Mb. Limiting layer key range can avoid such behavior.

what will be the ratio of length of correspondent rectangles?

I take those 4 numbers, sort them, deduplicate them, and replace them with their rank.
So the layers now become:
layer1: 0..1
layer2: 2..3

If there are no other layers, then their width would be the same. But
if there was a third layer 00_00000000_00000000_00004001_00_000000000..00_00000000_00000000_00004001_00_000000001
then we'd have
layer1: 0..2
layer2: 3..4
layer3: 0..1

Here's more on "coordinate compression" (competitive programming slang) https://medium.com/algorithms-digest/coordinate-compression-2fff95326fb I'm sure you'll hear the word again.
So if you're looking at a picture and want to estimate the actual width, maybe count how many image layers it gets covered by later.

The problem I am trying to address is not related with random/sparse updates ...

Yeah we can commit this and solve a special case, but that doesn't close https://github.com/neondatabase/neon/issues/2948

It means that only for databases with size larger than 4TB delta layers produced by compact_level0 cause generation of "useless" image layers

True, to see extreme problems we need extreme scale. But it's fairly common to see mostly useless images at any scale.
Look at this project https://neondb.slack.com/archives/C03UVBDM40J/p1669655812511299
num_images: 24070
num_deltas: 2620

That's an image:delta ratio of 12. Since (TIL) deltas are 2x larger, that means our bloat factor is 7 (1 delta + 6 times it's size in images). IMO we can get the bloat down to 2, and that will make a big difference for pricing.

If we perform random updates, then not wonder that image layers need to be generated for the whole database range even if percent of updated pages is relatively small.

I think it's a solveable problem. We just compact those sparse deltas instead of reimaging them. The cost of compaction is smaller, the effect is the same (to decrease the max diffuculty below 3). We should have a limit on key_range size such that any deltas larger than it should be compacted rather than reimaged.
If we really need to reimage because it's time to GC that sparse delta, then we could (ignoring it is also a fine choice). But if we're reimaging because of GC, then we can immediately delete the old images, so we're not causing any actual pageserver bloat. Only write amplification, s3 bloat. In all cases we've seen so far, pageserver bloat is the problem. Also for total cost of services, pageserver bloat will probably be a bigger problem.
All visualizations I've made so far come directly from pageserver files that haven't been GCed yet. See for example this pathological case https://neondb.slack.com/archives/C03UVBDM40J/p1669736655098039?thread_ts=1669655812.511299&cid=C03UVBDM40J

Hmmm...
First layer contains 0x1000 blocks of relation 4001, second - 3 relations 4002..4004 with at least 0x0x1000 blocks in 4004 and your third just one block of relation 4001. And you say that first layer will be represented by box with width 2  and second and third - with box with length 1? Seems to be either incorrect either confusing.
But in any case as far as this diagram doesn't represent real key range, it seems to be not possible to say just looking at this diagram that some long layer will cause  generation of page images for the whole key range.
Actually the fact that some delta layer at diagram is several times wider than other doesn't indicate some problem - yes it can happen with rando distribution.

Yeah we can commit this and solve a special case, but that doesn't close https://github.com/neondatabase/neon/issues/2948

First of all I do not agree that it is "special case". Append-only tables is quite frequent use case. And ore over almost an import of data to the database corresponds to such use case, so images created during such import will suffer from this problem.
Second - I once again do not completely understand picture at #2984 and it is not clear to me why this PR will not fix this problem.

We just compact those sparse deltas instead of reimaging them.

Let's return to the begging: why do we ever need to produce image layers?

To reduce number of layers which need to e inspected to perform page reconstruction and so reduce reconstruction time.
To be able to perform GC (which we perform at image layer).

With delta layers or with partial image layers  you can not say if layer contains wal records for the particular page or not without fetching this page and performing B-Tree search. We can use bloom filter or whatever else to avoid fetching all layers. But it will require additional memory for layer map. Also precise last written LSN cache can point us to the right LSN and so avoid scanning of large number of layers.
Sorry, but I do not currently understand how you suggestion can address case with rando updates and not sure if something can really be done in this case within current layering model.

But it's fairly common to see mostly useless images at any scale. That's an image:delta ratio of 12.

I believe (can nor prove) that it really caused by "corner effect" - when we create delta layers which covers large range of key space and so require multiple image layers. If we have some test to reproduce such layers layout, it will be quite trivial to test if this patch address this problem or not.

I have performed some experiments.
You are absolutely right - this "trick" with limiting key range is not working well even for single table.
There are several reasons for it: autovacuum, placing forknum after relnum in Key,...
So we really need some more generic algorithm for image layer generation.
I thought about your idea of sparse delta layers.
Actually our delta layers are already sparse: the contains only updates keys.
So we can add more precise check that delta layers really overlaps with image layer (contains some entries in this key range).
I have create new PR  #3348 where I have added Layer::overlaps method and use it in LayerMap::count_deltas.
Certainly it involves disk B-Tree, so requires reading layer data fro the disk. It may increase time of compaction.
Alternatives - bloom filter or BRIN.

And you say that first layer will be represented by box with width 2 and second and third - with box with length 1? Seems to be either incorrect either confusing.

It's not incorrect, just confusing. You're welcome to improve on the visualization method. Without any "coordinate compression" most rectangles will have 0 width and be invisible. Maybe a better method is to just use collect_keyspace and exclude unused keys. I haven't tried that.

But in any case as far as this diagram doesn't represent real key range, it seems to be not possible to say just looking at this diagram that some long layer will cause generation of page images for the whole key range.

Yes. Only after the images are created you can see in retrospect that one delta caused many images. But for some reason I found it hard to reproduce this locally (to get the useless images to actually appear). Maybe it takes too long and the test finishes before? Not sure. But in the specific workload of pgbench init no delta layer will be denser than an image layer, so if a delta spans 10 other deltas it probably spans 20 images too.

You are absolutely right - this "trick" with limiting key range is not working well even for single table. There are several reasons for it: autovacuum, placing forknum after relnum in Key,...

I'm sure there are other cases too. If most updates occasionally update the root of some BTree index, etc. (I don't know pg internals, just guessing that something like that would happen, or at least we can't rule it out)


Node npm 'windows' & 'multiple systems' examples are identical

See https://github.com/actions/cache/blob/main/examples.md#node---npm.
The first example uses ~/.npm for linux setup, fine.
The Windows example uses npm config get cache to find the cache path, and then caches that, rather than using any specific path.
The 'Using multiple systems' example also does exactly the same. In fact, it's character for character identical. It looks like #223 did this as part of fixing the Windows example, but I think it's a bit confusing to have the exact same code twice.
Imo, this would be simpler & clearer all round with just one single example for all 3 cases (i.e. the 'multiple systems' platform-agnostic one).

These is activity here! This'll be fixed once #690 is merged.


support Klusterlet.Spec.Namespace in hosted mode

Signed-off-by: Yang Le<EMAIL_ADDRESS>

/hold Hold until test cases are fixed

/lgtm

/lgtm

/unhold


Stop calling Free on Slot

I've touched base/threading/thread_local.h and content/browser/android/content_view_core_impl.cc.
Then did a merge to c5cf474 and  	f13a5b7. That is why, I guess, it shows many commits here.

It looks like you need to rebase your checkout.

@msisov You should keep your changes separate from the original commits until we do final merge into master so that we know what's changing. You can reference the original commits in your commit messages. Also if they are modifying separate original commits then you should have respective number of commits so it will be easier to merge them later.

check now, please


Create multiple features with different content

Hi,
I would like to create multiple features inside one update site. The platform feature should be the "All" feature containing everything specified with transitive dependencies, additionally there should be a feature without some transitive dependencies.
I thought of something like this (the plugin names are fictive):
platform {
    categoryId = 'fipro'
    categoryName = 'fipro'

    featureId = 'org.fipro.something.feature'
    featureName = 'fipro All'
    featureVersion = "1.0.0"
    featureProvider = 'Dirk Fauth'

    defaultQualifier = ''
    useBndHashQualifiers = false
    useFeatureHashQualifiers = false
    defaultQualifierMap.prefix = ''

	feature(id: 'org.fipro.sub.feature', name: 'fipro without slf4j') {
		plugin "org.fipro:fipro-service:1.0.0", {
			// exclude this transitive dependency
			exclude group: 'org.slf4j', module: 'slf4j-api'
		}
	}

	feature(id: 'org.fipro.something.feature', name: 'fipro All') {
		plugin "org.fipro:fipro-service:1.0.0"
	}
}

This should be possible with p2, but the generated feature.xml files are equal. It looks like the bundles that are configured in the feature.xml are simply a collection of all bundles in the bundles directory. The exclude is not taken into account on creating the xml file, only on downloading.
If you can give us a pointer on how this can be fixed, we could try to provide a pull request.
Greez,
Dirk

The way it works right now is that there is one Gradle dependency configuration that backs the dependencies. The plugin instruction is just delegating to the Gradle mechanism for adding a dependency. That means in this case the dependency is added twice.
The exclude is interpreted by the Gradle dependency mechanism and excludes the module from the configuration.The second time the dependency is added slf4j is added to the configuration (and thus voids the exclude). Later when the feature is assembled, there is no info about the exclude, just everything associated to the plugins included in the feature is added.
A workaround might be manipulating the feature objects before the feature Jars are written in task bundleFeatures. Could work well at least when the bundles to remove don't have transitive dependencies.
Right now I'm afraid I don't have a good idea for a better solution.

It turned out that the issue is
a) generatePlatformFeature is set to true by default
b) one of the features has the same id as the platform feature
This was explained in #65 also.
I therefore close this issue, as by setting generatePlatformFeature to false solves this issue.


Javascript Waypoint 39

Challenge http://freecodecamp.com/challenges/waypoint-bring-your-javascript-slot-machine-to-life has an issue. Please describe how to reproduce it, and include links to screenshots if possible.
Hi!
When I'm working on this way point I can only get as far as entering $($('.slot')[0]).html(slotOne) before it freezes up and crashes. I tried making it work on Safari and Google but they both crashed.

Possible duplicate of #1960

Closing as duplicate


Timing issue when removing an asset from fybrikapplication right before deleting the fybrikapplication

In my testing of creating and using a fybrikapplication, then removing it and doing it again, I found I was hitting a timing window when removing the asset from the fybrikapplication, then deleting the fybrikapplication.  Some sort of timing window during reconcilation, where the reconcilation of removing the asset may not have completed by the time the fybrikapplication is removed, causing the finalizer for the fybrikapplication to get stuck, and leaving the fybrikapplication CR around in kubernetes.  Sometimes the associated plotter/blueprint/module helm install and associated objects are also left behind, but usually they go away fine.
This doesn't happen every time.  In running ~1000 times, it maybe happened around 10-20 times, where I would be left with all these old fybrikapplications that weren't able to finalize the deletion.  Manually removing the finalizer from the application works, or, in the cases where blueprints/plotters/helm installs were left behind, manually deleting those objects helps too.
By adding a 5 second delay in my test script, after removing the asset from the application, and before deleting the fybrikapplication, this problem went away completely, and everything always cleaned up entirely fine.  (Note that while I originally though this might be related to issue #1409, with my 5 second delay added here, #1409 still occurs, so they don't appear related).
This also doesn't happen if you don't bother removing the asset before deleting the fybrikapplication.  In that case, deleting the fybrikapplication causes everything else to clean up fine, and finalization occurs fine as well.  But ideally we should be able to remove the asset first, and the fybrikapplication deletion should be able to fully clean up, even if the asset removal is in progress at that point.
In the manager pod logs, I see these errors when this happens:
{"level":"trace","Controller":"FybrikApplication","fybrikapplication":"fybrik-jlkaus/fybrik-1d99fba5-c82c-4aeb-b86f-658b7e0c023c","app.fybrik.io/app-uuid":"15b03e59-8328-49e2-ad03-43ad1223e714","Action":"Delete","time":"2022-04-05T19:04:54Z","caller":"/workspace/source/mesh-for-data/manager/controllers/app/fybrikapplication_controller.go:293","message":"Reconcile: FybrikApplication is deleting the generated Plotter"}
2022-04-05T19:04:54.332Z        ERROR   controller-runtime.manager.controller.fybrikapplication Reconciler error        {"reconciler group": "app.fybrik.io", "reconciler kind": "FybrikApplication", "name": "fybrik-1d99fba5-c82c-4aeb-b86f-658b7e0c023c", "namespace": "fybrik-jlkaus", "error": "plotters.app.fybrik.io \"fybrik-1d99fba5-c82c-4aeb-b86f-658b7e0c023c-fybrik-jlkaus\" not found"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
        /Projects/Proj1/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.5/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
        /Projects/Proj1/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.5/pkg/internal/controller/controller.go:214

Prior to that, I see messages about deleting the plotter, also hitting an error about the fybrikapplication having been updated in the meantime (probably the deletion of the fybrikapplication?)
{"level":"trace","Controller":"FybrikApplication","fybrikapplication":"fybrik-jlkaus/fybrik-1d99fba5-c82c-4aeb-b86f-658b7e0c023c","app.fybrik.io/app-uuid":"15b03e59-8328-49e2-ad03-43ad1223e714","Action":"Delete","time":"2022-04-05T19:04:53Z","caller":"/workspace/source/mesh-for-data/manager/controllers/app/fybrikapplication_controller.go:293","message":"Reconcile: FybrikApplication is deleting the generated Plotter"}
{"level":"info","Controller":"FybrikApplication","fybrikapplication":"fybrik-jlkaus/fybrik-1d99fba5-c82c-4aeb-b86f-658b7e0c023c","app.fybrik.io/app-uuid":"15b03e59-8328-49e2-ad03-43ad1223e714","time":"2022-04-05T19:04:54Z","caller":"/workspace/source/mesh-for-data/manager/controllers/app/fybrikapplication_controller.go:345","message":"No plotter will be generated since no datasets are specified"}
{"level":"trace","Controller":"FybrikApplication","fybrikapplication":"fybrik-jlkaus/fybrik-1d99fba5-c82c-4aeb-b86f-658b7e0c023c","app.fybrik.io/app-uuid":"15b03e59-8328-49e2-ad03-43ad1223e714","Action":"Update","time":"2022-04-05T19:04:54Z","caller":"/workspace/source/mesh-for-data/manager/controllers/app/fybrikapplication_controller.go:160","message":"Updating status for desired generation 4"}
{"level":"trace","Controller":"FybrikApplication","fybrikapplication":"fybrik-jlkaus/fybrik-1d99fba5-c82c-4aeb-b86f-658b7e0c023c","time":"2022-04-05T19:04:54Z","caller":"/workspace/source/mesh-for-data/manager/controllers/app/fybrikapplication_controller.go:82","message":"*** FybrikApplication Reconcile ***"}
2022-04-05T19:04:54.110Z        ERROR   controller-runtime.manager.controller.fybrikapplication Reconciler error        {"reconciler group": "app.fybrik.io", "reconciler kind": "FybrikApplication", "name": "fybrik-1d99fba5-c82c-4aeb-b86f-658b7e0c023c", "namespace": "fybrik-jlkaus", "error": "Operation cannot be fulfilled on fybrikapplications.app.fybrik.io \"fybrik-1d99fba5-c82c-4aeb-b86f-658b7e0c023c\": the object has been modified; please apply your changes to the latest version and try again"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
        /Projects/Proj1/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.5/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
        /Projects/Proj1/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.5/pkg/internal/controller/controller.go:214

Another run is ongoing right now, and so far, out of 238 iterations, this has happened 3 times, leaving the applications behind in a deleted, but unable to finalize state.  So far, the blueprints/plotters/helm installs have always gone away ok, during these runs.
These runs were against the 0.6.3 code, I believe.

According to the first log snippet - when Fybrik objects delete generated objects during  finalization processing and K8s DELETE call returns "not found" the error should not be propagated to the upstream.
For the second one, we have to cherry-pick #1385

For reference, from my 1000 runs last night, this happened 26 times.


Setup name and r3m drop

sets a custom name such that

python setup.py bdist_wheel --package_name robohive

can be run

removes the deps on r3m which makes pypi unhappy

Can you merge this @vikashplus ?


[BUG] Cannot cast ufunc subtract output in ComputeDVARS

On ds005:
160730-02:00:52,840 workflow ERROR:
	 ['Node ComputeDVARS.a00 failed to run on host nid00602.']
160730-02:00:52,841 workflow INFO:
	 Saving crash info to /scratch/03843/crn_plab/crn_plab/job-6856540562201907686-242ac114-0001-007-crn-automated-job/sjob-6856540562201907686-242ac114-0001-007-000/workdir_log/crash-20160730-020052-crn_plab-ComputeDVARS.a00-20137099-1ea0-441a-bcce-b384b62ae5c6.pklz
160730-02:00:52,841 workflow INFO:
	 Traceback (most recent call last):
  File "/work/01329/poldrack/lonestar/software_lonestar5/crn-agave/miniconda2/lib/python2.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 49, in run_node
    result['result'] = node.run(updatehash=updatehash)
  File "/work/01329/poldrack/lonestar/software_lonestar5/crn-agave/miniconda2/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 394, in run
    self._run_interface()
  File "/work/01329/poldrack/lonestar/software_lonestar5/crn-agave/miniconda2/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 504, in _run_interface
    self._result = self._run_command(execute)
  File "/work/01329/poldrack/lonestar/software_lonestar5/crn-agave/miniconda2/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 630, in _run_command
    result = self._interface.run()
  File "/work/01329/poldrack/lonestar/software_lonestar5/crn-agave/miniconda2/lib/python2.7/site-packages/nipype/interfaces/base.py", line 1034, in run
    runtime = self._run_wrapper(runtime)
  File "/work/01329/poldrack/lonestar/software_lonestar5/crn-agave/miniconda2/lib/python2.7/site-packages/nipype/interfaces/base.py", line 991, in _run_wrapper
    runtime = self._run_interface(runtime)
  File "/work/01329/poldrack/lonestar/software_lonestar5/crn-agave/miniconda2/lib/python2.7/site-packages/nipype/interfaces/utility.py", line 496, in _run_interface
    raise out
TypeError: Cannot cast ufunc subtract output from dtype('float64') to dtype('<i2') with casting rule 'same_kind'
Interface Function failed to run.

160730-04:29:13,602 workflow ERROR:
	 ['Node ComputeDVARS.a11 failed to run on host nid00140.']
160730-04:29:13,603 workflow INFO:
	 Saving crash info to /scratch/03843/crn_plab/crn_plab/job-7078397129346051610-242ac114-0001-007-crn-automated-job/sjob-7078397129346051610-242ac114-0001-007-000/workdir_log/crash-20160730-042913-crn_plab-ComputeDVARS.a11-6e678526-24b0-439b-b422-210a124d417a.pklz
160730-04:29:13,603 workflow INFO:
	 Traceback (most recent call last):
  File "/work/01329/poldrack/lonestar/software_lonestar5/crn-agave/miniconda2/lib/python2.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 49, in run_node
    result['result'] = node.run(updatehash=updatehash)
  File "/work/01329/poldrack/lonestar/software_lonestar5/crn-agave/miniconda2/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 394, in run
    self._run_interface()
  File "/work/01329/poldrack/lonestar/software_lonestar5/crn-agave/miniconda2/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 504, in _run_interface
    self._result = self._run_command(execute)
  File "/work/01329/poldrack/lonestar/software_lonestar5/crn-agave/miniconda2/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 630, in _run_command
    result = self._interface.run()
  File "/work/01329/poldrack/lonestar/software_lonestar5/crn-agave/miniconda2/lib/python2.7/site-packages/nipype/interfaces/base.py", line 1034, in run
    runtime = self._run_wrapper(runtime)
  File "/work/01329/poldrack/lonestar/software_lonestar5/crn-agave/miniconda2/lib/python2.7/site-packages/nipype/interfaces/base.py", line 991, in _run_wrapper
    runtime = self._run_interface(runtime)
  File "/home1/03843/crn_plab/workspace/mriqc/mriqc/interfaces/functional.py", line 41, in _run_interface
    output_all=self.inputs.output_all)
  File "/home1/03843/crn_plab/workspace/mriqc/mriqc/qc/functional.py", line 156, in compute_dvars
    mfunc -= mfunc.mean(axis=1)[..., np.newaxis]
TypeError: Cannot cast ufunc subtract output from dtype('float64') to dtype('<i2') with casting rule 'same_kind'
Interface ComputeDVARS failed to run.


[Carbon 10: Radio button] styling issues

Detailed description

Describe in detail the issue you're having.

I don't believe the following are intended

there isn't a hover state when the user mouses over the label or input
vertical radio buttons are center aligned

Is this issue related to a specific component?

Radio button

What did you expect to happen? What happened instead? What would you like to see changed?

there should be a hover state to inform the user they are hovering over the input (or label)
the radio button inputs are aligned with each other when using the vertical modifier

What browser are you working in?

Chrome

What version of the Carbon Design System are you using?

v10.0.0

What offering/product do you work on? Any pressing ship or release dates we should be aware of?

Watson Health Pattern & Asset Library (PAL)
Additional information
Vertical alignment

Use case when a label wraps to multiple lines (both horizontal and vertical layouts)

@aagonzales can you speak to radio buttons with vertical layouts?

Radio buttons intentionally do NOT have a hover.
Vertically stacked radio button should left aligned.

A workaround seems to be align-items: flex-start to .bx--radio-buttton-group.


Add ability to select a region of interesting using user created region

@aashish24 Does the annotation layer satisfy this issue?

yes it does

Added via PR #613 and other annotation work.


Client created an invalid block with REMOTE_STAKING_INPUT_BIGGER_THAN_OUTPUT

Describe the bug
I saw this in the testnet logs. Seems that somebody managed to create an block invalid because of REMOTE_STAKING_INPUT_BIGGER_THAN_OUTPUT.
2019-04-16 15:00:22 [finalization] Process candidate tip block_hash=fed812f0c99d1c07f95efc737f2e3fff583e1175b4c45a8e6dbe8f6ab391ae36 height=1203
2019-04-16 15:00:22 [  validation] ConnectBlock: Invalid stake found for block=fed812f0c99d1c07f95efc737f2e3fff583e1175b4c45a8e6dbe8f6ab391ae36 failure=REMOTE_STAKING_INPUT_BIGGER_THAN_OUTPUT
2019-04-16 15:00:22 [            ] InvalidChainFound: invalid block=fed812f0c99d1c07f95efc737f2e3fff583e1175b4c45a8e6dbe8f6ab391ae36  height=1203  log2_work=64.032837  date=2019-04-16 15:00:24
2019-04-16 15:00:22 [            ] InvalidChainFound:  current best=c4183e870f473070b47c1747b63a904c401c9511a914b54bff8a1ad3edaecb22  height=1202  log2_work=64.031509  date=2019-04-16 15:00:20
2019-04-16 15:00:22 [            ] ERROR: ConnectTip(): ConnectBlock fed812f0c99d1c07f95efc737f2e3fff583e1175b4c45a8e6dbe8f6ab391ae36 failed
2019-04-16 15:00:22 [            ] InvalidChainFound: invalid block=fed812f0c99d1c07f95efc737f2e3fff583e1175b4c45a8e6dbe8f6ab391ae36  height=1203  log2_work=64.032837  date=2019-04-16 15:00:24
2019-04-16 15:00:22 [            ] InvalidChainFound:  current best=c4183e870f473070b47c1747b63a904c401c9511a914b54bff8a1ad3edaecb22  height=1202  log2_work=64.031509  date=2019-04-16 15:00:20
2019-04-16 15:00:23 [   proposing] Not proposing this time (wallet=wallet.dat)
2019-04-16 15:00:27 [   proposing] Not proposing this time (wallet=wallet.dat)
2019-04-16 15:00:28 [            ] ERROR: AcceptBlockHeader: block fed812f0c99d1c07f95efc737f2e3fff583e1175b4c45a8e6dbe8f6ab391ae36 is marked invalid

To Reproduce
Don't know.
Expected behavior
An up-to-date client should not create invalid blocks.

Fixed in #1017


Can't seem to build the package

I have this package which is a version of your basic example
// swift-tools-version:5.8

import PackageDescription

let package = Package(
    name: "Test",
    products: [
        .library(name: "Test", type: .dynamic, targets: ["Test"]),
    ],
    dependencies: [
        .package(url: "https://github.com/migueldeicaza/SwiftGodot", from: "1.0.0"),
    ],
    targets: [
        .target(
            name: "Test",
            dependencies: ["SwiftGodot"],
            path: "swift_src",
            swiftSettings: [.unsafeFlags(["-suppress-warnings"])],
            linkerSettings: [.unsafeFlags(["-Xlinker", "-undefined", "-Xlinker", "dynamic_lookup"])]
        ),
    ]
)

However when I try to compile using  swift build --configuration release --arch x86_64 I get an error
Updating https://github.com/migueldeicaza/SwiftGodot
Updated https://github.com/migueldeicaza/SwiftGodot (0.41s)
Computing version for https://github.com/migueldeicaza/SwiftGodot
error: Dependencies could not be resolved because no versions of 'swiftgodot' match the requirement 1.0.0..<2.0.0 and root depends on 'swiftgodot' 1.0.0..<2.0.0.

Where is the "1.0.0" from the documentation?   I think I need to fix that.
And what is the error you get with the unsafe build flag?

Yeah I added 1.0.0 myself. I think it would be useful to have a more fleshed out Package.swift


Created test files for types_test.go

This clearly has a lot of redundant commits @archishou -- can you please clean that up?


Use new release management rules

fixes #39

Codecov Report

Merging #40 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #40   +/-   ##
=======================================
  Coverage   93.81%   93.81%
=======================================
  Files           7        7
  Lines         372      372
  Branches       49       49
=======================================
  Hits          349      349
  Misses         23       23

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c69f49c...d6e0c6e. Read the comment docs.


Fix notebooks

I've been asked to review the notebooks provided in this repo to ensure they are all accurate and working. This PR is the changes I made to that end for RAiDER v0.5.1.

Fixed inaccuracies where I could, plus all cells should run without errors now save for these cases:

Defining Custom Weather Models: Example custom weather model ABCD left unimplemented
Options: Run that use GMAO still affected by dbekaert/RAiDER #611


Spawn Issues

Issue with spawning, sometimes players will Spawn with a map etc, and other times they will not.

Would be solved by making a module that resets and adds player a default bunch of gear when respawning (EH?).

I think need to put Eventhandler "Respawn" and add base stuff

I agree with superihppo

This issue will be closed since it will be solved after the 4.0.0 release.


Add virtual to infrastructure

This is not yet tested

@binford2k : The note has me thinking - is this ready to be merged, as in, is it tested, and reliable?

ah. It's now tested in a live class!
That was an emergency fix, I meant to update the PR after the class happened and we saw it not explode.

@binford2k do you want this in the release?


Add support for lcov file pattern

Codecov Report

Merging #2 into master will decrease coverage by 0.88%.
The diff coverage is 94.52%.

@@            Coverage Diff             @@
##           master       #2      +/-   ##
==========================================
- Coverage   94.73%   93.85%   -0.89%
==========================================
  Files          10       12       +2
  Lines         133      179      +46
==========================================
+ Hits          126      168      +42
- Misses          7       11       +4

Impacted Files
Coverage Δ

lib/client/model/commit_coverage.dart
100% <100%> (ø)
:arrow_up:

lib/client/model/file_coverage.g.dart
100% <100%> (ø)
:arrow_up:

lib/client/model/file_coverage.dart
100% <100%> (ø)
:arrow_up:

...nverter/lcov/report_to_commit_coverage_mapper.dart
100% <100%> (ø)
:arrow_up:

lib/client/model/commit_coverage.g.dart
100% <100%> (ø)
:arrow_up:

...nt/coverage_converter/lcov/path_prefix_finder.dart
100% <100%> (ø)

...nt/coverage_converter/lcov_coverage_converter.dart
83.33% <66.66%> (-16.67%)
:arrow_down:

lib/bitbucket_code_coverage_command_runner.dart
92.3% <80%> (-7.7%)
:arrow_down:

lib/command/post/post_command.dart
92.85% <91.66%> (-1.88%)
:arrow_down:

lib/command/post/converter_strategy.dart
92.85% <92.85%> (ø)

... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3b0cd2f...5eae35f. Read the comment docs.


yarn.lock and package-lock.json

yarn.lock and package-lock.json should not both exist, choose one package manager to avoid confusion

Thanks for raising - the yarn.lock file was removed in our recent pull here 🙂


Create 0000-01-02-rashmimardi.md

#3  Github is Pretty much exiciting

#3 Github is Pretty much  exiciting

hello Mr. Bot

Step 7: Respond to a review
Your pull request is looking great!
Let’s add some content to your file. Replace line 5 of your file with a quotation or meme and witty caption. Remember: Markdown is supported.
⌨ Activity: Change your file

Click the Files Changed tab in this pull request
Edit your newly added file
Replace line 5 with something new
Scroll to the bottom and click Commit Changes

Note : Can't find the button to edit the file? It may look like a pencil, or it may look like three dots.
I'll respond when I detect a commit on this branch.

I changed the file.

Step 7: Respond to a review
Your pull request is looking great!
Let’s add some content to your file. Replace line 5 of your file with a quotation or meme and witty caption. Remember: Markdown is supported.
⌨ Activity: Change your file

Click the Files Changed tab in this pull request
Edit your newly added file
Replace line 5 with something new
Scroll to the bottom and click Commit Changes

Note : Can't find the button to edit the file? It may look like a pencil, or it may look like three dots.
I'll respond when I detect a commit on this branch.


Add binaries to formula

I launch IntelliJ exclusively from the terminal. With JetBrains Toolbox I was able to have idea and idea-eap binaries. The regular IntelliJ formula upstream installs an idea binary, but these EAP formula do not.
Would you be open to adding binary installation to these?

Of course I'd be happy to add it!  I just don't have the time right now, but if you send me a pull request, I'll review it as soon as I can.