NFC Control

Im really interested in NFC selector for Android, I have a half finished project that requires an NFC reader, any news?

We can implement nfc_manager widget.
A short guide: https://medium.com/flutter-community/nfc-with-flutter-f8c3515cb0e0
Do you have any experience with Flutter? I'm working on a guide on how to add 3rd-party widgets to Flet app as separate libraries. Could be a good case.

We can implement nfc_manager widget. A short guide: https://medium.com/flutter-community/nfc-with-flutter-f8c3515cb0e0
Do you have any experience with Flutter? I'm working on a guide on how to add 3rd-party widgets to Flet app as separate libraries. Could be a good case.

0 Experience on Flutter, i start in web deplay with Flet 4 months ago sorry :( But the guide sounds rly good! I just got the application to take the NFC permissions on the phone even though I have not been able to use the reader.
Whatever I can do to help, I'm here for you!


Update ALZ-Root-Default.jsonc

Data Collection Rule Resource Id should be dcrResourceId according to the initiative definition "Enable Azure Monitor for VMSS with Azure Monitoring Agent(AMA)".
Also, removing quotation marks for the boolean values of "Bring Your Own User-Assigned Managed Identity" as allowed values are false; true and not "false"; "true".

Yes - you are right - I swear the other day when I was looking at this there was a parameter called dataCollectionResourceId - I must have been looking at the wrong initiative.


HTML attribute ‘contenteditable’ can not bind data?

I test,it is not work, so ,is vue not support html attribute contenteditable?

https://github.com/vuejs/vue/issues/1670

@azamat-sharapov  @yyx990803 Is  binding event  a good way to solve it?

Use a custom directive. http://vuejs.org/guide/custom-directive.html


fix up python version to 3 and add apk upgrade

Purpose
Briefly describe the purpose of the change, and/or link to the JIRA ticket for context
To fix an issue with Zap DockerFile that was causing issues locally when spinning up in docker-compose. This does not commission the docker file to run the zap scripts it has yet.
Approach

Upgrade PIP to PIP3 and add some prerequisites and no-cache to prevent warnings.
add an Apk upgrade to prevent vuln creep.
Made the docker file clearer in terms of steps taken by running each step separately.

Learning

general research around docker and python:
https://stackoverflow.com/questions/7446187/no-module-named-pkg-resources
https://wiki.alpinelinux.org/wiki/Alpine_Linux_package_management#Upgrade_a_Running_System
also, what is zap?
https://www.zaproxy.org/

Checklist

[ X] I have performed a self-review of my own code
[ ] I have updated documentation (Confluence/GitHub wiki/tech debt doc) where relevant
[ ] I have added tests to prove my work
[ ] The product team have tested these changes

Coverage remained the same at 76.169% when pulling ca423de2342bdad421c8b5840d3487ad89cf1cad on zap-upgrade-python-docker into 68bb54b2d2ebabb7371fe09cc6586f12d809378a on master.


fix: improve docs

This PR drastically improves the documentation by improving the sidebars and adding a GitHub icon.
Fixed: https://github.com/puppeteer/puppeteer/pull/9176
Fixed: https://github.com/puppeteer/puppeteer/issues/9173

This PR should start with docs: as there is no code changes so it should not be mentioned in the changelog.

Do you want to update the Cloud Functions part of the troubleshooting as well?

Do you want to update the Cloud Functions part of the troubleshooting as well?

I'll do this in another PR.


cmap picker with imgui

I'm doing a bunch of imgui stuff now which has finally forced me to learn how to use wgpu directly :smile: . I can keep adding more examples if you're interested. There are certain imgui things which are backend-specific, for instance there are many examples out there on using textures with opengl but not with wgpu-py so it took me a while to figure out how to implement something like this.
https://github.com/user-attachments/assets/be28e2a0-72a9-4755-945d-f1cb48329c81
draft because I want to clean it up a bit and add comments

I think we can add an add_texture() method in the ImguiWgpuBackend class.

I think we can add an add_texture() method in the ImguiWgpuBackend class.

I was thinking that too! I want to get more familiar with imgui & wgpu before making a PR on that, I want to see what else in imgui might need things like this.

added a border and comments, should be good to go :)


feat: implement read -t

read -t reading stdin with timeout. Implemented with polling on unix and with a little bit insane and unsafe{} async implementation on windows. It works but work in progress. Just want to discuss different directions how we can properly implement this. Seems like non of the popular IO libraries support Windows's stdin/file IO operations because it is really a madness

Just read the crossterm sources, maybe crossterm can help https://github.com/crossterm-rs/crossterm. Brush already have it as indirect dependency. I've noticed that brush has sys::terminal with a lot of TODO's and crossterm has that functions as well.
read implementation in nushell is very clean https://github.com/nushell/nushell/blob/af9c31152a3926c5c85bb67e1125cdb52abe3ace/crates/nu-command/src/platform/input/input_.rs#L14

I skimmed your changes but didn't read them too closely yet. You weren't kidding when you said the Windows side would be complicated!
I'm not against getting it working on Windows, but I think the complexity required is beyond the scope of what we should do and maintain in brush. With some exceptions, we've tried to keep away from lower-level console/terminal I/O, instead delegating those responsibilities to reedline, crossterm, nix, et al.
My vote here would be to:

Leverage existing abstractions in terminal/console crates to keep the built-in implementation relatively clean and platform-agnostic; where those abstractions don't do enough, abstract away any truly platform-specific details via sys.
Lean on existing standard crates for Unix implementation.
Leave the Windows side stubbed out / unimplemented for now.
Consider contributing/integrating your Windows-specific solution with one of the terminal/console crates in the crates ecosystem.


Convert message to JSON fromat

Plain text message is exchanged between clients and server. Use JSON instead of plain text to be able to have additional information

https://github.com/ihiroky/live-comment/pull/30


Playerlist times out on Bungeecord Proxy setups

I am using Bungeecord and the
print(f"Players Online: {', '.join(query.players.names)}")
Does not work, everything else does. I don't think Bungeecord supports player lists, is there any way around this?
Thanks

Did you enable query? Check query_enabled in Bungee's config.

Could you provide the full traceback of the error, and a minimal reproducable code that demonstrates your issue? It's hard for us to tell what's wrong from just your example print statement.

query_enabled is set to enabled.
Figured it out anyway, I had my port set to the server port instead of the query port, my bad.

Thanks for your help!


Merge v2.18.1 tag from upstream

:open_mouth: I'll approve when it's been tested.

This was tested to see if it helped fix the inaccurate intrinsics issue but it did not. Closing for now


Dynamic bullet's width is not properly calculated in differently-sized bullets

Swiper Version: 6.5.5
Platform/Target and Browser Versions: All browers

What You Did
Hello swiper team. I think swiper-core's pagination module does not cope with bullets with dynamic sizes.
I overwrited 'swiper-pagination-bullet-active', and 'swiper-pagination-bullet' class to have wider width
in active bullets. such like below snippet
.swiper-pagination-bullet {
  display: inline-block;
  width: 4px;
  height: 4px;
  margin: 0 2px !important;
  border-radius: 2px;
  background-color: #6f7375;
  opacity: 0.5;
}
.swiper-pagination-bullet-active {
  width: 16px;
  opacity: 1;
}

and when i enabled dynamic bullets, total width of bullet container is not stable. when i slided from index 0 to index 1, dynamic bullet's container witdh jumps to 56px to 140px.
So that i look into swiperjs github repo's code, i found code like below snippet
/*  src/components/pagination/pagination.js -> from 40 line */
if (params.dynamicBullets) {
        swiper.pagination.bulletSize = bullets.eq(0)[swiper.isHorizontal() ? 'outerWidth' : 'outerHeight'](true);
        $el.css(swiper.isHorizontal() ? 'width' : 'height', swiper.pagination.bulletSize * (params.dynamicMainBullets + 4) + "px");

according to this code, dynamic bullet's container (a.k.a $el)'s width (or height in vertical mode) is calculated like this
bulletSize = current bullet list's first element's outerWidth with margin
$el's width = bulletSize * (dynamicBullet param + 4)
So, in bullets with differently-sized. $el's width will jump every time when we slide.
I think we need new parameter in pagination component. to support bullet with different sizes.
such as averageBulletSize.
here is simple reproduction sample link. when you slide from first to second, you can see contanier width jumps
https://stackblitz.com/edit/swiper-demo-1-default-1zjtlh?file=index.html
Expected Behavior
Swiperjs pagination components need average bullet size to properly calculate bullet's container size
in dynamic bullet mode
Actual Behavior
Swiperjs pagination components calculate bullet's container size via first element of bullet list.

Hello, I think the problem still exists in the current version.


Add description for documentation output

Similar in nature to #230, it would be great if we could add descriptive metadata to tests that explain why the test is there. This can be accomplished in the YAML file themselves via comments, of course, but it would be nice to have an output mode, like -f documentation, that renders these back out as strings, like:
[vagrant@host ~]$ goss v -f documentation
File: /etc/passwd: exists: matches expectation: [true], satisfies PCI requirement 8
File: /etc/passwd: mode: matches expectation: ["0644"], satisfies PCI requirement 8
File: /etc/passwd: owner: matches expectation: ["root"], satisfies PCI requirement 8
File: /etc/passwd: group: matches expectation: ["root"], satisfies PCI requirement 8
File: /etc/passwd: filetype: matches expectation: ["file"], satisfies PCI requirement 8
File: /etc/passwd: size: matches expectation: [1130], satisfies PCI requirement 8

given an optional parameter like:
file:
  /etc/passwd:
    exists: true
    mode: "0644"
    owner: root
    group: root
    filetype: file
    contains:
      - aloof
    documentation: satisfies PCI requirement 8

This output, normally supressed, would be useful to show an auditor, say, of the system.
If a PR implementing the above would be likely to be approved, I'd welcome pointers or suggestions on how best to implement it. I'm fairly new to golang.

Hey! Interestingly I thought about this earlier today too. I'm attempting to build the config for goss to validate servers to CIS benchmarks and thought it'd be cool to reference them in a comments or documentation field.
I guess we'd only want the output on certain formats  like documentation, JSON and nagios_verbose?

Ahmed is usually open to pull requests and new functions that add value not complexity. As a new golang user myself I've found his feedback really helpful even when what I'd done was perhaps not ideal.
My advice is to fork it, branch it, code it, and document it and open a PR. Just be open to the possibility that initially your implementation might need some reworking. I've opened plenty of PRs and then killed them because they werent great. There isn't a great deal of unit testing in this project the tests are more functional / integration focused.

Does the current title and meta not meet your needs? manually editing goss files Seems I need to add this features documentation to the main manual, it appears to only be in the readme.
user:
  root:
    title: ensure root user exists
    meta:
      hint.1: ctrl-alt-delete
      hint.2: Hold power button
      hint.3: Re-install your OS
      hint.4: Cry
      id: code-1234
      severity: 5
    exists: true

Output:
$ goss validate --format documentation
Title: ensure root user exists
Meta:
    hint.1: ctrl-alt-delete
    hint.2: Hold power button
    hint.3: Re-install your OS
    hint.4: Cry
    id: code-1234
    severity: 5
User: root: exists: matches expectation: [true]

Total Duration: 0.000s
Count: 1, Failed: 0, Skipped: 0

Fantastic, @aelsabbahy! That exactly meets my needs here!

Thanks alot for this tool! 🌟

I didn't know about that feature! 🌠


⚠ Bofrost.at has degraded performance

In d54489d, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1043 ms

Resolved: Bofrost.at performance has improved in 18fa032.


Fix test: handler losing connectivity

What this PR does / why we need it:
Fixing leak of the test that affect [test_id:2666]
.
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #
Special notes for your reviewer:
Release note:
NONE

/cc @fgimenez

/retest

/approve


Create Loading Page

Create loading page based on figma design
Add to all pages
Replace other loading screens

/cib create-loading-page

Branch 163-create-loading-page created!


add nrf51 p26 & p27 to analog api

Issue:
As of today nrf51 p26 & p27 can not be used as analog input as they are missing the analog api pin map definition. This might not be a problem for the DK as those pins are connected to the external XTAL anyways but on some prototyping boards like the xuntong ptr5188 these pins are the only way to use ADC.
So these two additional lines enable analog in on p26 & 27:
Lines to be added:
    {p26, ADC0_0, 1},
    {p27, ADC0_0, 2},

Current implementation:
static const PinMap PinMap_ADC[] = {
    {p1, ADC0_0, 4},
    {p2, ADC0_0, 8},
    {p3, ADC0_0, 16},
    {p4, ADC0_0, 32},
    {p5, ADC0_0, 64},
    {p6, ADC0_0, 128},
    {NC, NC, 0}
};

Hi,
thanks for reporting. Are you planning to send PR with corrections to pins to enable those 2?

Hi,
I've never done it before, but will give it a try.
Cheers,
Jens

Ok, done! Hope I did it right.

This issue can be closed now as the PR was merged


Plugin is empty

Looks like this plugin is missing the gatsby-ssr.js file when pulling down via NPM / Yarn

hmm weird.  let me take a look.  does it not work for you?  i got it working for me but lemme check.  if there is any output you can share, that would be great.

working now!  my apologies!  upgrade to  v2.0.0 !


feat: Add terragrunt_providers_lock hook

Put an x into the box if that apply:

[ ] This PR introduces breaking change.
[ ] This PR fixes a bug.
[x] This PR adds new functionality.
[ ] This PR enhances existing functionality.

Description of your changes
Add hook that generates .terrafrom.lock.hcl using the terragrunt providers lock command
Fixes #630
How can we test changes
Clone dummy repo
git clone git@github.com:wzooff/cats-dogs-infra.git
cd cats-dogs-infra/dogs/test/cluster
pre-commit install

Hook pointed to the latest commit in this PR
You can

remove .terraform.lock.hlc file and try to commit
edit modules/dummy/main.tf and set required version for null to 3.2.2 and then edit dogs/test/cluster/terragrunt.hcl name attribute (simulates version bump for module)

All above will trigger lock file regeneration.

Thank you for the rapid contribution 🙇 Can we have this feature merged and refactor all the code related to Terragrunt realms later?

After
git clone git@github.com:wzooff/cats-dogs-infra.git
cd cats-dogs-infra/dogs/test/cluster
pre-commit install

rm .terraform.lock.hcl
git add -A
pre-commit run -a

All these errors in outputs, it's expected?
[INFO] Initializing environment for https://github.com/wzooff/pre-commit-terraform.
Terragrunt providers lock................................................Failed
- hook id: terragrunt_providers_lock
- exit code: 1

ERRO[0000] Did not find any Terraform files (*.tf) in /tmp/cats-dogs-infra/cats/prod/cluster
ERRO[0000] Unable to determine underlying exit code, so Terragrunt will exit with error code 1
ERRO[0000] Did not find any Terraform files (*.tf) in /tmp/cats-dogs-infra/cats/test/storage/s3
ERRO[0000] Unable to determine underlying exit code, so Terragrunt will exit with error code 1
INFO[0000] Downloading Terraform configurations from git::ssh://git@github.com/terraform-aws-modules/terraform-aws-s3-bucket.git?ref=3.7.0 into /tmp/cats-dogs-infra/dogs-ua/test/storage/s3/.terragrunt-cache/wOb2gGSOIRhzEnVBnIOKou4IY40/9PaYfi3-KkZZxN2E0HJDpIPM9XI
- Fetching hashicorp/null 3.2.2 for darwin_arm64...
- Retrieved hashicorp/null 3.2.2 for darwin_arm64 (signed by HashiCorp)
- Fetching hashicorp/null 3.2.2 for darwin_amd64...
- Retrieved hashicorp/null 3.2.2 for darwin_amd64 (signed by HashiCorp)
- Fetching hashicorp/null 3.2.2 for linux_amd64...
ERRO[0002] downloading source url git::ssh://git@github.com/terraform-aws-modules/terraform-aws-s3-bucket.git?ref=3.7.0
1 error occurred:
        * error downloading 'ssh://git@github.com/terraform-aws-modules/terraform-aws-s3-bucket.git?ref=3.7.0': /usr/bin/git exited with 1: error: pathspec '3.7.0' did not match any file(s) known to git

ERRO[0002] Unable to determine underlying exit code, so Terragrunt will exit with error code 1
- Retrieved hashicorp/null 3.2.2 for linux_amd64 (signed by HashiCorp)
- Obtained hashicorp/null checksums for darwin_arm64; This was a new provider and the checksums for this platform are now tracked in the lock file
- Obtained hashicorp/null checksums for darwin_amd64; This was a new provider and the checksums for this platform are now tracked in the lock file
- Obtained hashicorp/null checksums for linux_amd64; This was a new provider and the checksums for this platform are now tracked in the lock file

Success! Terraform has updated the lock file.

Review the changes in .terraform.lock.hcl and then commit to your
version control system to retain the new checksums.

ERRO[0000] Did not find any Terraform files (*.tf) in /tmp/cats-dogs-infra/cats/test/cluster
ERRO[0000] Unable to determine underlying exit code, so Terragrunt will exit with error code 1
ERRO[0000] Did not find any Terraform files (*.tf) in /tmp/cats-dogs-infra/dogs-ua/test/cluster
ERRO[0000] Unable to determine underlying exit code, so Terragrunt will exit with error code 1
ERRO[0000] Did not find any Terraform files (*.tf) in /tmp/cats-dogs-infra/cats/prod/storage/s3
ERRO[0000] Unable to determine underlying exit code, so Terragrunt will exit with error code 1
ERRO[0000] Did not find any Terraform files (*.tf) in /tmp/cats-dogs-infra/dogs/test/storage/s3
ERRO[0000] Unable to determine underlying exit code, so Terragrunt will exit with error code 1

Can we have this feature merged

@Tensho If you're in an urgency you can temporary point your local pre-commit-terraform to use latest commit from this PR via rev arg: https://github.com/antonbabenko/pre-commit-terraform/pull/632#discussion_r1498310898

You can test it locally without committing changes by
# run inside cats-dogs-infra/
/usr/bin/bash <path-to-wzoof-repo>/hooks/terragrunt_providers_lock.sh dogs-ua/test/cluster/.

I'm afraid that this repo not ready for running hooks in all folders :) coz there a lot of modules that require proper aws creds.

@MaxymVlasov try now. I've made all modules dummy in the repo
➜  cats-dogs-infra git:(main) ✗ pre-commit run --all-files
Terragrunt providers lock................................................Passed
➜  cats-dogs-infra git:(main) ✗ git status
On branch main
Your branch is up to date with 'origin/main'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	cats/prod/cluster/.terraform.lock.hcl
	cats/prod/storage/s3/.terraform.lock.hcl
	cats/test/cluster/.terraform.lock.hcl
	cats/test/storage/s3/.terraform.lock.hcl
	dogs-ua/test/cluster/.terraform.lock.hcl
	dogs-ua/test/storage/s3/.terraform.lock.hcl
	dogs/test/storage/s3/.terraform.lock.hcl

nothing added to commit but untracked files present (use "git add" to track)

This PR is included in version 1.88.0 :tada:


refactor Volume to Directory

jenkins/superCI comment

Status: success, return code = 0
Result url: https://da.gd/XY6TG
Comment: This is testing CI

I hope I addressed all of them, let's see what CI will tell, b/c I was getting some odd failures locally

jenkins/superCI comment

Status: success, return code = 0
Result url: https://da.gd/h7L5
Comment: This is testing CI

jenkins/superCI comment

Status: failure, return code = 1
Result url: https://da.gd/VLtRG
Comment: This is testing CI

jenkins/superCI comment

Status: success, return code = 0
Result url: https://da.gd/IcmPk
Comment: This is testing CI


Update README.md

Links to the contents of the DiffusionFastForward course (summary in project repo README: https://github.com/mikonvergence/DiffusionFastForward)

Sorry for not adding the paper. I added it !

Sorry for not adding the paper. I added it !

Excellent, I hope it's helpful! Thanks! 🙏


Meld --strong --reify-generic-functions into --preview-dart-2

Meld --strong --reify-generic-functions into --preview-dart-2 so that we have a single opt in option to run dart 2.0 code in the command line VM

This has been done, closing issue.

What effect does --strong have that is not included in --preview-dart-2
What does --preview-dart-2 contain already?
Is there a doc that provides some insight?

--preview-dart-2 is the one single flag that supports Dart 2.0 semantics, please see  for more details.
All the other flags are internal VM development/debugging flags and are not intended for use by end users.


Feature Management for all frameworks

This is a great feature and I'm wondering if Feature Management will come to the other framework (.NET Core, .NET, Azure Functions, Java)?

Thanks, @calloncampbell. The short answer is yes, we do plan to make it available to all popular frameworks.
Please note, the Microsoft.FeatureManagement is a .NET Standard library, which means it already can be used in ASP.NET Core, .NET Core, .NET Framework and Azure Functions written in .NET.


[MM-13216] Add E2E for CTRL/CMD+K functionality

Summary

added E2E for CTRL/CMD+K functionality - on click of button and on key press
added elements IDs to QuickSwitchModal such as quickSwitchHint and quickSwitchInput
added commands such as toAccountSettingsModalChannelSwitcher and typeCmdOrCtrl

Ticket Link
Jira ticket: MM-13216
Checklist

[x] Ran make check-style to check for style errors (required for all pull requests)
[x] Ran make test to ensure unit and component tests passed
[x] Added or updated unit tests (required for all new features)

@saturninoabril is this going to 5.7?

0/5 but I'd recommend to get this in to 5.7 since this is a low-risk change and impacts tooling for quality.


PBJ failed to parse a record file

Description
When trying to parse a record file from Hedera main-net in V6 record format defined in https://hips.hedera.com/hip/hip-435. The first 4 bytes are a version int followed by protobuf object RecordStreamFile. PBJ failed with exception:
Exception in thread "main" java.nio.BufferUnderflowException
	at com.hedera.pbj.runtime.io.stream.ReadableStreamingData.readByte(ReadableStreamingData.java:111)
	at com.hedera.pbj.runtime.io.ReadableSequentialData.readVarLong(ReadableSequentialData.java:449)
	at com.hedera.pbj.runtime.io.ReadableSequentialData.readVarInt(ReadableSequentialData.java:433)
	at com.hedera.hapi.streams.codec.RecordStreamFileProtoCodec.parseInternal(RecordStreamFileProtoCodec.java:170)
	at com.hedera.hapi.streams.codec.RecordStreamFileProtoCodec.parse(RecordStreamFileProtoCodec.java:41)
	at com.hedera.hapi.streams.codec.RecordStreamFileProtoCodec.parse(RecordStreamFileProtoCodec.java:26)
	at com.hedera.node.app.records.RecordStreamTest.generateRecordStreamItems3(RecordStreamTest.java:151)
	at com.hedera.node.app.records.RecordStreamTest.main(RecordStreamTest.java:119)

Steps to reproduce
Parsing with PBJ throws exception
    private static void fails() throws Exception {
        final URL recordFileUrl = RecordStreamTest.class.getResource("/record-files/2023-05-01T00_00_00.026034002Z.rcd.gz");
        try (final ReadableStreamingData in = new ReadableStreamingData(
                new GZIPInputStream(Files.newInputStream(Path.of(recordFileUrl.toURI()))))) {
            int version = in.readInt();
            System.out.println("version = " + version);
            RecordStreamFile recordStreamFile =  RecordStreamFile.PROTOBUF.parse(in);
            System.out.println("recordStreamFile = " + recordStreamFile);
        }
    }

Parsing with Google protocol generated parser works fine
    private static void works() throws Exception {
        final URL recordFileUrl = RecordStreamTest.class.getResource("/record-files/2023-05-01T00_00_00.026034002Z.rcd.gz");
        try (final DataInputStream in = new DataInputStream(
                new GZIPInputStream(Files.newInputStream(Path.of(recordFileUrl.toURI()))))) {
            int version = in.readInt();
            System.out.println("version = " + version);
            com.hedera.services.stream.proto.RecordStreamFile recordStreamFile =
                    com.hedera.services.stream.proto.RecordStreamFile.parseFrom(in);
            System.out.println("recordStreamFile = " + recordStreamFile);
        }
    }

Additional context
No response
Hedera network
mainnet
Version
v0.38.0
Operating system
None

2023-05-01T00_00_00.026034002Z.rcd.gz
failing file

I found the issue, we are using ReadableStreamingData in while (input.hasRemaining()) {...}  loop in com/hedera/hapi/streams/codec/RecordStreamFileProtoCodec.java:167. It can not detect the end of file EOF because hasRemaining() is always true. We used to do one byte of read-ahead to make this work. @rbair23 removed that and I am not sure the new answer.

workaround

Fixed with https://github.com/hashgraph/pbj/pull/47


Default Project List Display Priority

Be default projects which are RED colour should appear on top, right now it is displaying as per date of creation

Duplicate #1512


OSInfo and utilities should be in separate module

Eventually, will be creating optional dependency for pluggable transports. Will need extraction tools for those as well. Would be best to preempt that (which will need OSInfo) by moving it to a separate module now.

Util module: binary-util
Pluggable Transports: binary-pt

Npmjs: kmp-tor-binary-pt-resources


Add support for etcd encryption in vSphere

Description of changes:
Add support for etcd encryption in vSphere along with an e2e test
Testing (if applicable):
Ran the e2e test and verified it works
    cluster.go:913: Skipping VM cleanup
--- PASS: TestVSphereKubernetes128EtcdEncryption (1130.91s)
PASS

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/approve


Much faster deserialization via new ImmutableRoaringBitmap(ByteBuffer.wrap(data)).toRoaringBitmap()

The two seemingly equivalent operations below yield deserialization performance results. "Example 1"  below seems to perform 5x to 10x faster than the typical bitmap deserialization as shown in "Example 2".
//Example 1
//byte[] -> ImmutableRoaringBitmap -> RoaringBitmap
byte[] = data
RoaringBitmap bitmap = new ImmutableRoaringBitmap(ByteBuffer.wrap(data)).toRoaringBitmap();

vs
//Example 2
//byte[] -> RoaringBitmap
byte[] = data
RoaringBitmap bitmap = new RoaringBitmap();
ByteArrayInputStream bais = new ByteArrayInputStream(data);

try (DataInputStream dis = new DataInputStream(bais)) {
            bitmap.deserialize(dis);
 }
...

Is the second example valid code?

Apologies, I pasted the wrong code.

I am puzzled. The new ImmutableRoaringBitmap(ByteBuffer.wrap(data)).toRoaringBitmap() approach appears to be much faster.
 A quick inspection shows that bitmap.deserialize spends most of its compute time at the the line below. https://github.com/RoaringBitmap/RoaringBitmap/blob/bd97775b254025be5ac12f525b13cc0152609ef0/roaringbitmap/src/main/java/org/roaringbitmap/RoaringArray.java#L329
I'm unable to find the analogous code in ImmutableRoaringBitmap.toRoaringBitmap().

These results need investigation but they point at a serious performance 'bug' as to how I have been recommending that people deserialize data from a byte array. It is not entirely surprising given that DataInputStream is not super fast as an interface... but the gap is huge here (between 5x and 10x difference).
cc @richardstartin @blacelle @ppiotrow @incaseoftrouble @owenkaser

@PedroAlvarado
Well. Util.toIntUnsigned(Short.reverseBytes(...)) should be something like two machine instructions. A much more likely cause of performance problems is Java's failure to inline the calls to readShort... but that's an API issue... I am not sure we can fix it per se.
This does not mean that this issue cannot be fixed. I think we should seriously consider a fast deserialization routine...

Side comment, not sure if relevant: DataInput offers a method to write to a byte[] buffer. As far as I understand the docs, the read bytes already are machine independent, in particular (short)((b[2 * i] << 8) | (b[2 * i + 1] & 0xff)) (where b is the buffer we wrote data into) should yield the shorts on the stream.

@incaseoftrouble
Yes. This may be relevant in the sense that a single call to read that retrieves a large block of bytes could be a lot better than many small calls to something like readShort.

@lemire Acknowledged. The point I was making is that such operation does not take place in a tight loop under the new ImmutableRoaringBitmap(ByteBuffer.wrap(data)).toRoaringBitmap() code path. This may help explain the performance difference, that is, somehow in the faster case we happen to be making bigger reads than a short at a time.

Which JVM version?

This can be reproduced with the JVM below.
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

IO effect is quite big: #320
I'm left with a 10%-30% penalty with recommended over Immutable

I'd be curious how large the difference is in a larger benchmark. There are some real-world benchmarks in the repo, right?
Direct reading should definitely be as least as fast as data -> immutable -> mutable and probably as fast as immutable reading up to a small factor?

Direct reading should definitely be as least as fast

This is not obvious to me. Count the minimal number of copies.

The data -> immutable -> mutable does zero copy between data -> immutable, and one copy between immutable -> mutable.
The current @blacelle  approach does one copy from the input data to temporary byte arrays, then another final copy.

So you have two copies versus just one, assuming that you are doing a minimal number of copies.
Now what @blacelle did not do, but we could do, is provide a new entry point where we take as an input a byte array. Then we could definitively beat or match the data -> immutable -> mutable path.

The real benefit of @blacelle 's current work is that it could help the more general case (e.g., where we are reading data from disk).

"should be" in the sense that one should aim for that. I don't see any reason why it should be fundamentally slower, if you see what I mean.

I don't see any reason why it should be fundamentally slower

Well... how do you go from the byte array to DataInput API and then back to the data we need without doing two copies of the data?
That is, the reason is the DataInput API.
It is the price of abstraction.

People tend to assume that abstract has no price... and sometimes that's true, but you can't assume that it general holds true. More general code, more often than not, also runs slower.  There is a trade-off. Easier software engineering vs. lower performance.

We speak of the same thing :) As someone who overeagerly abstracts things I learned how costly abstraction can be :D
In any case I'd be interested in the numbers.

Actually it has nothing to do with abstraction. The cost differential isn't even constant as a function of data set.

In our use case, we happen to have thousands of bitmaps cached in-memory as byte arrays. Allow me to second the idea of having an interface that takes a byte[] directly for deserialization.

@PedroAlvarado Let us see what @richardstartin has to reveal first. :-)

@lemire don't wait for me because I'm quite sure deserialising from a byte[] would be beneficial. Just wanted to blurt out in response to you blaming failure to inline that, if anything, there's too much inlining - look at the way the instruction and iTLB counters blow up and take a look at -XX:+PrintInlining. It's also extremely sensitive to run optimisation. Let me synthesise my results this weekend and get back to you.

Good.

 DataInputStream isn't really representative of a RandomAccessFile on a system with an SSD. In fact, we can sometimes do really well without copying any data or writing any new code by providing a ByteBuffer backed DataInput, which is also quite friendly to the garbage collector.
public static class BufferDataInput implements DataInput {

    private final ByteBuffer data;

    public BufferDataInput(ByteBuffer data) {
      this.data = data;
    }

    @Override
    public void readFully(byte[] bytes) throws IOException {
      data.get(bytes);
    }

    @Override
    public void readFully(byte[] bytes, int i, int i1) throws IOException {
      data.get(bytes, i, i1);
    }

    @Override
    public int skipBytes(int i) throws IOException {
      data.position(data.position() + i);
      return data.position();
    }

    @Override
    public boolean readBoolean() throws IOException {
      return data.get() != 0;
    }

    @Override
    public byte readByte() throws IOException {
      return data.get();
    }

    @Override
    public int readUnsignedByte() throws IOException {
      return data.get() & 0xFF;
    }

    @Override
    public short readShort() throws IOException {
      return data.getShort();
    }

    @Override
    public int readUnsignedShort() throws IOException {
      return data.getShort() & 0xffff;
    }

    @Override
    public int readInt() throws IOException {
      return data.getInt();
    }

    @Override
    public long readLong() throws IOException {
      return data.getLong();
    }
...
  }

full results
Benchmark                                                          (dataset)  (runOptimise)   Mode  Cnt      Score      Error  Units
RealDataSerializationBenchmark.bufferBackedDataInput              census1881           true  thrpt    5    754.988 ±   41.436  ops/s
RealDataSerializationBenchmark.bufferBackedDataInput              census1881          false  thrpt    5    259.896 ±   13.848  ops/s
RealDataSerializationBenchmark.bufferBackedDataInput            uscensus2000           true  thrpt    5  19097.221 ± 1378.555  ops/s
RealDataSerializationBenchmark.bufferBackedDataInput            uscensus2000          false  thrpt    5  21454.035 ±  447.602  ops/s
RealDataSerializationBenchmark.streamBackedDataInput              census1881           true  thrpt    5     93.026 ±   18.178  ops/s
RealDataSerializationBenchmark.streamBackedDataInput              census1881          false  thrpt    5     56.886 ±   38.233  ops/s
RealDataSerializationBenchmark.streamBackedDataInput            uscensus2000           true  thrpt    5   7342.377 ± 2062.902  ops/s
RealDataSerializationBenchmark.streamBackedDataInput            uscensus2000          false  thrpt    5   8140.991 ± 1940.242  ops/s
RealDataSerializationBenchmark.viaImmutable                       census1881           true  thrpt    5   1453.401 ± 1117.323  ops/s
RealDataSerializationBenchmark.viaImmutable                       census1881          false  thrpt    5   1482.254 ±  125.103  ops/s
RealDataSerializationBenchmark.viaImmutable                     uscensus2000           true  thrpt    5   6614.448 ±  733.655  ops/s
RealDataSerializationBenchmark.viaImmutable                     uscensus2000          false  thrpt    5   3036.661 ± 1342.902  ops/s

So I'm not sure this can be blamed on the abstraction, but on a bad implementation of the abstraction. In fact, binding to the abstraction allows the user to provide a better implementation, or a file, or a Kryo object... Despite that, I think providing a way to deserialise from a ByteBuffer (as opposed to byte[] as it can take fast paths to convert bytes to wider types) is a very good idea, so long as it doesn't assume the ByteBuffer is array backed.

Thanks to @blacelle  with contributions from @richardstartin, this is now resolved, complete with a new release where one has new functions to deserialize directly from a ByteBuffer, much faster than you could from a stream. (Richard has a blog post coming up which will bring some nuances.)
Now we can move to...
https://github.com/RoaringBitmap/RoaringBitmap/issues/322
I suspect that the same performance issues will arise again during serialization.

Closing.


[scrapers] scrape shtetl-optimized

closing, shtetl-optimized also contains other stuff, also would need to be updated


Add a Gitter chat badge to README.md

mwaylabs/generator-m now has a Chat Room on Gitter
@gruppjo has just created a chat room. You can visit it here: https://gitter.im/mwaylabs/generator-m.
This pull-request adds this badge to your README.md:

If my aim is a little off, please let me know.
Happy chatting.
PS: Click here if you would prefer not to receive automatic pull-requests from Gitter in future.

done with #151


List index out of range error fix

getting this error when attempting to pass my dataset through dtreeviz
IndexError                                Traceback (most recent call last)
 in
----> 1 viz = dtreeviz(clf, Xtrain_set, ytrain_set,
2                 target_name="target",
3                 feature_names=pkg_data.feature_names,
4                 class_names=list(pkg_data.target_names))
5
~\Anaconda3\lib\site-packages\dtreeviz\trees.py in dtreeviz(tree_model, x_data, y_data, feature_names, target_name, class_names, tree_index, precision, orientation, instance_orientation, show_root_edge_labels, show_node_labels, show_just_path, fancy, histtype, highlight_path, X, max_X_features_LR, max_X_features_TD, label_fontsize, ticks_fontsize, fontname, title, title_fontsize, colors, scale)
762
763     n_classes = shadow_tree.nclasses()
--> 764     color_values = colors['classes'][n_classes]
765
766     # Fix the mapping from target value to color for entire tree
IndexError: list index out of range

it can only handle 10 classes I think at the moment. how many do you have?


WIP: Issue #1703 - envoy config dump - data obfuscation

Task Accomplished

struct Filter that describes elements for filtering and hashing of logs
process method for Filter that processes logs based on the specific Filter instance
package level function ProcessLogs that filters logs with a default Filter instance
test cases for the newly added lines

Todos

salting the hash
adding checks on the fields of logs using regex

@jhengy re: lint error, run go mod tidy once and commit the change. then you will see real issues:
pkg/binary/envoy/debug/pii_test.go:72:41: Using the variable on range scope `tc` in function literal (scopelint)
                        actualLogs, actualErr := ProcessLogs(tc.logs, tc.format, tc.containsPII)
                                                             ^
pkg/binary/envoy/debug/pii_test.go:75:24: Using the variable on range scope `tc` in function literal (scopelint)
                        for i, log := range tc.want.logs {
                                            ^
pkg/binary/envoy/debug/pii_test.go:82:9: Using the variable on range scope `tc` in function literal (scopelint)
                                        if tc.containsPII[fieldName] {
                                           ^
pkg/binary/envoy/debug/pii_test.go:90:5: Using the variable on range scope `tc` in function literal (scopelint)
                                tc.want.logs[i] = shell.Join(fields)
                                ^
pkg/binary/envoy/debug/pii_test.go:93:26: Using the variable on range scope `tc` in function literal (scopelint)
                        errorDiff := cmp.Diff(tc.want.err, actualErr)
                                              ^
pkg/binary/envoy/debug/pii_test.go:94:25: Using the variable on range scope `tc` in function literal (scopelint)
                        logsDiff := cmp.Diff(tc.want.logs, actualLogs)
                                             ^
pkg/binary/envoy/debug/pii_test.go:91:5: commentFormatting: put a space between `//` and comment text (gocritic)
                                //tc.want.logs[i] = "'" + log + "'" // adding single quote to the test case
                                ^
pkg/binary/envoy/debug/pii.go:60: G104: Errors unhandled. (gosec)
        h.Write([]byte(s)) //nolint: errcheck
pkg/binary/envoy/debug/pii_test.go:25: line is 544 characters (lll)
        var FORMAT_STRING string = "[%START_TIME%] \"%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%\" %RESPONSE_CODE% %RESPONSE_FLAGS% \"%DYNAMIC_METADATA(istio.mixer:status)%\" \"%UPSTREAM_TRANSP
ORT_FAILURE_REASON%\" %BYTES_RECEIVED% %BYTES_SENT% %DURATION% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% \"%REQ(X-FORWARDED-FOR)%\" \"%REQ(USER-AGENT)%\" \"%REQ(X-REQUEST-ID)%\" \"%REQ(:AUTHORITY)%\" \"%UPSTREAM_HO
ST%\" %UPSTREAM_CLUSTER% %UPSTREAM_LOCAL_ADDRESS% %DOWNSTREAM_LOCAL_ADDRESS% %DOWNSTREAM_REMOTE_ADDRESS% %REQUESTED_SERVER_NAME%\n"
pkg/binary/envoy/debug/pii_test.go:32: line is 442 characters (lll)
        var VALID_LOG string = `[2019-09-05T17:53:36.908Z] "GET /productpage HTTP/1.1" 200 - "-" "-" 0 5183 47 46 "<IP_ADDRESS>" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.
3809.132 Safari/537.36" "4273c09c-2534-962a-b7da-04a04cf39e88" "<IP_ADDRESS>" "<IP_ADDRESS>:9080" inbound|9080|http|productpage.default.svc.cluster.local - <IP_ADDRESS>:9080 <IP_ADDRESS>:0 outbound_.9080_._.productpa
ge.default.svc.cluster.local`
pkg/binary/envoy/debug/pii.go:24:1: comment on exported function `ProcessLogs` should be of the form `ProcessLogs ...` (golint)
// process the array of logs by filtering it using format and containsPII, Hash the resulting array of logs and return the final array.
^
pkg/binary/envoy/debug/pii.go:57:1: comment on exported function `Hash` should be of the form `Hash ...` (golint)
// TODO: salt the Hash
^
pkg/binary/envoy/debug/pii_test.go:33:6: don't use ALL_CAPS in Go names; use CamelCase (golint)
        var LOG_WITH_ONE_FIELD string = `[2019-09-05T17:53:36.908Z]`
            ^
golangci-lint failed.
One or more linters failed.

Otherwise the code looks great, this last piece is a stylistic one that keeps things testable but hides a bunch of the details from a typical user just trying to consume the interface.

Ah, sorry - I lied a little bit: there's one other change we'll make in addition to introducing a struct to hang our state off of. We'll want to introduce a default, package level Process method. So for users there's two APIs: pii.NewFilter().Process(logs) and pii.Process(logs). That should use the default PII filter and hash function, as well as the default Istio Envoy format string; 99% of the time that's all we'll need and the user's API is dead simple and can't be misused.

@jhengy Just fyi, we should avoid force-pushing in this project? If it's on purpose, ignore my comment.

@jhengy Just fyi, we should avoid force-pushing in this project? If it's on purpose, ignore my comment.

Yes, I did it on purpose after I did a git rebase on my local branch. The normal git push does not work after I rebase my local branch to master in an attempt to solve the branch out of date error on this PR

Oh, and one other thing: let's go ahead and introduce our pii package. Since this is private to our tool, it should go in the pkg directory (this is a common go convention: stuff for others to consume goes in a top level package, stuff that's library code for this codebase is tucked into pkg). So we should have a new directory getenvoy/pkg/pii.


Need help to reinstall the bot

Hello there, I feel like my issue may be easy to fix for you guys but I'm kind of a noob so be easy on me.
So for the story, I was hosting Nadeko without problems, and that for months, but this morning I decided to check if there were any updates for the bot since I never updated it, and there were !
So I decided to update it of course, but I didnt do ".die" to kill it before hand (idk if that changes anything). And after the update was done, I tried to restart the bot, but the console was saying something like "error 400 : Bad request". I tried restarting it again but it didnt change anything, so I decided to try to uninstall redis and reinstall but now it just said "Images already exist in redis" or something like that.
I kept trying to uninstall stuff and everything and eventually decided to completely delete the NadekoBot folder, the NadekoUpdater and redis to start again from scratch, but here is my problem now, I downloaded the NadekoUpdater again and it still considers that it is on version 2.28.4 even though I unistalled everything. I can install redis no problem, but cannot click the big update button to reinstall the NadekoBot application, and it is nowhere to be found in my Pc either.
So here is my question, is there any way for me to "reset" the NadekoUpdater so I can start again from scratc as if I never hosted the bot in the first place, or have a way to download the NadekoBot files from elsewhere ?
Thanks for your time

How to reinstall Nadeko on Windows?

Close Nadeko completely.
Go to Control Panel > Programs > Programs and Features > Uninstall or change a program
Uninstall NadekoBot version
Open Nadeko Updater and click Update

If Nadeko Updater still detects the program as installed after removing it:

Close Nadeko Updater.
Open regedit and delete this: HKEY_LOCAL_MACHINE\SOFTWARE\NadekoBot
Open Nadeko Updater and try again.

Okay now it doesnt detect the version anymore, I cant believe you saved me so fast haha
Only problem now is that it tells me that I check for updates too often and the "Update" area is grayed, I still can't install, does that go away if I wait for a bit ?

So, it's still detecting the program as installed?

No its not, theres just this pop-up telling me to "chill a bit"

Then I we must chill a bit. Try restarting your pc.

I will, thanks for the help !

If you would, close this when ever you get it to stop telling you to chill out

@Nisouc  If you got it to work, please close this. Thank you.

Oh yeah Im sorry I was busy creating all the bot reactions again and I forgot, thanks for the help again and have a good day !

You have a good day too.


Prepare release 5.12.0

Overview of change:
Changes required to release 5.12.0.
Steps for me following this:

Create a release on github
Publish the release locally

Is there anything you'd like reviewers to focus on?
This should just work right? I haven't specifically smoke tested it - thoughts? Suppose you could install and build it locally, then force it to use your local version through yarn link magic to test? Our tests in repo are pretty thorough, but not sure they will cover non-rule bugs added.

Oops a lot of these were generated, going to take a closer look at the list and fix up your catches.

@JoshuaKGoldberg got them all, I think. This should be good to approve now, but I probably won't have time to run the post-merge steps until tomorrow morning (london time), so lets wait until then to merge.


bug: Telescope Trouble keybindings don't work

Did you check docs and existing issues?

[X] I have read all the LazyVim docs
[X] I have searched the existing issues of LazyVim
[X] I have searched the existing issues of plugins related to this issue

Neovim version (nvim -v)
0.9.4
Operating system/version
MacOS 14.1
Describe the bug
Telescope has predefined trouble keybindings, which used to work but now they don't. Instead, pressing, for example, <c-t> opens up the selected file.
Steps To Reproduce

<leader> s + w
Press <c-t> to send everything to Trouble

Expected Behavior
Search results are open in Trouble
Repro
Nothing's changed from default settings here.

It works for me like it should. <c-t> sends the results to Trouble. Maybe something to do with your terminal keymaps conflict?

@dpetka2001 thank you for looking into it. By some reason telescope provides the following keymaps

I'll investigate it further.

Maybe you changed the keybinding yourself in your personal Telescope spec?

I found that turning off my telescope changes fixes the issue, however I only disabled some of keybindings. I presume the pickers customization is to blame.

Since this is not a Lazyvim issue, maybe close it and open a new one in the Discussions for further help?

You are right, thank you.

You are right, thank you.

I was changing telescope's opts wrong. The working solution would be the following:
config = function(_, opts)
  local updates = {...}
      require("telescope").setup(vim.tbl_deep_extend("keep", updates, opts))
     end,


[Dict] Introduce dict component

ref #120

changes:
Features:

added Psl\Dict\associate
added Psl\Dict\drop
added Psl\Dict\drop_while
added Psl\Dict\equal
added Psl\Dict\filter
added Psl\Dict\filter_keys
added Psl\Dict\filter_with_key
added Psl\Dict\flatten
added Psl\Dict\flip
added Psl\Dict\from_entries
added Psl\Dict\from_iterable
added Psl\Dict\from_keys
added Psl\Dict\group_by
added Psl\Dict\map
added Psl\Dict\map_keys
added Psl\Dict\map_with_key
added Psl\Dict\merge
added Psl\Dict\partition
added Psl\Dict\partition_with_key
added Psl\Dict\pull
added Psl\Dict\pull_with_key
added Psl\Dict\reindex
added Psl\Dict\select_keys
added Psl\Dict\slice
added Psl\Dict\sort
added Psl\Dict\sort_by
added Psl\Dict\sort_by_key
added Psl\Dict\take
added Psl\Dict\take_while
added Psl\Dict\unique
added Psl\Dict\unique_by

Deprecations:

deprecated Psl\Arr\drop ( use Psl\Dict\drop instead )
deprecated Psl\Arr\drop_while ( use Psl\Dict\drop_while instead )
deprecated Psl\Arr\equal ( use Psl\Dict\equal instead )
deprecated Psl\Arr\filter ( use Psl\Dict\filter instead )
deprecated Psl\Arr\filter_with_key ( use Psl\Dict\filter_with_key instead )
deprecated Psl\Arr\filter_keys ( use Psl\Dict\filter_keys instead )
deprecated Psl\Arr\flatten ( use Psl\Dict\flatten instead )
deprecated Psl\Arr\flip ( use Psl\Dict\flip instead )
deprecated Psl\Arr\group_by ( use Psl\Dict\group_by instead )
deprecated Psl\Arr\map ( use Psl\Dict\map instead )
deprecated Psl\Arr\map_with_key ( use Psl\Dict\map_with_key instead )
deprecated Psl\Arr\map_keys ( use Psl\Dict\map_keys instead )
deprecated Psl\Arr\merge ( use Psl\Dict\merge instead )
deprecated Psl\Arr\partition ( use Psl\Dict\partition instead )
deprecated Psl\Arr\select_keys ( use Psl\Dict\select_keys instead )
deprecated Psl\Arr\slice ( use Psl\Dict\slice instead )
deprecated Psl\Arr\sort_by_key ( use Psl\Dict\sort_by_key instead )
deprecated Psl\Arr\sort_with_keys ( use Psl\Dict\sort_with_keys instead )
deprecated Psl\Arr\sort_with_keys_by ( use Psl\Dict\sort_with_keys_by instead )
deprecated Psl\Arr\take ( use Psl\Dict\take instead )
deprecated Psl\Arr\take_while ( use Psl\Dict\take_while instead )
deprecated Psl\Arr\unique ( use Psl\Dict\unique instead )
deprecated Psl\Arr\unique_by ( use Psl\Dict\unique_by instead )
deprecated Psl\Iter\drop ( use Psl\Dict\drop instead )
deprecated Psl\Iter\drop_while ( use Psl\Dict\drop_while instead )
deprecated Psl\Iter\slice ( use Psl\Dict\slice instead )
deprecated Psl\Iter\take ( use Psl\Dict\take instead )
deprecated Psl\Iter\take_while ( use Psl\Dict\take_while instead )
deprecated Psl\Iter\reindex ( use Psl\Dict\reindex instead )
deprecated Psl\Iter\pull ( use Psl\Dict\pull instead )
deprecated Psl\Iter\pull_with_key ( use Psl\Dict\pull_with_key instead )
deprecated Psl\Iter\to_array_with_keys ( use Psl\Dict\from_iterable instead )
deprecated Psl\Iter\map ( use Psl\Dict\map instead )
deprecated Psl\Iter\map_with_key ( use Psl\Dict\map_with_key instead )
deprecated Psl\Iter\map_keys ( use Psl\Dict\map_keys instead )
deprecated Psl\Iter\from_entries ( use Psl\Dict\from_entries instead )
deprecated Psl\Iter\from_keys ( use Psl\Dict\from_keys instead )
deprecated Psl\Iter\product ( no replacement provided )
deprecated Psl\Iter\filter ( use Psl\Dict\filter instead )
deprecated Psl\Iter\filter_with_key ( use Psl\Dict\filter_with_key instead )
deprecated Psl\Iter\filter_keys ( use Psl\Dict\filter_keys instead )
deprecated Psl\Iter\flip ( use Psl\Dict\flip instead )
deprecated Psl\Iter\flatten ( use Psl\Dict\flatten instead )
deprecated Psl\Iter\merge ( use Psl\Dict\merge instead )

Pull Request Test Coverage Report for Build 569781449

217 of 262 (82.82%)  changed or added relevant lines in 62 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-2.0%) to 97.99%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/Psl/Dict/merge.php
0
2
0.0%

src/Psl/Dict/from_entries.php
0
4
0.0%

src/Psl/Dict/from_keys.php
0
4
0.0%

src/Psl/Dict/pull.php
0
4
0.0%

src/Psl/Dict/pull_with_key.php
0
4
0.0%

src/Psl/Dict/reindex.php
0
4
0.0%

src/Psl/Dict/partition.php
0
7
0.0%

src/Psl/Dict/partition_with_key.php
0
7
0.0%

src/Psl/Dict/associate.php
0
9
0.0%

Totals

Change from base Build 564344617:
-2.0%

Covered Lines:
2242

Relevant Lines:
2288

💛  - Coveralls


CUSTOM SMTP not working

Bug report
Describe the bug
I am getting error with sending magic links (and password recovery) emails when trying to use custom smtp

The issue is happening on both on my next.js app and also supabase web ui.

Note:
Gmail less secure access has been activated. I could send emails via python's smtplib library.
To Reproduce
Enter SMTP details for Gmail

When right clicking on user to "send magic link" it fails. This also happens to any other actions

Expected behavior
Receive an email from custom email for magic links and/or password recovery
Screenshots
If applicable, add screenshots to help explain your problem.
System information

OS: macOS
Browser: Chrome
Version of supabase-js: 1.3.2
Version of Node.js: v14.17.3

Additional context
N/A

Hey @gpaw, hmm i tried reproducing this error on my end and can't seem to do so. Could you check the "Logs" section at the bottom of the "Authentication" -> "Settings" page and see what the issue is? It could be an issue with your SMTP custom config.

@kangmingtay
So the logs does not produce any recently actioned activity, maybe verbose needs to be turned on? idk
But I followed your thread and I did enabled "less secure app" (which is a must). But what I didn't do was "http://www.google.com/accounts/DisplayUnlockCaptcha " - this fixed the issue!
Mate you went above and beyond to help me on this issue, thank you so much!
Issue resolved.


Updated modalFactory React.createClass() to ES6

Replaced old React Component Class creation method with ES6 to prevent this deprecated error in console:
"Warning: A Component: React.createClass is deprecated and will be removed in version 16. Use plain JavaScript classes instead. If you're not yet ready to migrate, create-react-class is available on npm as a drop-in replacement."
Link: https://reactjs.org/docs/components-and-props.html

This should be merged asap.

Good point, but it seems like you've applied your own code-style rules to the file and removed all ; from the code. So, it's really hard to see what you've actually changed.
I've added create-react-class component in my pull request, you can check it out: #49


SL Review: issue with the ministry names in reporting

Issue: The current ministry names sent to ODS in SL Review submissions lack spaces, which creates compatibility issues with the reporting team's ministry name table structure. They require ministry names with spaces (e.g., my ministry).

The problem occurs when a user searches for a senior leader and then changes their ministry from the drop-down menu. In such cases, the system saves the key of the selected ministry, which does not contain spaces, into the database and later sends it to the ODS. This issue does not arise if the user does not modify the auto-populated ministry name that appears when the senior leader is searched.
To address the problem, I changed the value property under the ministry name field to label. This ensures that the displayed ministry name with spaces is used, resolving the discrepancy between the displayed name and the stored key in the database.

@Stella-Archer
The issue was fixed in all environments. It's ready to be tested.
@mikeshi-bcgov
Do you mind, manually fixing the existing records in ODS that have the space issue in their ministry names? thank you

Thanks  @iman-jamali-fw @ayushdamani could you start testing the in the TEST environment when you get in? Thx!

Tested and confirmed working ✅
Selected Meghan and manually changed the ministry.

thx will move it to PO

@mikeshi-bcgov
Did you have a chance to fix the records with ministry name issues?
Also, could you let us know if we need to coordinate with the ODS team if we plan to make a change/add a ministry name to the SL form? Is there a table on your end that maps the ministry names?

org hierarchy is maintained by DI team (Steven Jones' team) - submit a ticket to this area as needed

One option for consideration for manage ministry names or department name changes is to pass the ID to ODS.  If you are leveraging the Datamart_Telework_meta_dept_org_levels, there is a departmentID that can be used.
Doing this however will impact how reporting is done today and it will change automatically as name changes.

Iman noted we could test with a submission and see if the data is correct on ODS side - fix is on dev and on test.  Can be tested on Test and then tested with the ODS to see that the data is confirmed.

@mikeshi-bcgov @T854236
I sent two submissions on the TEST environment with application Ids of 2710 and 2711. Could you confirm the ministry name/branch/division is correctly mapped to the ODS table (for reporting)? thank you
If it looks good, @Andrew-Vargas-bcgov can move the fix to PROD.

@iman-jamali-fw
Looks good.

yes- ready to go

tested and approved


Incorrect TopCount returned result

Steps to reproduce
Hi,
Referring to the attached notebook customer-churn.zip.
I have more than 1000 records:

I want the top 10 customers with the highest monthlyCharges:

But only 8 records got returned.

Environment

atoti: 0.4.1
Python: 3.8.3
Operating System: win32

I cannot run your examples as it uses dependency such as utils which is missing : from utils import data_utils, prediction
Can you please provide all the required  files or a more minimal example ?
This behavior can be caused by another filter removing some of the top 10 CustomerID, can you check if there is another filter ?

Sorry, the attached utils:
utils.zip

By removing all other filters, it returns me 6 records instead of the 10.

Looks like an issue in ActivPivot. I don't always have the issue but I can reproduce it with this small example:
import atoti as tt
import pandas as pd
df = pd.DataFrame({
    "CustomerID": ['7590-VHVEG', '5575-GNVDE', '3668-QPYBK', '7795-CFOCW', '9237-HQITU', '9305-CDSKC', '1452-KIOVK', '6713-OKOMC', '7892-POOKP', '6388-TABGU', '9763-GRSKD', '7469-LKBCI', '8091-TTVAX', '0280-XJGEX', '5129-JLPIS', '3655-SNQYZ'],
    "MonthlyCharges": [ 29.85,  56.95,  53.85,  42.3,  70.7,  99.65,  89.1,  29.75, 104.8,  56.15,  49.95,  18.95, 100.35, 103.7, 105.5, 113.25]
})
session = tt.create_session()
customer_store = session.read_pandas(
    df, keys=["CustomerID"], store_name="customer_store", partitioning="hash6(CustomerID)"
)
cube = session.create_cube(customer_store, "customer_cube")
cube.visualize()

Then make a pivot table with the following MDX query:
SELECT
  NON EMPTY {
    [Measures].[MonthlyCharges.SUM],
    [Measures].[contributors.COUNT]
  } ON COLUMNS,
  NON EMPTY Hierarchize(
    DrilldownLevel(
      [Hierarchies].[CustomerID].[ALL].[AllMember]
    )
  ) ON ROWS
  FROM (
    SELECT
    TopCount(
      Filter(
        [Hierarchies].[CustomerID].Levels(
          1
        ).Members,
        NOT IsEmpty(
          [Measures].[MonthlyCharges.SUM]
        )
      ),
      10,
      [Measures].[MonthlyCharges.SUM]
    ) ON COLUMNS
    FROM [customer_cube]
  )
  CELL PROPERTIES VALUE, FORMATTED_VALUE, BACK_COLOR, FORE_COLOR, FONT_FLAGS

This is all available in this notebook:
topcount.zip
I will open an internal issue.

I think https://github.com/atoti/atoti/issues/99 is probably related as well.

Thanks for reporting, this will be fixed in next release.
Best


bold rewards

@tzarebczan can you send me a screenshot of Fresh Unclaimed Reward in Fresh App Installer, i will put on this faq

No need for that screenshot, it's fairly obvious how to get there.

thank you sir


[Fix](inverted index) fix memory leak when closing DorisCompoundDirectory::FSIndexInput

Proposed changes
Issue Number: close #xxx
Problem summary
This patch fixes a memory leak issue in the DorisCompoundDirectory::FSIndexInput class when closing the input. Previously, the code was using a dynamically allocated mutex, which wasn't getting deleted properly in certain cases, leading to a memory leak. The patch modifies the code to use a direct instance of doris::Mutex instead of a pointer, which simplifies the handling and avoids the leak. The locking mechanism and other related logic have been updated accordingly.
Affected files:
be/src/olap/rowset/segment_v2/inverted_index_compound_directory.cpp
be/src/olap/rowset/segment_v2/inverted_index_compound_directory.h
Checklist(Required)

[ ] Does it affect the original behavior
[ ] Has unit tests been added
[ ] Has document been added or modified
[ ] Does it need to update dependencies
[ ] Is this PR support rollback (If NO, please explain WHY)

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run buildall

run buildall

run buildall

run buildall


New Package Request: pigz

Package name and purpose
The parallel gzip implementation is useful to speed up the decompression time spent for large container image imports/exports.

Package upstream repo: https://github.com/madler/pigz
Gentoo ebuild: https://packages.gentoo.org/packages/app-arch/pigz

Impact of adding this package to the Flatcar OS image

The package improves on the following core values:

[ ] Secure by default
[ ] Always up to date
[x] Improve container experience
[x] Operate at scale / automation / telemetry

The package will increase the image size by: [estimated] 128 KB
How might this package increase the attack surface:

not really because pigz is well maintained

Benefits of adding this package
The speedup is quite large and we make use of pigz in the Flatcar build pipeline to import huge Docker images faster, currently we do this by building and pulling a custom Docker image that provides pigz but that means we rely on an external container registry to be reachable which is sometimes not the case (at the very moment for us this is due to downtime but I could also imagine that others have constrains like not wanting to pull external binaries for supply chain security)
Additional information

Just to be sure, this issue is about making pigz a part of production image, right? Because we already have pigz in SDK.

Yes

Will be available in the next alpha release.


Fix Zone creation

settings_for_resource shouldn't blow up if the instance isn't saved yet
It should return the settings from the parent resource instead.
Only attempt to reload the record if the record is persisted
https://bugzilla.redhat.com/show_bug.cgi?id=1509172
@bdunne @Fryguy

Gaprindashvili backport details:
$ git log -1
commit e4363530c0ae9e4a48b6e87e8b333b4d196a5f54
Author: Jason Frey<EMAIL_ADDRESS>Date:   Fri Nov 3 15:23:54 2017 -0400

    Merge pull request #16391 from chrisarcand/bz-1509172

    Fix Zone creation
    (cherry picked from commit f8fbc91ab6624f3ebf9a2a4ef0a1676b8d2cf1f5)

    https://bugzilla.redhat.com/show_bug.cgi?id=1509452


Remove global vars from webapi package.

Just code refactoring here, no functional changes. Removing global vars from the webapi package and instead instantiating a Server to hold those variables.

Contributes to #339


ESP-RZXe Support

Hi,
i own an ESP-RZXe and got following Log Output:
DEBUG: Received a response from Rainbird controller DEBUG: {"serialNumber":"0000000000000000","_type":"SerialNumberResponse"} DEBUG: Received a response from Rainbird controller DEBUG: Received a response from Rainbird controller DEBUG: {"pageNumber":"00","activeStations":"00000000","activeZones":[0,0,0,0],"_type":"CurrentStationsActiveResponse"} DEBUG: Received a response from Rainbird controller DEBUG: {"modelID":"0003","protocolRevisionMajor":"02","protocolRevisionMinor":"09","_type":"ModelAndVersionResponse"} DEBUG: Received a response from Rainbird controller DEBUG: {"pageNumber":"00","setStations":"3F000000","_type":"AvailableStationsResponse"}
Stefan

@iondarie, @StofLE, can you guys try the latest beta?

@donavanbecker it seems to work, ill try to do some more tests and come back

@iondarie, @StofLE, can you guys try the latest beta?

Sure, could you pleas

Clicked wrong...
Seems to work! :-)

v0.1.0 is out. When I release v1.0.0 You will have to change your config a little and clear cache on your device to re add it. so don't setup to much automation. Currently you can only add 1 controller in, with v1.0.0 (if you have multiple controllers on one network, rare but possible) you will be able to config more than 1.

@donavanbecker thank you, any chance that the valves could be exposed as sensors? So we can receive notifications when the sprinkler zones’s start

@iondarie, what type of sensor are you thinking of? Perhaps a leak sensor?

@iondarie, what type of sensor are you thinking of? Perhaps a leak sensor?

We could make this a config option. Let's do this for 1.1.0, we want to get v1.0.0 out first then we can look at this.

@donavanbecker thank you, any chance that the valves could be exposed as sensors? So we can receive notifications when the sprinkler zones’s start

We will track this with #8.

v1.0.0 released


pow tests fail with -march=native -mtune=native flags and g++

Description
When adding CXXFLAGS += -march=native -mtune=native the mix tests for pow fail due to all tests resulting in zero gradients, but only when compiling with g++. The tests compile and pass under clang.
No other tests (so far) fail, so it looks like it might be some weird interplay between the binary vectorisation framework and the pow function specifically.
I've replicated the failures on two linux machines, both with gcc 9.30 and clang 10
Current Version:
v3.3.0

Can confirm on ubuntu 20.04 with g++-10. It looks like a near zero problem?
./test/unit/math/expect_near_rel.hpp:33: Failure
The difference between x1 and x2 is 3.8049220956170699e-07, which exceeds tol_val, where
x1 evaluates to 3.8049220956170699e-07,
x2 evaluates to 0, and
tol_val evaluates to 1e-08.
expect_near_rel_finite in: expect_near_rel; require items x1(i) = x2(i): gradient() grad

./test/unit/math/expect_near_rel.hpp:33: Failure
The difference between x1 and x2 is 1.1190947340050207e-07, which exceeds tol_val, where
x1 evaluates to 1.1190947340050207e-07,
x2 evaluates to 0, and
tol_val evaluates to 1e-08.
expect_near_rel_finite in: expect_near_rel; require items x1(i) = x2(i): gradient() grad

./test/unit/math/expect_near_rel.hpp:33: Failure
The difference between x1 and x2 is 7.3171578761866729e-08, which exceeds tol_val, where

g++ info:
g++ -v
Using built-in specs.
COLLECT_GCC=g++-10
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/10/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:amdgcn-amdhsa:hsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 10-20200411-0ubuntu1' --with-bugurl=file:///usr/share/doc/gcc-10/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-10 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none,amdgcn-amdhsa,hsa --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 10.0.1 20200411 (experimental) [master revision bb87d5cc77d:75961caccb7:f883c46b4877f637e0fa5025b4d6b5c9040ec566] (Ubuntu 10-20200411-0ubuntu1)

Do they pass for you under Clang or was that just a quirk of my system?

Figured this out (near-zero problem like Steve suspected). In the tests, the failures are caused by the very last pair of variables. This means that errors can be reproduced with just:
  Eigen::VectorXd in1(3);
  in1 << 5.2;
  Eigen::VectorXd in2(3);
  in2 << 6.7;
  stan::test::expect_ad_vectorized_binary(f, in1, in2);

To test whether this was an issue with the framework, I put together a simple test with var and fvar for this combination of variables (code at end), and all tests passed. Based on this, I would assume that the issue is actually in the finite-differencing not having a great resolution around zero (or a great resolution for arguments that result in near-zero). There is likely some math implementation difference between clang++ and g++ that gives slightly better resolution in this case (did not dig down that far though).
Code for testing values/adjoints:
#include <stan/math/fwd.hpp>
#include <stan/math.hpp>
#include <gtest/gtest.h>
#include <test/unit/util.hpp>

TEST(MathMatrixRevMat, EltPow) {
  using stan::math::vector_v;
  using stan::math::var;

  vector_v l1(1);
  l1 << 5.2;
  vector_v l2(1);
  l2 << 6.7;

  var com_l1 = l1[0];
  var com_l2 = l2[0];

  vector_v out = stan::math::pow(l1,l2);
  stan::math::var com_out = stan::math::pow(com_l1,com_l2);

  EXPECT_FLOAT_EQ(out[0].val(), com_out.val());

  com_out.grad();
  out[0].grad();

  EXPECT_FLOAT_EQ(l1[0].adj(), com_l1.adj());
  EXPECT_FLOAT_EQ(l2[0].adj(), com_l2.adj());
}

TEST(AgradFwdMatrix, EltPow) {
  using stan::math::vector_fd;

  vector_fd l1(1);
  l1 << 5.2;
  vector_fd l2(1);
  l2 << 6.7;

  stan::math::fvar<double> com_l1 = l1[0];
  stan::math::fvar<double> com_l2 = l2[0];

  vector_fd out = stan::math::pow(l1, l2);
  stan::math::fvar<double> com_out = stan::math::pow(com_l1, com_l2);

  EXPECT_FLOAT_EQ(out[0].val(), com_out.val());
  EXPECT_FLOAT_EQ(out[0].d_, com_out.d_);
}

Given this, I'll just open a PR to change these test values to something that the finite diffs can handle a bit better

Word much appreciated! I forgot about this, as I was about to fall asleep tonight I went, "omg does exponential still not work???". Lol glad to see it was just a testing thing and not an actual scary thing

Ha yeah I had the panic thought that the vectorisation framework had been broken all along, was relieved to find it wasn't the case


Close outstanding .NET DPG Model design questions

Context
This issue tracks outstanding model design questions that aren't tracked in other GH issues in the .NET DPG - Model Generation project.
Questions

[ ] Should models use #nullable disable?
[ ] Should strings passed to a model constructor be checked for both null and empty?
[ ] If a @doc decorator provides a description of a property, should the C# doc comment for a getter say "Gets the <@doc value>" or just output the doc value?
[ ] What is the mapping from Cadl built-in types to C# types?
[ ] Do Cadl built-in types let us represent everything we need in Azure and if so, how?
[ ] How does @format work?
[ ] Do we need to do anything special to use the @key Cadl decorator?
[ ] How do we handle @secret?
[ ] Do we handle all the built-in decorators?
[ ] Should generated models always go in a .Models namespace?
[ ] Can we represent all shapes of client/subclient hierarchy?
[ ] Should round-trip model properties have setters?
[ ] How will we map built-in cadl-azure types (e.g. cadl-azure-core) to .NET Core/other types?

Working model taxonomy:
(Or, model considerations/test cases)

Property of model

input/output/round-trip

Property of model property

primitive/collection/dictionary/model type
value type/reference type
required/optional
nullable/non-nullable

@archerzz, could you follow up on the remaining items?  Thanks


Evershop

What do you think of Evershop? Should it be included in this repo?

Hell yeah! Can you submit a PR?


Scanning folders for Git repositories and The extension page is always loading

VS Code Version: 1.89.1 (user setup)
OS Version: Windows_NT x64 10.0.19042
Node.js: 18.18.2
Electron: 28.2.8
ElectronBuildId: 27744544
V8: 12.0.267.19-electron.0

I don't know how to reproduce it.
This happens when my computer is not turned off for 2 to 3 days.
The source control tab and the extension tab are always loading.
The source control tab always shows: Scanning folders for Git repositories...
The extension tab: It is always loading, and the installed list is empty, nothing is displayed.

Thanks for filing this issue. This looks like an issue that was recently tracked down and fixed. The issue is fixed in the latest VS Code Insiders release and will be included in the next VS Code release that is planned for the second half of next week.


fix: make danger zone more dangerous

Description
This pr makes danger zone more "dangerous"

What type of PR is this? (check all applicable)

[ ] 🍕 Feature
[ ] 🐛 Bug Fix
[ ] 📝 Documentation Update
[x] 🎨 Style
[ ] 🧑💻 Code Refactor
[ ] 🔥 Performance Improvements
[ ] ✅ Test
[ ] 🤖 Build
[ ] 🔁 CI
[ ] 📦 Chore (Release)
[ ] ⏩ Revert

Related Tickets & Documents

Closes #2916
Mobile & Desktop Screenshots/Recordings

Steps to QA

Added to documentation?

[ ] 📜 README.md
[ ] 📓 docs.opensauced.pizza
[ ] 🍕 dev.to/opensauced
[ ] 📕 storybook
[x] 🙅 no documentation needed

[optional] Are there any post-deployment tasks we need to perform?
[optional] What gif best describes this PR or how it makes you feel?

@isabensusan can you verify what design you want to see here?

@isabensusan can you verify what design you want to see here?

FWIW the danger zone is modeled after GitHub's

What is represented currently in this PR could lean into our existing design system to create or something cohesive.
my suggestion
Put items into grey box with distinct sections.
Make it make sense

@malezjaa thank you for your contribution! I see that you've based the design on GitHub's danger zone, but it'd be great if we could get it closer to what we have on our current design system.
I threw this quick mock together, essentially:

Move the "transfer to other workspace" section to the same gray box as the "delete" section
Make the danger zone title font-semibold text-lg text-red-600

Hi I followed your comment. I also adjusted contributor insights page


Two questions: 1.Release entity vocab's wikipedia pageid? 2. Does [mask] occupy bert's 512 input?

Right now, some titles in entity vocab can not align to a unique wikipedia paged or wikidata entity id: some are missing, and some titles refer to a same pageid. Can you release the mapping between entity vocab's title and wikipedia pageid / wikidata entity id?
It seems that [mask]s for span representation don't occupy bert's 512 input? For example, I have a sequence with 512 tokens, and I want to use LUKE to extract 10 spans, then I can input 512 tokens+10 [mask], rather than 502 tokens + 10 [mask]? (as long as [mask] position embedding is correctly aligned to the 10 spans?)

Hi, thank you for opening the issue.

Right now, some titles in entity vocab can not align to a unique wikipedia paged or wikidata entity id: some are missing, and some titles refer to a same pageid. Can you release the mapping between entity vocab's title and wikipedia pageid / wikidata entity id?

LUKE does not use page_id since entity titles can also be used as unique identifiers. To resolve entity titles to page ids, did you use the December 2018 version of Wikipedia dump? I think entity titles can be resolved to page ids without issues if you use the same Wikipedia dump as the one used to create the entity vocabulary.

It seems that [mask]s for span representation don't occupy bert's 512 input? For example, I have a sequence with 512 tokens, and I want to use LUKE to extract 10 spans, then I can input 512 tokens+10 [mask], rather than 502 tokens + 10 [mask]? (as long as [mask] position embedding is correctly aligned to the 10 spans?)

Yes, LUKE takes two kinds of inputs: word_ids and entity_ids and there are corresponding two kinds of maximum lengths: max_seq_length and max_entity_length.

Hello, there are two dumps in December 2018: 20181201 and 20181220, which one?

Thanks for the reply! I haven't noticed that there are two Wikipedia dumps. We used the December 20, 2018 version of Wikipedia dump.

Here is an aligned entity vocab, I align title with wikpedia pageid and wikidata entityid, wish you can release it with checkpoint.
aligned_entity_vocab.csv
Alignment is an important and dirty work for real-world system, but entity titles are not stable. In some bad case, the historical wiki dump version is not available, and current version may also disappear in the future. So I advise entity researchers releasing aligned title-wikipedia_pageid/wikidata_entity id :)

Although I use your dump version, some entities still miss, fortunately, not too much:
175999	Railway speed record	99
230697	TBWA\Chiat\Day	76
304473	Romanian Footballer of the Year (Gazeta Sportu...	58
390345	Customs broker	45
397086	Indoor hockey	44
399981	Capaccio Paestum	44
419400	Institute of Race Relations (disambiguation)	42
478537	John Dunn (radio presenter)	36

@dalek-who Thank you so much for sharing the data!!
As this is clearly useful for other researchers, I will add a link to this file to README.
Regarding the missing entities, I confirmed that the Wikipedia database directly built from the December 20, 2018 dump contains these entities. The database was built using build-dump-db command defined here. I check the existence of each entity using the following commands:
>>> from wikipedia2vec.dump_db import DumpDB
>>> db = DumpDB('enwiki_20181220_2.db')
>>> db.get_paragraphs('Railway speed record')[:3]
[<Paragraph Rocket sled that achieved Mach 8.5...>,
 <Paragraph An L0 Series trainset, holding the non-conventiona...>,
 <Paragraph TGV 4402 (operation V150) reaching 574.8 km/h (357...>]

I am not sure why, but it may need further investigation.

@ikuyamada Another personal request: can you provide a full entity counting result? Since entity vocab only contain about top 10% entity. (Even unaligned is ok).

@dalek-who I uploaded the entity vocabulary file containing all entities with at least one inbound hyperlink here :)

@ikuyamada Does count means how many mentions (anchor text) links to this entity in whole wikipedia? And as a result, if an entity is not mentioned, it's excluded out of the full vocab?

@ikuyamada Does "count" means how many mentions (anchor text) links to this entity in whole wikipedia? And as a result, if an entity is not mentioned, it's excluded out of the full vocab?

@dalek-who Yes, the count represents the number of hyperlinks pointing to the entity, and an entity is excluded if there is no hyperlink.

@ikuyamada Do you specially deal with entity imbalance problem in pretraining ?

@dalek-who No, similar to the BERT’s masked language model, we simply predict every entity from the all entities in the vocabulary during pretraining.


Fix upload of Nx1 tensors in upload endpoint

Remedies https://github.com/trustyai-explainability/trustyai-explainability/issues/430
MM can return an N element tensor as [a, b, ..., n] and mark its shape as [n, 1]. This case was not correctly supported, this PR ensures that the correct parsing codec is used.
Many thanks for submitting your Pull Request :heart:!
Please make sure that your PR meets the following requirements:

[ ] You have read the contributors guide
[ ] Pull Request title is properly formatted: FAI-XYZ Subject
[ ] Pull Request title contains the target branch if not targeting main: [0.3.x] FAI-XYZ Subject
[ ] Pull Request contains link to the JIRA issue
[ ] Pull Request contains link to any dependent or related Pull Request
[ ] Pull Request contains description of the issue
[ ] Pull Request does not include fixes for issues other than the main ticket

/test trustyai-explainability-e2e


GEODE-393: Providing cache for FunctionContext

Added the ability to pass a Cache to a function context as well as tests to check for this.
Thank you for submitting a contribution to Apache Geode.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

[x] Has your PR been rebased against the latest commit within the target branch (typically develop)?

[x] Is your initial contribution a single, squashed commit?

[ ] Does gradlew build run cleanly?

[x] Have you written or updated unit tests to verify your changes?

[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and
submit an update to your PR as soon as possible. If you need help, please send an
email to<EMAIL_ADDRESS>

@ladyVader @nabarunnag @boglesby @jhuynh1 @upthewaterspout @gesterzhou


feat(lro): honour retry-after and enlarge default polling interval

Description
Problem
This is found during diagnosis of #1653 Currently we have 2 issues in the LRO logic:

The polling interval is 1 second, which is too short. That could: a) make clients more likely to be rate limited by server side; b) generate a lot of LRO records during test recording.
We don't honor the Retry-After header in previous response, per spec, we should follow that if available.
There is no wait between the operation request and the 1st polling request.

Solution

enlarge default polling interval from 1 second to 30 seconds
wait the given interval before sending a polling request, instead of after, thus there is an interval between the operation request and the 1st polling request
use retry-after header in the last response as the interval before the next polling request, use the last available value, otherwise roll back to default polling interval

Checklist
To ensure a quick review and merge, please ensure:

[ ] The PR has a understandable title and description explaining the why and what.
[ ] The PR is opened in draft if not ready for review yet.

If opened in draft, please allocate sufficient time (24 hours) after moving out of draft for review

[ ] The branch is recent enough to not have merge conflicts upon creation.

Ready to Land?

[ ] Build is completely green

Submissions with test failures require tracking issue and approval of a CODEOWNER

[ ] At least one +1 review by a CODEOWNER
[ ] All -1 reviews are confirmed resolved by the reviewer

Override/Marking reviews stale must be discussed with CODEOWNERS first

Supported through OperationInternal

Changes were made in https://github.com/Azure/autorest.csharp/pull/1705


use Arr and Str helper classes

As of Laravel 6 all str_ and array_ helpers have been removed from the framework, this change update all calls to these helpers to use the Illuminate\Support\Str and Illuminate\Support\Arr classes.

Thank you. bump v2.1.1


Remove payload field from table sys.segment

Motivation
This PR proposes to remove the payload field from table sys.segments mainly to handle performance issue caused by serializing object to JSON string.
Drop alternative
Add a projection rule for scanning table sys.segment.
Operational impact
Since the payload field has been already public documented, it's a little aggressive to remove it. But I would like to suppose that there may be rare possibility someone has a dependency on this field. The RESTful api /druid/coordinator/v1/datasources/{dataSourceName}/segments/{segmentId}  should be the priority option to retrieve segment metadata.
This PR has:

[X] been self-reviewed.
[ ] added documentation for new or modified features or behaviors.
[ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
[ ] added or updated version, license, or notice information in licenses.yaml
[ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
[ ] added unit tests or modified existing tests to cover new code paths.
[ ] added integration tests.
[X] been tested in a test Druid cluster.

Key changed/added classes in this PR

SystemSchema

Hi @yuanlihan, it seems reasonable to me to not show the whole payload since it has duplicate fields and a garbage (pruned loadSpec). But, I believe other fields such as shardSpec, dimensions, and metrics are still useful. What do you think about adding them instead of payload?

But, I believe other fields such as shardSpec, dimensions, and metrics are still useful. What do you think about adding them instead of payload?

Hi @jihoonson
I agree that these fields should be useful in some use cases. But I still have some concerns about performance of serializing these fields. Can we add fields when they are needed?
Note that currently almost all sql queries that scan the sys.segment table have a sorting clause. And if there are a large number of segments(more than 1 million segments in my case), it will be really slow to load the Segments page of the console. And the frontend page already uses the point query API /druid/coordinator/v1/datasources/{dataSourceName}/segments/{segmentId} to get the payload.

Oh I meant adding them as new columns but without serialization. For example, we can change this code as below:
              return new Object[]{
                  segment.getId(),
                  segment.getDataSource(),
                  segment.getInterval().getStart().toString(),
                  segment.getInterval().getEnd().toString(),
                  segment.getSize(),
                  segment.getVersion(),
                  Long.valueOf(segment.getShardSpec().getPartitionNum()),
                  numReplicas,
                  numRows,
                  IS_PUBLISHED_TRUE, //is_published is true for published segments
                  isAvailable,
                  isRealtime,
                  val.isOvershadowed() ? IS_OVERSHADOWED_TRUE : IS_OVERSHADOWED_FALSE,
                  segment.getShardSpec(),  // new column
                  segment.getDimensions(), // new column
                  segment.getMetrics()     // new column
              };

Can we add these fields when they are needed?

I think we are already using them. The purpose of the "Segments" table or system table is mostly for operators to easily debug their druid cluster. Dimensions and metrics are needed to see if the schema has changed. ShardSpec is needed to see how segments are partitioned so that you can repartition them if needed.

Hi @jihoonson, thanks for your suggestion.
I tried it and it works well for me. And kindly take an another review when you are available. Thanks!

thanks @yuanlihan for raising this PR, i'll also take a look at this soon.


Delay crawling for slow websites

I am trying to crawl a website, but it takes a while to load, and from my screenshot I can see that by the time the crawler starts, it's still just skeleton content. Is there a way to delay the crawling? I couldn't find it from the documentation, and tried to use the custom js_code to implement a function that blocks the main loop:
js_code=["const sleep = (ms) => { const start = Date.now(); while(Date.now() - start < ms) {}}; sleep(5000)"]

def create_crawler():
    crawler = WebCrawler(verbose=True)
    crawler.warmup()
    return crawler

crawler = create_crawler()
result = crawler.run(
    url=url,
    js=js_code,
    screenshot=True,
    chunking_strategy=NlpSentenceChunking()
)

But it doesn't seem to have any effect. Anyone else had success with something like this, or could a delay feature be added?
PS: I'm excited about this project, thanks for making it!

@knandersen It is actually supposed to detect such situations. Could you please provide the URL? I can assist you with that.

@knandersen Just to inform you that if you desire a custom time delay before fetching the HTML, you can utilize the "after_get_url" hook. Here's an example:
import time
def create_crawler():
    crawler = WebCrawler(verbose=True)
    crawler.warmup()
    return crawler

crawler = create_crawler()
crawler.set_hook('after_get_url', lambda _: time.sleep(5))
result = crawler.run(url="https://example.com")

This instructs the crawler to wait for 5 seconds before attempting to extract the HTML.

@unclecode thanks for the hook trick! I'm trying to find a url I can provide you as an example (unfortunately not allowed to share the one I encountered the problem with)

@knandersen I apologize for my mistake!  Kindly refer to this code:
import time

from crawl4ai.web_crawler import WebCrawler
from crawl4ai.crawler_strategy import *

def delay(driver):
    print("Delaying for 5 seconds...")
    time.sleep(5)
    print("Resuming...")

def create_crawler():
    crawler_strategy = LocalSeleniumCrawlerStrategy(verbose=True)
    crawler_strategy.set_hook('after_get_url', delay)
    crawler = WebCrawler(verbose=True, crawler_strategy=crawler_strategy)
    crawler.warmup()
    return crawler

crawler = create_crawler()
result = crawler.run(url="https://google.com", bypass_cache=True)

this works, thank you so much!


prometheus-podman-exporter: init at version 1.1.0

Description of changes
prometheus-podman-exporter is a prometheus metrics exporter that enables monitoring podman containers, pods, images, volumes and network information.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin  (n/a, not supported)
[ ] aarch64-darwin (n/a, not supported)

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)  (n/a, not supported)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/1544

@SuperSandro2000 Thank you for the review; comments have been addressed.
I've removed the nixos module from this PR at this time, changing it to just introducing the package.  There's some further tweaking required on the exporter's systemd to get it the right permissions to view podman data successfully, which I didn't realize in the earlier version.


openstackclient: Get rid of kolla_start

... so that we don't need to allow the cloud-admin users to use sudo. Usage of kolla_start is redundant because we have no config files copied into the container by the command.

Looks OK to me but we have no e2e deploy job for this repo yet. Have you deployed this change?


CVE-2022-25883 @ Npm-semver-7.3.4

Vulnerable Package issue exists @ Npm-semver-7.3.4 in branch main
The package semver versions prior to 7.5.2 are vulnerable to Regular Expression Denial of Service (ReDoS) via the function new Range, when untrusted user data is provided as a range.
Namespace: westonphillips
Repository: CheckmarxOnePOV
Repository Url: https://github.com/westonphillips/CheckmarxOnePOV
CxAST-Project: westonphillips/CheckmarxOnePOV
CxAST platform scan: 6a8170d0-38fa-4efc-81df-42628474102c
Branch: main
Application: CheckmarxOnePOV
Severity: MEDIUM
State: NOT_IGNORED
Status: RECURRENT
CWE: CWE-1333

Additional Info
Attack vector: NETWORK
Attack complexity: LOW
Confidentiality impact: NONE
Availability impact: LOW

References
Advisory
Commit
Pull request

Namespace: westonphillips
Repository: CheckmarxOnePOV
Repository Url: ،د
،,mol.GOV.SA

CxAST-Project: westonphillips/CheckmarxOnePOV
CxAST platform scan: 6a8170d0-38fa-4efc-81df-42628474102c
Branch: main
Application: CheckmarxOnePOV
Severity: MEDIUM
State: NOT_IGNORED
Status: RECURRENT
CWE: CWE-1333


TTML writer should output in timecode hh:mm:ss:ff unit

At this time it seems not possible to output the TTML subtitle timestamps in a timecode format with frames unit (and not ms), based on the EBU STL source frame rate. A such enhancement should be welcome for using ttconv within the broadcast workflows.

Can you share the use case you have in mind? Are you looking to synchronize the TTML output with a video signal with embedded SMPTE timecode?
When ttp:timeBase="media", which is required for IMSC and the only practice in streaming, a TTML time expression HH:MM:SS:FF does not generally correspond to a SMPTE continuous timecode of HH:MM:SS:FF -- even though the two expressions look the same.
For example, at 23.98 fps, the TTML time expression 00:00:02:22 corresponds to an offset of:
2 + 22 / (24 * 1000/1001) = 2.9175833…s if ttp:frameRate="24" and ttp:frameRateMultiplier="1000 1001"
which is not equal to the duration of 70 frames at 23.98 fps (2.9196s).
(see https://www.w3.org/TR/ttml2/#time-expression-semantics-media and https://www.palemieux.com/2016/07/02/hhmmssff-expressions-in-imsc1/).

The aim is to merge with another TTML transcoding source, where the SMPTE timecode format is respected, knowing the frame rate within the EBU STL source file./

Can you share a sample of the other output?

2c9e41d9-9809-44db-a7b7-bc3c3f0b52ab_subtitles.ttml.zip

Thanks! This is super helpful.
Couple of observations/questions:

the document does not declare ttp:timeBase, which means that, by default, ttp:timeBase="media"
the document declares neither ttp:frameRate nor ttp:frameRateMultiplier, which means that, by default, ttp:frameRate="30" and ttp:frameRateMultiplier="1 1", i.e. the frame is 30fps
as a result, 00:00:10:19 means 10.6333s, and not the SMPTE timecode 00:00:10:19, is that the intent?

I guess there is some syntax issues. It is a french TTML file used for the catch-up in production, so the frame rate is 25fps for sure, and 00:00:10:19 means 10.760s. The aim is that ttconv provide this kind of output without the need to parse and convert the results with 3PP.

What is the input to ttconv that would result in the generation of an output with time expression like 00:00:10:19?
As exemplified above, my reluctance to supporting timecode-like expressions in ttconv is that such expressions are used incorrectly most of the time in TTML.
What is 3PP?
DM me if you are interested in joining the ttconv discord channel: palemieux#8530

Please find attached an example of input file.
3PP is for « Third Party Product » e.g sed substitution with regex and so on to convert ms like .760 as frame :19 within the TTML output from ttconv.
15ee6584-7c5a-4f60-8d2e-d0b456ae69c2.stl.zip

to convert ms like .760 as frame :19

Ok. It would be straightforward to add an option to output HH:MM:SS:FF instead of HH:MM:SS.mmm with FF = floor(0.mmm * frame_rate). Can you confirm this is what you have in mind?
The caveat is this will result in significant confusion when non-integer frame rates are used. Perhaps ttconv could simply refuse to output HH:MM:SS:FF in those cases.
@xchange11 Any opinion?

@Vincent-Dabouineau Thanks for bringing this use case up.
@palemieux I agree with your consideration. Your proposal to only support integer frame rates as a first step makes sense. In general I assume that time expression are used as if timebase smpte would be specified. There has been some discussions esp. amongst European broadaster to support SMPTE time base in IMSC but this is not suported yet.
@Vincent-Dabouineau As I understand a postprocessing step is not an option for you. I am asking this because a quick solution would be to do a XSLT transformation where only the time expressions in the begin and end attributes are replaced with the desired output. Can you explain your reluctance? Would be an XSLT with tests be an option?

@palemieux: I'm agree with your first step proposal.
@xchange11: any quick win solution should be welcome, so XSLT for sure, if you can share an example please. Thank you.

@Vincent-Dabouineau Please review https://github.com/sandflow/ttconv/pull/352


Renaming freeflyer_joint to root_joint in buildSampleModelHumanoid

Having two naming standards looks like a design bug. This is not an ideal choice and I propose to rename it to root_joint. I expect to use the same naming standard to include a root joint.
If you agree with this issue, I could create a PR handling it.

It seems the freeflyer_joint name is hard coded in https://github.com/stack-of-tasks/pinocchio/blob/devel/src/parsers/sample-models.hxx#L320.
You can see the change of the name by running the following code:
import pinocchio
model = buildSampleModelHumanoid()
print(model.names[1])

which is different to
import example_robot_data
model = example_robot_data.load("talos").model # this happens in all the models with FF root.
print(model.names[1])

Thanks for the clarification !
Please open a PR with the fix. Thanks for your collaboration !