chore: rename custom_records to tlv_records

This makes it more consistent with the usage in the keysend functions

@bumi if we want to make it consistent I would also change the layout of the TLV records to match the NIP-47 spec. Is that ok? I can make the change.
"tlv_records: [ // tlv records, optional
            {
                "type":<PHONE_NUMBER>, // tlv type
                "value": "0123456789abcdef" // hex encoded tlv value
            }
        ]

true, makes sense. can you add this?
(damn, then I need to convert it back in the rails app to handle it like lndhub/lnd :) )


Bluetooth: audio: Update context type defines

The values are taken from Generic Audio Assigned Numbers document.

@asbjornsabo @jhedberg Could you please take a look?


rbd: conditionally advertise topology-aware provisioning capability

Describe what this PR does
Since external-provisioner (>v5) enabled Topology FeatureGate by default. It expects CSI to advertise VOLUME_ACCESSIBILITY_CONSTRAINTS capability and provide the AccessibleTopology in NodeGetInfoResponse which is required by external-provisioner during provisioning to generate accessibility requirement. AccessibleTopology is generated by domainlabels passed.
Issue: If AccessibleTopology is empty, provisioning fails as VOLUME_ACCESSIBILITY_CONSTRAINTS is advertised.
Fix: advertise VOLUME_ACCESSIBILITY_CONSTRAINTS only if domainlabels are passed.
Checklist:

[x] Commit Message Formatting: Commit titles and messages follow guidelines in the developer guide.
[x] Reviewed the developer guide on Submitting a Pull
Request
[x] Pending release notes updated with breaking and/or notable changes for the next major release.
[x] Documentation has been updated, if necessary.
[x] Unit tests have been added, if necessary.
[x] Integration tests have been added, if necessary.

Show available bot commands
These commands are normally not required, but in case of issues, leave any of
the following bot commands in an otherwise empty comment in this PR:

/retest ci/centos/<job-name>: retest the <job-name> after unrelated
failure (please report the failure too!)

/test ci/centos/mini-e2e/k8s-1.29

Note: Testing is pending as am unable to build image.
Will update once everything is done.

Since GetPluginCapabilities call is from external-provisioner, domainlabels needs to be passed to deployment.
https://github.com/ceph/ceph-csi/blob/d46b7d7ff4d0dc992b8feb5e9d9fd54c6424a7f4/internal/rbd/driver/driver.go#L140-L146

@mergifyio rebase

Test Results -

Capability advertise -

I0725 09:44:27.408441       1 utils.go:235] ID: 14 GRPC call: /csi.v1.Identity/GetPluginCapabilities
I0725 09:44:27.408600       1 utils.go:236] ID: 14 GRPC request: {}
I0725 09:44:27.408898       1 utils.go:242] ID: 14 GRPC response: {"capabilities":[{"Type":{"Service":{"type":1}}},{"Type":{"VolumeExpansion":{"type":1}}}]}

After passing --enable-topology=true
I0725 09:47:01.998269       1 utils.go:235] ID: 16 GRPC call: /csi.v1.Identity/GetPluginCapabilities
I0725 09:47:01.998308       1 utils.go:236] ID: 16 GRPC request: {}
I0725 09:47:01.998512       1 utils.go:242] ID: 16 GRPC response: {"capabilities":[{"Type":{"Service":{"type":1}}},{"Type":{"VolumeExpansion":{"type":1}}},{"Type":{"Service":{"type":2}}}]}

Provisioning PVC

With --enable-topology=true and not --domainlabels
(fails as domainlabels are not passed)

I0725 09:51:17.846580       1 event.go:389] "Event occurred" object="rook-ceph/rbd-pvc" fieldPath="" kind="PersistentVolumeClaim" apiVersion="v1" type="Normal" reason="Provisioning" message="External provisioner is provisioning volume for claim \"rook-ceph/rbd-pvc\""
W0725 09:51:17.847704       1 topology.go:319] No topology keys found on any node

With --enable-topology=true and --domainlabels=topology.rook-ceph.rbd.csi.ceph.com/zone

89] "Event occurred" object="rook-ceph/rbd-pvc" fieldPath="" kind="PersistentVolumeClaim" apiVersion="v1" type="Normal" reason="Provisioning" message="External provisioner is provisioning volume for claim \"rook-ceph/rbd-pvc\""
I0725 09:54:29.984083       1 connection.go:264] "GRPC call" method="/csi.v1.Controller/CreateVolume" request="{\"accessibility_requirements\":{\"preferred\":[{\"segments\":{\"topology.rook-ceph.rbd.csi.ceph.com/zone\":\"test\"}}],\"requisite\":[{\"segments\":{\"topology.rook-ceph.rbd.csi.ceph.com/zone\":\"test\"}}]},\"capacity_range\":{\"required_bytes\":1073741824},\"name\":\"pvc-a7b4d312-eac8-40d7-83bb-e1f1a350e807\",\"parameters\":{\"clusterID\":\"rook-ceph\",\"csi.storage.k8s.io/pv/name\":\"pvc-a7b4d312-eac8-40d7-83bb-e1f1a350e807\",\"csi.storage.k8s.io/pvc/name\":\"rbd-pvc\",\"csi.storage.k8s.io/pvc/namespace\":\"rook-ceph\",\"encryptionKMSID\":\"azure-test\",\"imageFeatures\":\"layering\",\"imageFormat\":\"2\",\"pool\":\"replicapool\"},\"secrets\":\"***stripped***\",\"volume_capabilities\":[{\"AccessType\":{\"Mount\":{\"fs_type\":\"ext4\"}},\"access_mode\":{\"mode\":1}}]}"
I0725 09:54:30.343330       1 connection.go:270] "GRPC response" response="{\"volume\":{\"capacity_bytes\":1073741824,\"volume_context\":{\"clusterID\":\"rook-ceph\",\"encryptionKMSID\":\"azure-test\",\"imageFeatures\":\"layering\",\"imageFormat\":\"2\",\"imageName\":\"csi-vol-c2b2e8dd-cd0e-4e63-9e19-e09c5ecdf7d7\",\"journalPool\":\"replicapool\",\"pool\":\"replicapool\"},\"volume_id\":\"0001-0009-rook-ceph-0000000000000002-c2b2e8dd-cd0e-4e63-9e19-e09c5ecdf7d7\"}}" err=null
I0725 09:54:30.343382       1 controller.go:853] create volume rep: {CapacityBytes:1073741824 VolumeId:0001-0009-rook-ceph-0000000000000002-c2b2e8dd-cd0e-4e63-9e19-e09c5ecdf7d7 VolumeContext:map[clusterID:rook-ceph encryptionKMSID:azure-test imageFeatures:layering imageFormat:2 imageName:csi-vol-c2b2e8dd-cd0e-4e63-9e19-e09c5ecdf7d7 journalPool:replicapool pool:replicapool] ContentSource:<nil> AccessibleTopology:[] XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0}
I0725 09:54:30.343539       1 controller.go:925] createVolumeOperation: set annotation [volume.kubernetes.io/provisioner-deletion-secret-namespace/volume.kubernetes.io/provisioner-deletion-secret-name] on pv [pvc-a7b4d312-eac8-40d7-83bb-e1f1a350e807].
I0725 09:54:30.343674       1 controller.go:955] successfully created PV pvc-a7b4d312-eac8-40d7-83bb-e1f1a350e807 for PVC rbd-pvc and csi volume name 0001-0009-rook-ceph-0000000000000002-c2b2e8dd-cd0e-4e63-9e19-e09c5ecdf7d7

Note that this is not suitable:

Introduce a EnableTopology flag on controller server (default: false) to enable/disable topology based provisioning

Some StorageClasses may need to use topology based provisioning, others not. They are handled by the same Ceph-CSI driver. I guess that when a CreateVolume call does not have the topology parameters (come from the StorageClass) available, it should just not try to use it either.

@iPraveenParihar , last week we discussed about alternatives by setting an other option for the node-plugin. Do you want to leave some notes about that here? Or maybe update the PR if you have time for that?


1.5.0

LGTM, dont't forget to update the commit hash before creating the tag

@jpopelka Isn't that a chicken-egg problem? You don't know the commit hash of the release commit.

LGTM (atomic-reactor-1.5, openshift-1.0.5, osbs-0.14 -> successful simple build)


Support for props in the preview app

Hi!
It would be great if the preview app could be used to edit props of a specific email. That could be useful to test out edge cases with long values, different alphabets,...
Would also be nice if we could supply default values only meant to be used for the preview app.

I support this suggestion, however there is a solid solution for the default value problem.
You can destructure your prop object and add whatever defaults you want.
When the preview app renders the email, the defaults are used:
interface EmailProps {
  /**
   * User's first name
   */
  firstname: string;
  /**
   * User's last name
   */
  lastname: string;
}

const Email: React.FC<EmailProps> = ({
  firstname = "John",
  lastname = "Doe",
}) => {
  return (
    <Html>
      <Section>
        <Text>
          Hello {firstname} {lastname}
        </Text>
      </Section>
    </Html>
  );
};

Best part of this solution is that even though you supply defaults and the preview app doesn't throw an error, the typescript server does:

This is a SUPER interesting feature.
An inspiration that came to mind was Microlink Cards: https://cards.microlink.io/editor?preset=workos

Great point about the default values @ImBaedin, I feel like we should update all demos to contain that pattern. Do you want to send a PR?

Great point about the default values @ImBaedin, I feel like we should update all demos to contain that pattern. Do you want to send a PR?

You're talking about these demos right?
If so, sounds like fun 😊

Yep ;)

Yep ;)

#377


Add integration_ids

What does this PR do?
Adds a new field to the manifest.json file integration_id
This is part of a larger work to standardize how an integration is identified in various locations. This will always be the kebab casing of the integration name.
openstack_controller's integration id would become openstack-controller
Additional Notes
Anything else we should know when reviewing?
Review checklist (to be filled by reviewers)

[ ] PR title must be written as a CHANGELOG entry (see why)
[ ] Files changes must correspond to the primary purpose of the PR as described in the title (small unrelated changes should have their own PR)
[ ] PR must have changelog/ and integration/ labels attached
[ ] Feature or bugfix must have tests
[ ] Git history must be clean
[ ] If PR adds a configuration option, it must be added to the configuration file.

Already added to the validation here https://github.com/DataDog/integrations-core/pull/3232
Can you point me to which docs?


v2.7.0: punycode is deprecated in 'whatwg-url' and 'tr46'

Reproduction
Steps to reproduce the behavior:
Using node v21.2.0:
package.json:
{
    "dependencies": {
        "node-fetch": "^2.7.0"
    }
}

index.js:
const fetch = require('node-fetch');
fetch("https://google.com").then((res) => {
    console.log(res.status);
});

output:
(node:25053) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
200

Expected behavior
Lack of deprecation note

Your Environment

software
version

node-fetch
2.7.0

node
21.2.0

npm
10.2.3

Operating System
macOS

see: https://github.com/node-fetch/node-fetch/pull/1793#issuecomment-1839050150


Added Hover effect + improved bg of category font

made these 3 changes, hope you'll merge it.

@aryanu027 kindly star the repository, and I'l merge it then.

Done!!


Allow hiding of deceased/0 HP NPC's with the entity hider

A simple toggle of "Hide deceased" to hide npcs that have reached 0 hitpoints would help with the annoyance of casting barrage on the wrong npc in a stack, and misclicking on dead npcs with prolonged death animations.

This would not work as intended, as it would hide the stack of NPCs. We're unable to show NPCs that are hidden under other players for a similar reason, due to engine limitations. See: #1979

Jagex's recent statement prohibits Removes dead NPC entities or the minimenu options from them


Unlimited loading for "The Trade Desk" Company questions

Describe the bug
When I search question from the "Trending Companies" Tab, On home screen. The questions take unlimited time to load.
To Reproduce
Steps to reproduce the behavior:

Go to 'Problem Tab'
Click on 'Search bar of Trending Companies' (present on right side)
Search for the company "The Trade Desk".
Questions take unlimited amount of time to load.

Expected behavior
A list of questions should've been displayed.
Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):
Chrome
Additional context
Add any other context about the problem here.

Unfortunately, our database is outdated. Make sure to save this repo we gonna update it soon. Thank you for your issue.


feat(convert): remove convert file

:tada: This PR is included in version 4.2.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Process GCP persistent disks with embedded raw image

Note:  Also needed to fully support this type of evidence format is a task to do the conversion from raw image files in GCS to GCP PD with embedded raw images (Issue #63 ).

Thanks!


Fix single file error emiting #164

Hello,
I open pull request for issue #164.
Sass output errors from single file compilation to stderr and from directory compilation to stdout. (messages are in slightly different format).
I add test for single file compilation, but there is still strange behaviour. Directory test is failing, because no .on('error') event is emitted. But when I run this plugin with Gulp, directory and single file compilation error emitting now works as expected. So I don't know what can be wrong. (OS X 10.10, node 0.12.0, sass 3.4.13)

Thanks for this! More Sass error weirdness, I should have known.
I'm currently refactoring all error handling into a separate file required by the main gulp-ruby-sass module. I'll work this PR into that, or complete it without the addition and let you PR into the new error handling system.

Ok. It's up to you.

@robwierzbowski Any progress on the refactoring? Can you open a PR even though it's WIP so people can give feedback?

Ya, can do.
On Friday, April 24, 2015, Sindre Sorhus<EMAIL_ADDRESS>wrote:

@robwierzbowski https://github.com/robwierzbowski Any progress on the
refactoring? Can you open a PR even though it's WIP so people can give
feedback?
—
Reply to this email directly or view it on GitHub
https://github.com/sindresorhus/gulp-ruby-sass/pull/211#issuecomment-95866039
.

--
Rob Wierzbowski
@robwierzbowski http://twitter.com/#!/robwierzbowski
http://github.com/robwierzbowski
http://robwierzbowski.com

Closing in favor of #221.


Implement IsAdmin functionality

Currently the IsAdmin functionality just returns true by default.
Implement the isAdmin functionality that only returns true only if the user has the admin flag set on his/her account.

The admin functionality has already been implemented. I think there is some issue in the front-end.

Will look into the feasibility of a quickfix.

implemented


Add support for /proc/net/dev, /proc/[pid]/net/dev, and /proc/[pid]/ns

/proc/net/dev and /proc/[pid]/net/dev provide network interface metrics. These are the values that ifconfig reads.
/proc/[pid]/ns is directory containing symlinks that describe what namespaces the process is a member of.

Nice @andrewkroh

@andrewkroh we should add a CHANGELOG.md to the repo and add travis that at least does the go fmt.

@ruflin With the barebones .travis.yml in the project it actually does both a go fmt check and a go test. I was surprised too.

@andrewkroh oh, yes that is a surprise :-) Travis magic.


Add Bearer token auth to the JIRA exporter

Adds the usage of Bearer token without username to the JIRA exporter.
Fixes #917
Testing:

Get the token on the JIRA instance which uses Bearer Tokens. E.g. issues.redhat.com
Deploy exporter with the

SERVER=https://issues.redhat.com
TOKEN=<YOUR_TOKEN>

@redhat-cop/mdt

Since the hotfix has already been provided to user, I would work more in this PR. For example:

add in the docs the new usage
write integration test for new usage

PS: integration tests are breaking, but do not worry, I will fix them (again...)

@mateusoliveira43 agree on your points, that was quick hotfix, so we can polish the PR now.
Example config to deploy with this PR:
- app_name: failure-exporter
    exporter_type: failure
    extraEnv:
    - name: LOG_LEVEL
      value: DEBUG
    - name: SERVER
      value: https://issues.redhat.com/
    - name: TOKEN
      value: <YOUR_TOKEN>
    source_ref: refs/pull/918/head
    source_url: https://github.com/dora-metrics/pelorus.git

/lgtm


Feat/mute 482 block explorer

https://transmute.atlassian.net/browse/MUTE-482
API

Add "transactions" and "transaction summary" endpoints
Remove unused "operations" endpoint

APP

Use transactionHash instead of transactionTimeHash to represent a transaction
Remove unused logic for filtering the Block explorer
Add link to ethereum transaction in the SidetreeTransaction component

LIB

Add transactionHash field to the Transaction object
Add a getEthereumTransaction method to get an Ethereum Transaction by transactionHash
Add getInstance method to avoid recreating the instance every time (2s performance improvement)
Add a readThenWriteToCache method for more caching
Add methods for getting transactions

Codecov Report

Merging #122 into master will decrease coverage by 0.14%.
The diff coverage is 29.16%.

@@            Coverage Diff             @@
##           master     #122      +/-   ##
==========================================
- Coverage   41.78%   41.63%   -0.15%
==========================================
  Files         240      241       +1
  Lines        3413     3437      +24
  Branches      261      263       +2
==========================================
+ Hits         1426     1431       +5
- Misses       1877     1896      +19
  Partials      110      110

Impacted Files
Coverage Δ

...nts/SidetreeOperation/SidetreeOperation.stories.js
0% <ø> (ø)
:arrow_up:

...nt-lib/src/adapters/database/ElementRXDBAdapter.js
65.21% <ø> (ø)
:arrow_up:

packages/element-app/src/index.js
0% <ø> (ø)
:arrow_up:

...SidetreeTransaction/SidetreeTransaction.stories.js
0% <ø> (ø)
:arrow_up:

...nts/SidetreeBatchFile/SidetreeBatchFile.stories.js
0% <0%> (ø)
:arrow_up:

...ackages/element-app/src/redux/fullNode/handlers.js
0% <0%> (ø)
:arrow_up:

...ExplorerTransactionPage/ExplorerTransactionPage.js
0% <0%> (ø)
:arrow_up:

.../components/SidetreeBatchFile/SidetreeBatchFile.js
0% <0%> (ø)
:arrow_up:

...ckages/element-app/src/redux/lightNode/handlers.js
0% <0%> (ø)
:arrow_up:

...ponents/SidetreeTransaction/SidetreeTransaction.js
0% <0%> (ø)
:arrow_up:

... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0d82f1c...0f9a71f. Read the comment docs.

addressed comments @OR13


Enhacement: adding tests

Adding few tests for the application.

@kom-senapati Please assign me this issue.

Tests for which part ?

For the different routes using pytest-flask

But I think for doing so I need to change the structure of the project a little bit, so that I can create a new tests directory containing the tests separately while transferring the business logic of the application in a package that way I can import the functions and variables from different files from that package to be imported to be used in tests.

I can open a new issue for the restructure then do the tests after if you need.

Ok then raise issue for restructure


Implement comment system

I don't like Disqus. Maybe use staticman?

Firebase might be a better option
https://css-tricks.com/building-a-jekyll-site-part-3-of-3/

I'll do this one if I run into some problems using Disqus.


Mtc route attr docs fixes

Checklist

[x] Appropriate branch selected => This is an MTC documentation PR against the mtc-docs branch.
[na] Any modified or new methods or classes have helpful JSDoc and code is thoroughly commented
[na] The description lists all applicable issues this PR seeks to resolve
[na] The description lists any configuration setting(s) that differ from the default settings
[x] All tests and CI builds passing

Description
This PR contains documentation updates to the GTFS+ editor and project/feed management requested by MTC.
Also contains some setting updates so that the documentation builds in both the CI and readthedocs.io (previously, two different, slightly incompatible versions were used).

Will proceed to merge the documentation changes by request of client.


on parse error the string "invalid date" is set in the model

This is due to Moment.js returning "invalid date" as string literal for invalid dates when formatting. When parsing the value again for displaying it in the date picker, this results in the date "1900-01-01", which is rather annoying.
Instead the view value should be set to null, which allows the date picker to reset itself to the default date when opened the next time.

Fixed on master (v1.1.0) and release-1.0.x (v1.0.1).


Tasks with no due date

It appears to not be working currently with tasks with no due date. I need to add handling for no due date.

Fixed on latest update - tested and tasks with due date (recurring and not reucrring) and without due date both work.


Xamarin.Android build error related to proguard

We recently upgraded our xamarin.android project to support android 7.0 and the VSTS build broken with below error, on our local build we can fix the error by upgrade to latest version of proguard( https://sourceforge.net/projects/proguard/files/proguard/), how can we get this fixed with VSTS?
2017-01-04T07:07:36.9634026Z ##[error]CREATEMULTIDEXMAINDEXCLASSLIST(0,0): Error : Can't read [C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\MonoAndroid\v7.0\mono.android.jar] (Can't process class [android/app/ActivityTracker.class] (Unsupported class version number [52.0] (maximum 51.0, Java 1.7)))

Are you using our hosted build image?
Have you tried with a private agent (azure) where you can control the toolsets?  I think 7.0 is in the process of getting on the image.

Android 7 support is available on the hosted build VMs now. But you need to use JDK 8 explicitly in your build process as Xamarin requires JDK 8 but by default Xamarin will use JDK 6 if installed on the build server. Please try setting the JDK version on Xamarin Android task to 8 and see if this helps.

@bryanmacfarlane Yes hosted agent used.
@madhurig the JDK 8 option was already set and we tried both x86 and x64 options, all same results, here the build summary link if you need: https://boundlss.visualstudio.com/Boundlss/_build/index?buildId=635&_a=summary&tab=details

@jamesatgithub: It looks like the proguard version is not being updated with Android SDK and has to be installed separately: http://stackoverflow.com/questions/39514518/xamarin-android-proguard-unsupported-class-version-number-52-0.
We don't have ProGuard 5 on the hosted build VMs, so you are seeing this error. We will update in the next rollout, I will post a timeline for the availability here once we have it.
Thanks,
Madhuri

The Xamarin bug resulting in this error is fixed for Cycle 9 release but is not in the stable channel yet. Replacing ProGuard version in the Android SDK location is a workaround suggested by Xamarin team but we don't want to do that on the hosted build VMs since they are shared machines and we don't know if there will be side-effects of the workaround for native Android users.
We will upgrade to Cycle 9 on hosted build VMs which has the fix once it is stable. Till then the alternate will be to use your own build server.
Thanks,
Madhuri

Have the build VMs been upgraded to Cycle 9? If not, when is that happening?

Cycle 9 is not available in the Stable channel yet, it is in Beta: https://releases.xamarin.com/release-candidate-cycle-9-rc6-refresh/. We will update the Hosted build VMs to Cycle 9 after it is available in the Stable channel.
Thanks,
Madhuri

Hello :)
Any update on this? It's been some days since the Cycle 9 was released and I'm still getting the error
 java.io.IOException: Can't read [C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\MonoAndroid\v7.0\mono.android.jar] (Can't process class [android/app/ActivityTracker.class] (Unsupported class version number [52.0] (maximum 51.0, Java 1.7)))

@GiusepeCasagrande : What Java version are you targeting? Please update your build task to use Java8 and see if helps.

Hi @madhurig,
Just checked, I'm already using JDK8

@GiusepeCasagrande : This was marked as fixed in Cycle 9: https://github.com/xamarin/xamarin-android/pull/267, so it should have worked. I will follow up .

@madhurig I'll give it a few more tries, maybe I'm making something wrong here. But please, let me know if you find something else about it.
And thanks for your help so far :)

Hello, I am experiencing the same issue as Guisepe.
I have both MultiDex and ProGuard enabled. I tried targeting API 24 and API 25, but I am getting the same issue. I spent the day trying various workaround, but without luck on hosted VSTS machines, although it's building fine on my local machine.
I followed advice to add Proguard folder to my source control and set custom ProGuardJarPath and the ProGuardToolPath. I did that in proguard.cfg (BuildAction is set to ProguardConfiguration like expained in here):
ProGuardJarPath = ..\..\..\Shared\tools\proguard\lib\proguard.jar
ProGuardToolPath = ..\..\..\Shared\tools\proguard\

I am not sure if proguard.cfg is the right place for this lines, but it didn't fix the issue.
I set up my local environment following this blog post. My guess is that hosted machines have something different.

@GiusepeCasagrande : My xamarin android project with ProGuard enabled builds without errors on the hosted build VMs. Can you share your debug build logs for further investigation? You can upload here or email me: madhurig at microsoft dot com.
My build definition settings look like this:

Thanks,
Madhuri

@markolazic88: I checked the blog you linked to. The hosted build VMs don't have all the android build tools versions that the blog lists. We have an update with those in the pipeline but I can't be sure if it will fix your problem since I can't repro it.
If you continue to have the issue after our next update, please open a new issue with your build logs.

@madhurig Thank you for the info. Could you please share the date when those tools are going to be updated? That info is important for us, as we are still blocked, and we would have to create private build machine if the update is not going to happen any time soon.

Adding @thechrisjohnson for timeline on the update.

@madhurig Hi, I'm experiencing this issue too, I'm targeting Android 7.0 and building with android 7.0, when I build in VSTS, I get the error:
java.io.IOException: Can't read [C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\MonoAndroid\v7.0\mono.android.jar] (Can't process class [android/app/ActivityTracker.class] (Unsupported class version number [52.0] (maximum 51.0, Java 1.7)))
However, in my build configuration, I have selected Java 1.8:

On my local machine, I fixed this by upgrading to Proguard 5.3, as the others have mentioned above. I note that you said the same was going to happen on VSTS hosts, but now it seems that it has been removed from the milestone, can you suggest when/if this will happen? This is a major dealbreaker for moving into VSTS :(

@AviateX14: Can you please try the "VS 2017" agent and see if this issue is resolved there?

@madhurig:This raises a new issue. The build has proguard enabled, proguard will not work with spaces in the path to it's directory (see the following post on the Xamarin forums on my local machine I've set up a symlink to correct this, however in VSTS using the Hosted VS 2017 agent, there is no link, and as such, I get this error:
"C:\a\1\s\HarnessInspections.Android\HarnessInspections.Android.csproj" (PackageForAndroid target) (1) ->
(_CompileToDalvikWithDx target) ->
  PROGUARD : error : C:\Program Files (Access is denied) [C:\a\1\s\HarnessInspections.Android\HarnessInspections.Android.csproj]
    103 Warning(s)
    1 Error(s)
Time Elapsed 00:01:10.05
PROGUARD(0,0): Error : C:\Program Files (Access is denied)
PROGUARD : error : C:\Program Files (Access is denied) [C:\a\1\s\HarnessInspections.Android\HarnessInspections.Android.csproj]
  Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8

Disabling Proguard stops the build because there ends up being too many methods defined, so this isn't really an options as I don't want to use multi-dex or have massive APK's.

@AviateX14: MobileCenter builds are run on Mac, so these issues don't apply there. Thanks for the details, we will investigate and provide a solution.

@madhurig thanks, I look forward to your response.

I'm facing the same issue as @AviateX14 . Despite setting MSBuild parameters /p:ProGuardToolPath=C:\a\1\s\{projectdir}\proguard\ /p:ProGuardJarPath=C:\a\1\s\{projectdir}\proguard\lib\proguard.jar /p:EnableProguard=true  it errors with "Error : C:\Program Files (Access is denied)" ... maybe because it uses proguard config files in the default android sdk proguard folder (which is under program files)
I tried passing /p:ProGuardConfigurationFiles="proguard\proguard-android.txt;obj\Release\proguard\proguard_xamarin.cfg;obj\Release\proguard\proguard_project_references.cfg;obj\Release\proguard\proguard_project_primary.cfg;proguard.cfg" which didn't work either - it still used the standard files

We are following up on the 2 Xamarin bugs with the Xamarin team:

https://bugzilla.xamarin.com/show_bug.cgi?id=44187
https://bugzilla.xamarin.com/show_bug.cgi?id=32861

We don't want to apply workarounds for these bugs since they might impact native Android builds. Till we figure out the right solution here, the option will be to setup your own build server and apply the workarounds for Xamarin Android ProGuard builds.
Thanks,
Madhuri

While we are continuing to work with Xamarin team on the bugs, I have a workaround that can be applied to build on the Hosted and Hosted VS 2017 VM pools.

Download latest proguard from: https://sourceforge.net/projects/proguard/files/proguard/5.3/ and commit/push to your repository.

In the build definition use "Extract Files" to unzip to a location without spaces, I set it to $(build.sourcesdirectory)

If using "Hosted VS 2017" pool, a symlink needs to be created for the Android SDK location so the path does not have spaces. This is not required for "Hosted" pool. Create a batch script and commit to your repo with this command: mklink /J C:\AndroidSDK "C:\Program Files (x86)\Android\android-sdk"

In the Xamarin Android task "MSBuild Options > Additional Arguments", set
(Hosted pool)
/p:ProguardToolPath=$(build.sourcesdirectory)\proguard5.3.3\

(Hosted VS 2017 pool)
/p:AndroidSdkDirectory=C:\AndroidSDK /p:ProguardToolPath=$(build.sourcesdirectory)\proguard5.3.3\
Thanks,
Madhuri

Step 3:  " Create a batch script"?  Does the batch script contain the command
mklink /J C:\AndroidSDK "C:\Program Files (x86)\Android\android-sdk" ?
What should I call the batch script?  x.bat?  what invokes the x.bat script?

ok i found a build task to execute the batch script.

Progress!  Now the build fails with a different error.  I checked we are using JDK 8.
Any suggestions on this one?
CREATEMULTIDEXMAINDEXCLASSLIST : error : Can't read [C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\ReferenceAssemblies\Microsoft\Framework\MonoAndroid\v7.1\mono.android.jar] (Can't process class [android/app/ActivityTracker.class] (Unsupported class version number [52.0] (maximum 51.0, Java 1.7))) [d:\a\1\s\Ageas..App.Droid\Ageas..App.Droid.csproj]

@jonathan305: It looks like if multidex is enabled, the /p:ProGuardToolPath is not being used. Looks like replacing the ProGuard version in the AndroidSDK folder is the only way to get this working. I was able to workaround that as well but it is a crazy set of workarounds :(. I will update the workarounds post and communicate back to the Xamarin team.

@madhurig: I've tried the workaround you said.

But i have this error : ##[error]C:\Program Files (x86)\MSBuild\Xamarin\Android\Xamarin.Android.Common.targets(2037,3): Error MSB6006: "java.exe" exited with code 1
What i have done in my Android project is to enabled multidex and proguard.
Any news about these Xamarin bugs that this post is talking ?

@smedel: The Xamarin bug fixes are going to be available in the next update of Xamarin as per the update from the Xamarin team. The progress can be tracked on bugzilla too.
Can you share the full log or email it to me at madhurig at microsoft.com?

@madhurig i just sent you an email.

@madhurig Can you please update us on the status of this bug?

@ramyzbeidy: One of the issues (outdated proguard) is addressed in a newer version of Xamarin but the other one (issue with spaces in the path to AndroidSDK) will be included in a future release. Please try posted workaround if you wish to use the Hosted Build VMs. The alternate is to setup your own build server.
Thanks,
Madhuri

Hi,
I have setup my build agent on a Mac. I have an Android project with Multidex and proguard enabled.
I have installed JDK 8 and updated proguard to the latest version. Unforunately now I am getting an out of memory errror. I have the Java Heap Size set to 1GB in my Android csproj file.
The error:
error XA5213: java.lang.OutOfMemoryError. Consider increasing the value of $(JavaMaximumHeapSize). Java ran out of memory while executing 'java -Xmx1G -jar
Here is my configuration:

Any help would be greatly appreciated.

Based on this thread (https://forums.xamarin.com/discussion/2132/java-heap-space-outofmemoryerror-when-binding-a-big-jar), you might have to change the setting in the .csproj in multiple places for different configurations/platforms.
Please try passing /p:JavaMaximumHeapSize=1G in "Msbuild > Additional Arguments" and see if it helps.
Thanks,
Madhuri

Thanks for the help. In the end I had to increase the Heap Size to 2G to get it working. Strange since it was building on other machines with 1G and I have never had to go higher than that.
It took a while but at least its finally building. Summary of what was done in case anyone has a similar scenario:

App targeting Android 7.0
Multidex enabled
SDK Platform Tools v25+ used
Increase JavaMaximumHeapSize to 2G (Both Debug and Release Configurations)
Switch Hosted Agent to my own agent (Mac)
Install JDK 8
Download Proguard 5.3.3 files
Copy the 3 .jar files in the lib directory and place them in the lib directory where the Android SDK is installed

To Build on Mac VSTS Agent:

Used the following arguments in the Build Step:
JDK Path: /Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home
JDK Architecture: x64

To Build on Visual Studio Windows machine:

The Android SDK needs to be moved to a path that does not contain any spaces such as C:\Android\android-sdk
Update the Path inside Visual Studio

Hiya,
I'm currently experiencing the same issue, we are unable to build on both Mac in The Cloud Agents and also hosted VSTS agents.
We've had to revert to running jenkins internally as a build server for the Android portion of our Xamarin project.
Is this likely to be fixed soon?

We are experiencing the same problem using VSTS Hosted or Hosted 2017, any ETA on a fix?

@smedel did you ever get this working?  I'm now getting the same error as you
2017-08-28T16:02:29.4246673Z ##[error]C:\Program Files (x86)\MSBuild\Xamarin\Android\Xamarin.Android.Common.targets(2037,3): Error MSB6006: "java.exe" exited with code 1.
Any help would be appreciated.
Thanks in advance,
Stephen

The VMs on Hosted VS2017 queue have been updated with latest Xamarin version which has the fix for the original issue on this thread - Xamarin build fail with ProGuard + multidex when there are spaces in the AndroidSDK folder. So we are closing this issue.
In my offline conversation with @smedel, we found that removing spaces in output paths helped him resolve the issue as well.
I am closing this issue since the original issue is resolved and the thread is getting too long. Please create a new issue if you have other problems.

@madhurig thanks for the update, awesome timing !!  The project now builds if I just have just multidex on, which is fine for me.
FYI if I have ProGuard + multidex on I get:
C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\Xamarin\Android\Xamarin.Android.Common.targets(1929,3): Error MSB6006: "java.exe" exited with code 1.
Process 'msbuild.exe' exited with code '1'.

Thanks again for all your help.  Awesome! 👍

@madhurig  same here as @YARG

@ramyzbeidy @YARG: Seems like you are seeing another issue. Please open a new issue with details. If you can attach the logs from your build or email them to me (madhurig at microsoft.com), I can investigate.
Thanks,
Madhuri


Switch to govendor

This commit mostly preserves the same versions of dependencies, with a
few exceptions:

github.com/pborman/getopt appears to have been entirely unused
github.com/davecgh/go-spew and github.com/pmezard/go-difflib, both
dependencies of github.com/stretchr/testify, have been updated to use
the versions of those dependencies explicitly vendored by
github.com/stretchr/testify.

👍 thanks!


Build configuration

I would like to know the differences of options ALLOW_CUDA and REQUIRE_CUDA?
1. Why do you need an option REQUIRE_CUDA, if exists ALLOW_CUDA and that means I want to build tensorflow with GPU support?
2. I think the build tensorflow with GPU support should be in default mode, so this is more demand, than build on the CPU.

Hi, ALLOW_CUDA makes an attempt to find CUDA and if it fails, it falls back to CPU build. REQUIRE_CUDA is a stronger option - it also makes an attempt to find CUDA and if it fails, the whole build fails. ALLOW_CUDA is the default, so if your OS contains CUDA, it should be auto-detected and the library should be built with CUDA support. I did not want to force CUDA support, because a lot of users use the library in server environment and virtual server environment without a GPU (but sometimes with a lot of CPU power).


Contributing each package to Node.js

I would like to ask if you could suggest (or if you authorize someone else to suggest) on Node.js community to replace the original array methods to yours.
The difference is clear. :)

Sorry, now I realize that you already informed that: Most functions have slight differences from the native built-ins!
Maybe, for this reason, they would be not precisely a pure replacement candidate.

The Array methods are built/produced by C++ code, so it couldn't be a drop-in PR or anything. Not by me, anyway
When this repo came out, there was some attention from V8 about it and IIRC it did instigate some performance improvements, I just can't remember which.
As for differences, array holes aren't treated the same and I don't pass in a this context to function callers. The latter can be solved easily with ~6B or so, but I don't think either of these behaviors are used very much


Hoodie Client: replace PouchDB dependency with options.PouchDB

We want the Hoodie Client to not only run in browsers, but also on servers (server-side rendering) and in Service Workers. We are currently removing @hoodie/client-store's dependency on PouchDB by replacing it with options.PouchDB
This will also give developers more control about the used PouchDB version and custom build. Only hoodie should have a dependency on PouchDB, all lower-level dependencies should work with options.

[ ] depends on https://github.com/hoodiehq/hoodie-client/pull/107

@gr2m Can I take this issue, instead of PR?
And, I'll ask you about what PR is.

@Taekyoon I’m already working on this one, here is the work-in-progress pull request: https://github.com/hoodiehq/hoodie/pull/644

I got it!


Misc fixes

Changes:

Remove unneeded etc directory and contents (obsolete since v1.0.0)
Fix commit hash of tag in release notes
Add back:

texttable
tinydb
unidecode

Pass GITHUB_REF to docker images so that construct.yml has access
Test on Ubuntu 20.04 instead of 19.10 and test on OpenSUSE
Improve quality of tests to at least import all modules

Note: There was a very strange error, and I am not sure what the source was. GitHub went into degraded status while it was popping up. I don't think anything substantial changed from when the error happened to when it was fixed. Some things are unversioned, so the builds are not necessarily as repeatable as one would like. The error was occurring on both x86_64 Linux (py2 and py3) builds with the following message:
fetching: conda-4.8.3-py37hc8dfbb8_1.tar.bz2
Checking for duplicate files ...
Traceback (most recent call last):
  File "/opt/conda/bin/constructor", line 8, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.7/site-packages/constructor/main.py", line 244, in main
    dry_run=args.dry_run, conda_exe=args.conda_exe)
  File "/opt/conda/lib/python3.7/site-packages/constructor/main.py", line 119, in main_build
    fcp_main(info, verbose=verbose, dry_run=dry_run)
  File "/opt/conda/lib/python3.7/site-packages/constructor/fcp.py", line 245, in main
    ignore_duplicate_files, verbose, dry_run
  File "/opt/conda/lib/python3.7/site-packages/constructor/fcp.py", line 215, in _main
    pc_recs, platform, ignore_duplicate_files
  File "/opt/conda/lib/python3.7/site-packages/constructor/fcp.py", line 126, in check_duplicates_files
    getsize(join(extracted_package_dir, short_path)))
  File "/opt/conda/lib/python3.7/genericpath.py", line 50, in getsize
    return os.stat(filename).st_size
FileNotFoundError: [Errno 2] No such file or directory: '/home/conda/.conda/constructor/linux-64/sysroot_linux-64-2.12.2-h3a0023d_0/x86_64-conda-linux-gnu/sysroot/lib/libgomp.so'


Release 0.3?

I'm currently working on a side project which is a music player(termusic) written in rust. I'm using the 0.3 version of lofty from master branch. However, this can only work locally as 0.3 is not on crates.io yet.
Would you please publish it? Thanks.

I had planned to release 0.3 awhile ago, but kept finding issues and adding features. The published versions currently only read a single tag from the files. I've been working in the multi-tag-files branch for awhile now, as it required completely rewriting the crate. I'm nearly done with it and that will be released under 0.3.

Thanks for the update and I'll keep watching for your progress. It's a great work and simplified things a lot.

I'd like to migrate a project of mine to Lofty, but I don't want to have to rewrite it a few weeks later. Should I use Lofty 0.2, git, or wait for 0.3?

@leo60228 I'd recommend waiting for 0.3. 0.2 and the main branch have a fair amount of issues I've addressed in the multi-tag-files branch. I should finally have some time in the coming weeks to push it over the finish line.

It seems works fine now. Would you please publish a version first? I cannot publish as the dependency is not published yet. For some bugs, later we can fix when we find it or user report it. Thanks.

Yeah, I was planning on publishing. I just need to look at the docs to make sure I haven't left anything behind from previous versions. If all goes well it'll be out later today or tomorrow.

It'll be a great help! I'm currently using id3, metaflac, mp4ameta, lewton, ogg, and even write some nasty code for ogg pictures. With lofty, all things about metadata will be solved!

0.3 is out 🎉


No such module 'BlueFalcon'

opening an example project throw error in DeviceViewModel.swif file

Which example project? Sounds like its not building the framework, you should have a console error in the build.
Also What version of xcode and kotlin are you building for?

example project for iOS . My xcode version is 12.4 .

Do you have a screenshot or copy of the build output?

I am getting build time error . there is no build output

Do you have kotlin installed?

You have this output? it should show more details on the error.

The iOS example needs to be built against a real device target.

Just realised now, the IOS example, you need to open the whole project in Intelli-j and build the project, this will produce a BlueFalcon framework, which you can then use.
Here is one i made earlier.
BlueFalcon.framework.zip

ok let me try thanks a lot

Any lick @iphonedev4radixweb ?

Closing due to inactivity.


request for notifications UI

The snackbar/toast seem explicitly designed as a 'only one on screen at a time'. It would be nice
if we could get library support for this part of the MD spec:
https://www.google.com/design/spec/patterns/notifications.html
Which requires multiple toast/snackbar like panels to be present on screen at once.

You may take a look at what I did : http://codepen.io/jfily/pen/Bjvxyp

The notification spec is unrelated to toasts and snackbars. The Snackbar spec states specifically to only show one at a time.
The notification MD spec is talking about OS level notifications. This is not something web sites are capable of building on their own. However, the web itself offers a notification API that fulfills this need (however with limited browser support.) It also is not something we are capable of re-styling or manipulating.
We could look into doing a site-level notification UI. Guidance from the MD team would be required for positioning of notifications since the spec currently provides no guidance on that for web apps.

Thanks @jfily , that looks like a good starting point.
@garbee, yes. I also hadn't noticed it was labeled 'Android only' (not sure how, it was fairly prominent at the top of the page!). It is interesting that this form of notification is seen as android only given I think that type of multiple stacked notification probably works best on desktop, and can be a little cumbersome on mobile (which I guess is why we have the 'one per page' toast and snackbar components). It would be great if this could be considered part of the library at some point, on desktop, I think the 'allow multiple notifications' use case is probably quite common.

Well, notifications are OS-level so cross-apps. So they want to provide consistency there for those developers looking into how to build their stuff to look good with other peoples stuff. App-level notifications, that is what Snackbar is for. Nice simple alerts that something has happened.

Not wanting to nitpick, and perhaps I'm mis-interpreting , but I don't really read that section of the MD spec as OS-level only stuff. In fact they open with 'Notifications inform your app’s users about relevant and timely events in your app.'  'timely events in your app' makes it sound like that section is also addressing app-level notifications, but of a different type (or a different way) than that addressed by Snackbar/Toast. In fact the Snackbar component's section of the spec suggests Toast's themselves are primarily for system level messaging (which I assume is equivalent to OS-level?) with "Android also provides a toast, primarily used for system messaging."

Updated for Android N notifications
=> https://codepen.io/jfily/pen/eBEYmZ
No real specs but followed this : https://material.google.com/patterns/notifications.html#notifications-anatomy-of-a-notification

@jfily Can't find any specs for MD desktop app notifications either. Your CP looks good tho !

Just use the Notifications API


Using @iopipe/iopipe causes an upgrade warning to be displayed

When using @iopipe/iopipe , the latest version available is 1.7.0  However in the dashboard it tells you a newer version is available (1.11.0).   When you then go to do an NPM update you can't get later than 1.7.0 so the warning stays

https://dashboard.iopipe.com/project/d02087b2-71a3-4260-a984-9dfa9ae8a670/functions/arn:aws:lambda:us-east-1:554407330061:function:aws-elasticsearch-write-prod-es55-us-east-1/invocation/AWNQxPOl5wveYmiJohfV#start=2018-05-11T19%3A42%3A46.748Z&end=2018-05-11T19%3A57%3A46.748Z&timeframe=15m

Version mismatch here, closing.


The input is no longer normalized, right?

Thanks very much for your works, In the latest version there is no normalization before the network forward, and not even divided by 255. right?

Yes. Since we don't use ImageNet pre-trained weights, these preprocess are redundant.

@ruinmessi  Thank you very much.

@ruinmessi  Seems not /255 as well, directly using raw input? Does this helps boost performance?

@jinfagang It's only used for simplifying the preprocessing pipeline. Due to the existence of BN, we no more need to normalize the training image.


Revert Only pass --build-option to bdist_wheel in build_meta

Summary of changes

Reverts #4079 which broke the ability to pass build_ext arguments via config settings.
For example this command was broken by #4079 when building psycopg2.
python -m build -n -s -C--build-option=build_ext -C--build-option=--pg-config=/home/buildroot/buildroot/output/per-package/python-psycopg2/host/aarch64-buildroot-linux-gnu/sysroot/usr/bin/pg_config

Closes
Pull Request Checklist

[x] Changes have tests
[x] News fragment added in newsfragments/.
(See documentation for details)

Hi @jameshilliard , thank you very much for the contribution.
There is a problem of consensus here.
According to the discussion the solution of passing the arguments to all hooks also breakes the build process for some users in a different way...
The best here would be working towards a more prevalent solution.

There is a problem of lack of consensus here.
According to the previous discussion the solution of passing the arguments to all hooks also breakes the build process for some other users in a different way...

The impression I got from the various threads is that changing the behavior of --build-option like this would cause a regression and it seems nobody from that thread noticed #4079 to comment on it. It's unclear why the behavior for this flag which which was previously working correctly for a number of use cases was changed to do something very different without any usable replacement option for those use cases.

The impression I got from the various threads is that changing the behavior of --build-option like this would cause a regression and it seems nobody from that thread noticed #4079 to comment on it.

Hi @jameshilliard, I don't think this assessment is entirely accurate.
In the thread linked, there is a lot of people asking for a long term solution, which is the point being discussed in #4083.
When working on the error that prompted #4079, I proposed the patch (describing the approach I would use for it) in the same thread, so even if the participants did not had the time review #4079, they were informed about the proposed changes before they were applied.
Moreover, after the patch proposal, there was one comment "against", but my interpretation is that it was a comment in the nature "we need a proper/major fix or long-term solution" other than against the particular proposed patch (I did point that out in the thread).
In the same thread, I got feedback from one of pip's maintainers, that agreed the patch would be good enough for a quick-and-dirty solution that does not solve the overall problem, but does improve the error being discussed.
#4079 makes it clear that the change is not the final solution for the config_settings saga, but just a stopgap. Handling config_settings is a hard challenge because the way PEP 517 runs each build hook in a brand new process and because evaluating setup.py imposes a strong level of decoupling between setuptools.build_meta and the rest of setuptools. #4083 aims at discussing this challenge and trying to find solutions for it.

Now, my problem with this PR is that it fixes problems for one group of users, but re-introduces problems for another group of users... We are between a rock and a hard place here, so I don't think adopting this PR exactly how it is right now would be beneficial, we would only be caught in the middle of a tug of war...
The best approach in my opinion is proceeding towards #4083, which would fix the problem for all groups of users.

Suggested Workaround
Please also note that the way it is right now config_settings is at best supported in a BETA/best-case basis, for all intents and purposes... It is not something we can really consider an stable interface on the setuptools side.
The only known "stable" mechanism so far for passing configuration parameters for individual commands is via setup.cfg.
If no other tool is relying on DIST_EXTRA_CONFIG[^1], you can also use it to avoid patching an existing setup.cfg and instead add an extra one.
This is an arbitrary example that does not correspond to the reproducer mentioned in the PR original text, but gives a good idea on what is possible to do:
export DIST_EXTRA_CONFIG=/tmp/setuptools-build.cfg
echo -e '[build_ext]\nparallel = 8\n[bdist_wheel]\npy_limited_api = cp311' > $DIST_EXTRA_CONFIG
python -m build

[^1]: I know that cibuildwheel takes over DIST_EXTRA_CONFIG for their needs to communicate with setuptools.

When working on the error that prompted #4079, I proposed the patch (describing the approach I would use for it) in the same thread, so even if the participants did not had the time review #4079, they were informed about the proposed changes before they were applied.

I'm mostly referring to this comment which seems to quite clearly indicate that this would be a significant regression.
In that comment a stopgap approach for not breaking existing use cases is suggested as well and it's unclear to me why that approach was not taken instead of this one which breaks existing use cases without providing a usable replacement for those use cases.
See:

The easiest thing for setuptools to do in my mind is to introduce a set of “legacy” namespaced build options, one each for egg_info, sdist and wheel. Paper over the cracks while we hash out what is to become of setuptools’ config settings.

In the same thread, I got feedback from one of pip's maintainers, that agreed the patch would be good enough for a quick-and-dirty solution that does not solve the overall problem, but does improve the error being discussed.

When it comes to python packaging there seems to be an issue where non-pip based use cases are an afterthought.

Now, my problem with this PR is that it fixes problems for one group of users, but re-introduces problems for another group of users... We are between a rock and a hard place here, so I don't think adopting this PR exactly how it is right now would be beneficial, we would only be caught in the middle of a tug of war...

So why not just offer a new flag for the new behavior and preserve the previous behavior under the old flag as a stopgap?

Please also note that the way it is right now config_settings is at best supported in a BETA/best-case basis, for all intents and purposes... It is not something we can really consider an stable interface on the setuptools side.

So setuptools warns quite loudly that directly invoking setup.py is deprecated, which kinda implies that one should be able to transition to pep517. This approach of deprecating/breaking features before having working replacements are ready has made transitioning to pep517 quite difficult in general for distros, this was a major issue with even having a pep517 frontend working in the first place as it wasn't originally possible to bootstrap a pep517 toolchain properly as it was incorrectly assumed everyone could just use pip.

If no other tool is relying on DIST_EXTRA_CONFIG1, you can also use it2 to avoid modifying an existing setup.cfg and instead add an extra one.

Hmm, I'll give this a try and see if it's workable for us at all.
I'll reopen #4217 for now since it seems to at least partially reduce the breakage from this change.

I'm mostly referring to this comment which seems to quite clearly indicate that this would be a significant regression.

I wasn't aware that $DIST_EXTRA_CONFIG existed when I made that comment.  You might be able to generate a throw-away config file using process substitution for a more direct alternative to config settings - unsure if that's supported by the env var.

Hi @jameshilliard, thank you for taking the time to go through my comments and add your toughts.
I just want to highlight that I am definitely not against fixing this bug, and that when the patch you reported as problematic was introduced, to the best of my knowledge, it was not a given that it would end up causing exceptions (you were the first one to openly report crashes that to us, after a couple of months that the PR was merged). I honestly apologize for the inconvenience.
For the sake of increasing productivity and avoid further stress, the criteria that I am considering wheatear to accept PRs targeting the issue is:

The PR should not introduce other problems (or re-introduce)
The PR should not introduce significant changes in user interface that has not been previously discussed (e.g., adding a config_settings["--temporary-legacy-options"] would need to be first discussed).

... + the usual stuff (pass the tests, add relevant test cases, etc...)

I'm mostly referring to this comment which seems to quite clearly indicate that this would be a significant regression.

You probably already saw this, but these concerns and viability were discussed in:
https://github.com/pypa/setuptools/issues/2491#issuecomment-1743723820
https://github.com/pypa/setuptools/issues/2491#issuecomment-1745126670

In that comment a stopgap approach for not breaking existing use cases is suggested as well and it's unclear to me why that approach was not taken instead of this one which breaks existing use cases without providing a usable replacement for those use cases.

You mentioned a proposed stopgap approach that would not break existing use cases. Please note that the proposal was not accompanied by a PR and that adding more namespaces/flags/options/structure to the interface would require more discussion... My honest interpretation of this comment was: "this is not enough users have the need for passing config_settings to all build hooks"; to which my line of thought was: "look passing config_settings to all hooks is already broken right now, it does not work. Not doing anything does not help. Let's make --global-option and --build-option partially work as they were intended to be legacy backward compatible with pip, while we keep looking for a long-term solution".

When it comes to python packaging there seems to be an issue where non-pip based use cases are an afterthought.

This is a very fair comment, and I agree with that. Ideally we should involve people that have stakes of different use cases, so everyone has a say.
In Setuptools we are open to work with everyone and we try to find compromises that also would work for non-pip based use cases. There were changes in both pypa/distutils and pypa/setuptools code bases that were prompted and developed together with people from the fedora/debian/arch/nix/conda/... communities. We just ask everyone to please bear with us because setuptools/distutils is complex.
In this particular instance, it is clear that both config_settings["--build-option"] and config_settings["--global-option"] were added as a temporary backward-compatibility hack to interoperate with pip's legacy options. Not only they use the same name as the pip's options, but also they are hilariously convoluted to write in the CLI, which cannot be justified otherwise. So for better or worse, pip is the primary target. What happens is that people have other needs that, so out of necessity they have to resort to using this interface as well.
But equally in other places in the code base, you can find compatibility specific hacks for non-pip use cases.

So why not just offer a new flag for the new behavior and preserve the previous behaviour under the old flag as a stopgap?

Compounding more fragile APIs on top of an already fragile API does not sound like a good idea to me.
I might to sound like a broken record but: it was not immediately a given that the change would break other workflows. It took around 3 months for us to be reported otherwise - it there was an immediate report before pip have changed their code base, I would probably have reverted the change on the spot.
The reason why we are having the discussion in the first place is because config_settings implementation when PEP 517 was first introduced in setuptools was seriously lacky, if not completely unusable. In hindsight, probably it would be best to add a warning every time that config_settings was passed as a non-empty dict, so people should be at least aware that they are not stable (if anyone is interested on that, please feel free to open a PR).

So setuptools warns quite loudly that directly invoking setup.py is deprecated, which kinda implies that one should be able to transition to pep517. This approach of deprecating/breaking features before having working replacements are ready has made transitioning to pep517 quite difficult in general for distros, this was a major issue with even having a pep517 frontend working in the first place as it wasn't originally possible to bootstrap a pep517 toolchain properly as it was incorrectly assumed everyone could just use pip.

Please note that the whole transition to PEP 517 was extremely difficult to setuptools itself as well. We still have problems nowadays with it, and we still have spend tons of energy on it. We are all kind of in the same boat.
Since deprecation warnings and documentation changes are the only means setuptools have to let their users know that something will change at some point in the future, our approach is to add them as soon as possible. However, we still keep deprecated features around for months in a row (and more often years) to avoid major breaks...

I'll reopen https://github.com/pypa/setuptools/pull/4217 for now since it seems to at least partially reduce the breakage from this change.

Thank you very much. I think that is a good idea. If we can fix this problem without re-introducing other issues, that would be the best.

We probably can close this now, in favour of #4217.


[server] Use block cache for RMD column family in PlainTable format

Use block cache for RMD column family in PlainTable format

Use block cache for RMD column family in PlainTable format as its used only in write path and need not be stored in memory as plain table. Also added more configs to control pool ratios for index blocks, currently it is enabling the cacheIndexAndFilterBlocksWithHighPriority but the the LRUCache is constructed with highPriPoolRatio set to 0.
Setting default highPriPoolRatio to be 0.1, but probably needs tuning.
How was this PR tested?

Does this PR introduce any user-facing changes?

[x] No. You can skip the rest of this section.
[ ] Yes. Make sure to explain your proposed changes and call out the behavior change.

Hey @majisourav99 , do you have any bandwidth to iterate the PR and resolve the conflicts? We'd like to keep the open PR list short so folks can focus on high-priority ones and get changes into production ASAP. If there's no traction or interest to get this PR merged, we will close this PR later.


Update setup.md by removing broken script ref.

Link ref. is broken and should be removed
💡 Is there a permanent link that always points to the latest version of the script? I think that would be a better approach.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

We started putting the version number in the filename, so that we can provide a SHA sum (this was after the Codecov hack). So I don't think we can permalink.


chore: fix discv5 npm link

The link in the readme is pointing to the wrong discv5 npm package,discv5 => @chainsafe/discv5

Fixed via afafd08943fcb6084ab6bad0de18fbf44b4728ff, I updated the package list file there to accompany the readme change.


Does the leshan client demo simulator support below failures?

Hi guys, is there any way to simulate on leshan client below failures? Any chance to do it via configuration?

BootstrapFailureCause.UNAUTHORIZED
BootstrapFailureCause.NO_BOOTSTRAP_CONFIG
BootstrapFailureCause.REQUEST_FAILED
BootstrapFailureCause.FINISH_FAILED
BootstrapFailureCause.CANCELLED
BootstrapFailureCause.INTERNAL_SERVER_ERROR

(I suppose you are you leshan-bsserver-demo and leshan-client-demo , but my answer should be more or less usable too if you are using leshan-server-cf and leshan-client-cf  as library and you try to create your own bootstrap server)
Some should be done without client modification other can not :
UNAUTHORIZED :
Meaning : the device does not connect with expected credentials.
How to do it :

add config for your device saying it must use DTLS and connect to bs server without DTLS
OR use valid credentials of another clients (without changing your client endpoint name)

NO_BOOTSTRAP_CONFIG
Meaning : the device is authorized to talk to the server but there is no configuration to write.
How to do it :

Just connect to server without DLTS and without adding any configuration at server side.
OR add SecurityInfo to SecurityInfoStore without adding a BootstrapConfig to BootstrapConfigStore (can not be done with leshan-bsserver-demo without code modification for now)

REQUEST_FAILED
Meaning : if a request failed (and the BootstrapHandler says the bootstap session must be stopped).
How to do it :

I guess it's hard to do without modifying leshan-client-demo code maybe you can do it if you succeed to stop a client just after it receives a write request. (but that's sounds pretty much hazardous)

FINISH_FAILED
Meaning : same as above but for FINISHED request
How to do it :

same as above OR (writing an invalid config like adding 2 BS server to security Object)

CANCELLED
Meaning : this append when 2 bootstrap session opened for same client at same time
How to do it :

maybe you can succeed to do that launching your client,  stoping it in the middle of the bootstrap session and restarting here just after.

INTERNAL_SERVER_ERROR
Meaning : Append when there is an unexcepted error at server side
How to do it : You can not do it without modifying leshan-bsserver-demo and adding a kind of bug. if you face if without modification,  you probably find a bug please report it.

Wow! Great, what a quick and great answer :) Thanks for all the info. And yes, I'm implementing my own BS based on Leshan dependency.

By the way if you want to monitor bootstrap session.
You could have a look at BootstrapSessionListener
Should we close this issue ?

Yes, please, the answer was totally fine for me. Thanks

you're welcome


cuda memory

Hi,
thanks for this great work,
I have a question, since I want to run the training code in my desktop. I use a single 3090 GPU. I saw you use 8 GPUs and set batch size to 32. If I want to run the pre-train code in my GPU, what's the batch size should I set? because I set it to 2 but still out of cuda memory.
Thanks,
zihui

Hi,
we use v100 with 32gb mem that fits 4 samples; you can calculate accordingly to your gpu mem; another trick is you can make fewer sampling points such as 1024 to reduce the mem. Or you can accumulate the gradients, such as 2 times forward with 1 time backward to make the effective batch size large.
Note that larger batch size can contribute more to pre-training.

exactly

Thanks,  and I have another problem now.
When I run the ddp_main.py for S3DIS semantic segmentation training.
I find, in line 85 in semseg/datasets/voxelizer.py, the variable 'lim' is 4.
This 'lim' seems used for clip the point cloud.
But, here, the range of 'coords' is very larger.
For example,  the ' bound_min' is [-387, 1537, -10] and 'bound_max' is [193, 1840, 326] (this range varies for different point cloud).
So, use 4 as a radius to crop the point cloud will lead to an empty cropped point cloud.
BTW, I use MinkowskiEngine 0.5.4, but I checked the 'ME.utils.sparse_quantize', which seems used for reading the input data, its computation method doesn't have differences with 0.4.3 version.
Sorry for disturbing you again

Hi, sry for the late response.
I cannot reproduce your errors in ME043. Please see my screenshot:

Sorry, I got a mistake about it.
And I still have 2 quesrions.

I find you concatenete the batch_idx in front of coords, can you tell me the reason?
when I run the codes, in line133 of downstream/semseg/models/modules/common.py, it will raise an error:
AssertionError: Axis types must be None when region_type is given.

Thanks,
zihui

Hi,

That is the requirement of ME to process the different number of points in a batch
if migrating to ME0.5, you need some adaptions; region_type is one of them.


Nightly Build Failure 2024-04-03

Overview
What do you think happened, if anything?
The _out_ferc714__hourly_demand_matrix job ran into an OOM kill.
From the logs we can see that it started at 07:10:43 UTC and got the kill at 07:11:46 UTC.
here are the VM instance RAM metrics over the 07:00 - 07:20 timeframe, you can see that when the job started we were using ~35GB of RAM already and then we quickly jumped up to 60+ before getting killed at 07:12.

We also saw this same issue 2024-04-02, but didn't make an issue for it.
Next steps
What next steps do we need to do to understand or remediate the issue?
We could maybe force this asset to run non-concurrently.
We could maybe reduce the RAM usage of this asset.
Verify that everything is fixed!
Once you've applied any necessary fixes, make sure that the nightly build outputs are all in their right places.
- [ ] [S3 distribution bucket](https://s3.console.aws.amazon.com/s3/buckets/pudl.catalyst.coop?region=us-west-2&bucketType=general&prefix=nightly/&showversions=false) was updated at the expected time
- [ ] [GCP distribution bucket](https://console.cloud.google.com/storage/browser/pudl.catalyst.coop/nightly;tab=objects?project=catalyst-cooperative-pudl) was updated at the expected time
- [ ] [GCP internal bucket](https://console.cloud.google.com/storage/browser/builds.catalyst.coop) was updated at the expected time
- [ ] [Datasette PUDL version](https://data.catalyst.coop/pudl/core_pudl__codes_datasources) points at the same hash as [nightly](https://github.com/catalyst-cooperative/pudl/tree/nightly)
- [ ] [Zenodo sandbox record](https://sandbox.zenodo.org/doi/10.5072/zenodo.5563) was updated to the record number in the logs (search for `zenodo_data_release.py` and `Draft` in the logs, to see what the new record number should be!)

Relevant logs
link to build logs from internal distribution bucket
2024-04-03 07:10:03 +0000 - dagster - DEBUG - etl_job - be4a0899-56fa-429f-831a-93cd28c6648b - 1779 - _out_ferc714__hourly_demand_matrix - STEP_WORKER_STARTING - Launching subprocess for "_out_ferc714__hourly_demand_matrix".
2024-04-03 07:10:40 +0000 - dagster - DEBUG - etl_job - be4a0899-56fa-429f-831a-93cd28c6648b - 23579 - _out_ferc714__hourly_demand_matrix - STEP_WORKER_STARTED - Executing step "_out_ferc714__hourly_demand_matrix" in subprocess.
2024-04-03 07:10:42 +0000 - dagster - DEBUG - etl_job - be4a0899-56fa-429f-831a-93cd28c6648b - 23579 - _out_ferc714__hourly_demand_matrix - RESOURCE_INIT_STARTED - Starting initialization of resources [io_manager].
2024-04-03 07:10:42 +0000 - dagster - DEBUG - etl_job - be4a0899-56fa-429f-831a-93cd28c6648b - 23579 - _out_ferc714__hourly_demand_matrix - RESOURCE_INIT_SUCCESS - Finished initialization of resources [io_manager].
2024-04-03 07:10:43 +0000 - dagster - DEBUG - etl_job - be4a0899-56fa-429f-831a-93cd28c6648b - 23579 - _out_ferc714__hourly_demand_matrix - STEP_START - Started execution of step "_out_ferc714__hourly_demand_matrix".
2024-04-03 07:11:46 +0000 - dagster - DEBUG - etl_job - be4a0899-56fa-429f-831a-93cd28c6648b - 1779 - _out_ferc714__hourly_demand_matrix - ENGINE_EVENT - Multiprocess executor: child process for step _out_ferc714__hourly_demand_matrix was terminated by signal 9 (SIGKILL). This usually indicates that the process was killed by the operating system due to running out of memory. Possible solutions include increasing the amount of memory available to the run, reducing the amount of memory used by the ops in the run, or configuring the executor to run fewer ops concurrently.

 We do this for the core_epacems__hourly_emissions asset because using all CPUs locally destroys our computers.
Side note: I don't know if we're are using all CPUs for processing EPA CEMS on the VM. Is this intentional? In gcp_pudl_etl.sh we're using the default number of works for the --epacems-workers argument which is 2.

With 64MB of memory in the nightly builds, we could definitely be using more cores on EPA CEMS, but we're approaching 400 assets, and it's not deterministic which ones will be running at the same time. Right now the FERC-714 imputations and the EPA CEMS are the long poles. I really wish there were some way to chunk the EPA CEMS into a smaller memory footprint. The state-year chunks we had before were very manageable and let us do a lot more parallelism.
I wish there were some way for Dagster to explicitly manage memory usage as well as CPU usage.

Short-term, reducing the concurrency for the internally parallelized work of the FERC-714 asset seems like the simplest fix!

I would be tempted to blame the extremely parallelized extraction of spreadsheets, which we've just added a bunch of additional work to, but that happens right at the beginning of the ETL, and this crash isn't happening for 5 hours!

TBH I am also very confused how the ETL can take 5 hours to get to this point.  With 16 CPUs and 64 GB of memory, it's 5 hours into doing something that takes my laptop with 10 CPUs and 32 GB of memory 2 hours tops.

The only other thing in there that I could imagine being unusually resource intensive is the FERC to FERC matching step, but I don't know if that particular merge is a big memory hog.

TBH I am also very confused how the ETL can take 5 hours to get to this point. With 16 CPUs and 64 GB of memory, it's 5 hours into doing something that takes my laptop with 10 CPUs and 32 GB of memory 2 hours tops.

The ETL returns with a failure code 5 hours in, but if you look at the timestamps we see that the ETL starts at 06:08 UTC:
2024-04-03 06:08:39 +0000 - dagster - DEBUG - ferc_to_sqlite_job - 20634ea8-e16f-4b96-b880-c208266b0a27 - 102 - RUN_START - Started execution of run for "ferc_to_sqlite_job".

And then the OOM happens an hour in, at 07:11:
2024-04-03 07:11:46 +0000 - dagster - DEBUG - etl_job - be4a0899-56fa-429f-831a-93cd28c6648b - 1779 - _out_ferc714__hourly_demand_matrix - ENGINE_EVENT - Multiprocess executor: child process for step _out_ferc714__hourly_demand_matrix was terminated by signal 9 (SIGKILL). This usually indicates that the process was killed by the operating system due to running out of memory. Possible solutions include increasing the amount of memory available to the run, reducing the amount of memory used by the ops in the run, or configuring the executor to run fewer ops concurrently.

And then finally the run fails:
2024-04-03 11:00:06 +0000 - dagster - ERROR - etl_job - be4a0899-56fa-429f-831a-93cd28c6648b - 1779 - RUN_FAILURE - Execution of run for "etl_job" failed. Steps failed: ['_out_ferc714__hourly_demand_matrix'].

Oooh, interesting. So it's just the one asset materialization that fails, and everything else keeps running -- the VM doesn't crash. I wonder if we could just add a retry to this asset?

On second thought, I might be misunderstanding dagster's concurrency limit feature:

Limits can be specified for all ops/assets with a specific tag key or key-value pair. If any limit is exceeded by launching an op/asset, then the op/asset will be queued.

I think limiting pudl.etl.epacems_assets.process_single_year using tags works because we are running this function multiple times. Our default value for CEMS is 2. Does the tag limiting capability restrict the number of processes available to the specific op or for all ops trying to run in the job. For example, when pudl.etl.epacems_assets.process_single_year is running can other ops be running?

This happened again on 2024-04-04, which is to be expected because #3541 wasn't merged yet. Just noting that down here for posterity.

@jdangerx do you think that #3541 would have any effect? There's already only 1 CPU being used in the asset that's getting killed.
This problem seems to be happening reliably in the nightly builds, but I've still never seen it running the full ETL locally on a machine with half as much memory.  If it's just that a couple of high memory assets happen to be running at the same time could we use an op retry to just keep attempting to materialize it until there's enough free memory for it to complete? Or does the fact that it's getting killed rather than raising an exception mean that won't work. Locally this asset takes ~3 minutes to run and peaks at ~4GB of memory usage.

I think the ETL fails more often on the VM because GCP VMs do not have swap enabled by default:

Caution: You must ensure that tmpfs volumes on your instance do not use up all of your instance memory. By default, Compute Engine instances do not have any swap space. If you use all of the available instance memory in a RAM disk, the instance can crash and potentially cause you to lose your data.

I don't think it's trivial to enable it.


Fix error output on Linux

Motivation
On linux, this action makes an error log every time.
example:  https://github.com/fwal/setup-swift/actions/runs/1506129234

Integrate Swift 5.5 on ubuntu-latest
Failed to setup requested swift version

I want to calm down the error logs.
Reason
getVersion function anytime returns nullon linux .
The regex searches Apple Swift version.
https://github.com/fwal/setup-swift/blob/a94947b51dc16f23683cbdded9b5313f716b04b9/src/get-version.ts#L32
But the output of swift --verion on Linux does not contains Apple keyword.
macOS:
$ swift --version
swift-driver version: 1.26.9 Apple Swift version 5.5.1 (swiftlang-13<IP_ADDRESS> clang-13<IP_ADDRESS>)
Target: x86_64-apple-macosx12.0

Linux(running on docker for mac):
$ docker run -it --rm swift:5.5 /usr/bin/swift --version
Swift version 5.5.1 (swift-5.5.1-RELEASE)
Target: x86_64-unknown-linux-gnu

How to fixed
just removed Apple keyword from the regex.

Hi @sidepelican and thanks for the PR!
Could we add a test case with the output from linux to avoid any regressions of this in the future? 🙂
You can add it in /__tests__/get-version.test.ts
Then, to fix the linting issues please run npm run format and commit the changes. 👍🏻

Thanks for the review!
I add test case and formatted.

Nice! 👍🏻


Replace pkg_resources with importlib.metadata; bump minor version number

pkg_resources is depreciated.
I changed the required plaster-yaml version number to correspond with the bumped version number in a pull request for that package that also removes pkg_resources.  (plaster-yaml pull request #3)  This removes some deprecation warnings from the regression test run.
Notes on backported standard library modules:
I like to conditionally import backported standard library modules.  I like to see the version number in the code so the version number is explicit, and in an obvious place.  That way it is clear when it is time to remove the conditional import and just use the standard library.
I find that creating a class to act as the top-level module in the module path, when the backported library is not top-level, helps with mocks when unit testing.

Hi @kpinc
Thank your for this update.
importlib_metadata backport has been a pain and I tend to avoid it.
Since importlib.metadata is available since python 3.8 and python 3.7 reach its EOL,
can you please take the time to adjust the merge request and set
python = ^3.8

https://github.com/mardiros/pyramid-helloworld/blob/master/pyproject.toml#L11
and drop all the complexity tied to importlib_metadata >

Hi Guillaume,
On Sat, 30 Mar 2024 01:32:07 -0700
Guillaume Gauvrit @.***> wrote:

Thank your for this update.

You're welcome.  I happened to recently remove pkg_resources
from another project.  I'm interested in using plaster-yaml
and it seemed straightforward to make the change.
(Of course it took longer than anticipated.)

importlib_metadata backport has been a pain and I tend to avoid it.
Since importlib.metadata is available since python 3.8 and python 3.7
reach its EOL,
can you please take the time to adjust the merge request and set
python = ^3.8

https://github.com/mardiros/pyramid-helloworld/blob/master/pyproject.toml#L11
and drop all the complexity tied to importlib_metadata ?

Will do.  Thanks for the prompt reply.  Unfortunately I can't
do much until at least Monday.
Regards,
Karl @.***>
Free Software:  "You don't pay back, you pay forward."
-- Robert A. Heinlein

On Sat, 30 Mar 2024 05:38:24 -0500
"Karl O. Pinc" @.***> wrote:

On Sat, 30 Mar 2024 01:32:07 -0700
Guillaume Gauvrit @.***> wrote:

Since importlib.metadata is available since python 3.8 and python
3.7 reach its EOL,
can you please take the time to adjust the merge request and set
python = ^3.8

https://github.com/mardiros/pyramid-helloworld/blob/master/pyproject.toml#L11
and drop all the complexity tied to importlib_metadata ?

Will do.  Thanks for the prompt reply.  Unfortunately I can't
do much until at least Monday.

On second thought, I'm going to wait to do this until you
decide what python versions you want to support in
plaster-yaml.  Because plaster-yaml uses pyramid-helloworld
for regression tests the latter should run on the versions
the former supports.
Regards,
Karl @.***>
Free Software:  "You don't pay back, you pay forward."
-- Robert A. Heinlein


Plugin description migration probe

Description
Resolves #427.
The idea is to detect plugin still using the description tag of their pom.xml to hold the plugin description.
This should influence the documentation scoring and help resolving the problem by pointing to https://www.jenkins.io/doc/developer/tutorial-improve/move-description-to-index/

Testing done

### Submitter checklist
- [x] If an issue exists, it is well described and linked in the description
- [x] The description of this pull request is detailed and explain why this pull request is needed
- [x] The changeset is on a specific branch. Using `feature/` for new feature, or improvements ; Using `fix/` for bug fixes ; Using `docs/` for any documentation changes.
- [ ] If required, the documentation has been updated
- [x] There is automated tests to cover the code change / addition or an explanation why there is no tests in the description.

I am planning to implement these issues during the GSOC period

as you can see, the implementation is done.

if i am selected, that;s why I raised the issue for confirmation of whether they are valid or not.

You shouldn't have create this in a issue on the project but raised your concerns on gitter


Multiple column sorts?

Seems like this was partially discussed / referenced in the issue regarding turning off sorting, but I just wanted to clarify: are there any near-term plans / upcoming PRs to support multi-column / complex sorts (like in the way react-datagrid supports multiple column sorts basically by default)?
If not, perhaps we'll contribute one soon...
Thanks!

I don't have any concrete plans for multiple column sorting. I can see it's quite neat, though, and I can appreciate the idea.
Currently SurviveJS related efforts take most of time given that brings the bread to the table. My smaller projects like this are in "discuss issues and accept PRs" mode. :smile:
Given Reactabular's design, sorting logic (src/sort_column.js) isn't coupled with it. Now it's just provided with the distribution for convenience. I believe it would be a good idea to aim to extract that out of Reactabular and then provide instructions on how to use the logic with it. We might not need new hooks even. I think this would be a good way to solve it.
I did something similar with react-pagify. I was able to extract its core logic to segmentize.
The cool thing about this kind of split is that it allows you to reuse the logic outside of React. It's a little theoretical point at the moment, but I don't see any harm in that. Extension points like this are always good and consistent with the way people like to use npm.

Thanks for the quick response!
(Very much enjoyed the SurviveJS book by the way, glad you're continuing with that effort...)

There's support for sorting multiple columns now. I simplified the interface further in the current beta.


delete/update post - issue #123

this addresses issue #123
I had to change AWS configuration to my own bucket to get it working locally. I also added some configuration to get capybara working correctly, specifically for a user to sign in.
A user can delete a post, and update it - but only update the caption.
Let me know of any questions or comments!

Sweet work @jk1dd - This is really helpful. I think the next step is for some styling and perhaps a confirmation button for the delete action. Can all users modify all posts @jk1dd?
Any thoughts @Pratt0923 @maebeale?
Here are the places I think need some styling. These would be new issues for anyone to pick up.

@kalimar currently, all users can modify all posts! i can absolutely add some logic to restrict that to the post creator, if that's the way you'd like to go.

thanks @jk1dd !
we do want all users to be able to add/edit tags on existing posts, but not necessarily be able to edit the photo or title.
we will need some admin-type permissions so they can hide a post deemed problematic, so were leaving all posts editable until we could add the permissions. if you're up for adding the permissions and then making posts only editable by the creator and any admin, that'd be amazing!!!


additional help description

For the --encode option

Codecov Report

Merging #127 into master will decrease coverage by 0.13%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #127      +/-   ##
==========================================
- Coverage   94.29%   94.15%   -0.14%
==========================================
  Files           6        6
  Lines         719      719
  Branches      120      120
==========================================
- Hits          678      677       -1
  Misses         27       27
- Partials       14       15       +1

Impacted Files
Coverage Δ

webssh/settings.py
100% <ø> (ø)
:arrow_up:

webssh/handler.py
97.12% <0%> (-0.24%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f2c9766...6be01b9. Read the comment docs.


Slow performance in async execution because of DirectivesExtension

The use of DirectivesExtension in async graphql execution wraps every field of the result graph in an async function. This slows down performance.
We have one resolver returning a huge list of objects where every field of the model is already filled with a native type (str, int ..). So there is no need for an asynchronous resolve of these model fields.
But the DirectivesExtension calls await_maybe for each field of each model in the result list. Now there are thousands of async function calls which should be awaited.
We don't use custom directives so I replaced the Schema.execute function (from strawberry.schema.schema.py). In this function I only register the DirectivesExtension if there are custom directives. For us this solves the problem, as long as we don't use custom directives. But I think it would be a modification that could also be merged in the strawberry sourcecode.
Here is a demo script with the used patch, showing the dramatic performance improvement for this case.
import asyncio
import time
import typing
from typing import Any, Dict, Optional

import strawberry
from strawberry import Schema
from strawberry.extensions.directives import DirectivesExtension
from strawberry.schema.execute import execute
from strawberry.types import ExecutionContext, ExecutionResult

@strawberry.type
class Book:
    author: str
    title: str

async def get_books():
    res = []
    await asyncio.sleep(0.1)
    for i in range(200000):
        res.append(
            Book(
                author="Unknown",
                title=f"Book nr {i}",
            )
        )
    return res

@strawberry.type
class DemoQuery:
    books: typing.List[Book] = strawberry.field(resolver=get_books)

async def test():
    start_time = time.time()
    schema = strawberry.Schema(query=DemoQuery)
    query = "{ books { title author } }"
    result = await schema.execute(
        query,
    )
    if result.errors:
        print(result.errors)

    print("complete", time.time() - start_time)

async def schema_execute(
    self,
    query: str,
    variable_values: Optional[Dict[str, Any]] = None,
    context_value: Optional[Any] = None,
    root_value: Optional[Any] = None,
    operation_name: Optional[str] = None,
) -> ExecutionResult:
    # Create execution context
    execution_context = ExecutionContext(
        query=query,
        schema=self,
        context=context_value,
        root_value=root_value,
        variables=variable_values,
        provided_operation_name=operation_name,
    )

    # BEGIN - CHANGED
    extensions = list(self.extensions)
    if len(self.directives) > 0:
        extensions += [DirectivesExtension]

    result = await execute(
        self._schema,  # pylint: disable=protected-access
        query,
        extensions=extensions,
        execution_context_class=self.execution_context_class,
        execution_context=execution_context,
    )
    # END - CHANGED

    if result.errors:
        self.process_errors(result.errors, execution_context=execution_context)

    return result

def apply_directives_patch():
    Schema.execute = schema_execute

def run():
    asyncio.run(test())

def run_with_patch():
    apply_directives_patch()
    asyncio.run(test())

if __name__ == "__main__":
    print("run")
    run()
    print("run with patch")
    run_with_patch()

On my Machine I get the following output:
run
complete 24.10178303718567
run with patch
complete 4.264432191848755

hi @QSHolzner, I'll take a look at this, ut I think it makes sense. I wonder if we can make it faster but still enable directives.
But in any case I think we can disable the extension when there's no custom directive added

@patrick91 Thanks

Hello, we have witnessed the same issue and applied the same patch. This drastically reduce the time took by strawberry to serialize the response (divided by 4 or 5) but i still find it quite slow especially as the data we hand out to strawberry is basically already serialized - Do you see any other area of improvements in the near future ?


Question: how is it possible to integrate the form into an electron window, without using the web-browser?

Please, I just need an answer to this:
How is it possible to integrate the form into an electron window, without using the web-browser?

There is no notable difference for the library when in electron or in a real browser.
It should be possible to have it in an electron window.
What have you tried ?


Error: Key is not included in URL

title: "Key is not included in URL"
detail: "The URL does not support the key 946591862"
I can't find anything in the documentation explaining what this actually means - what does "Key" refer to here?
I'm bumping into it in my Controller tests, for context.
The following code triggers it.
sign_in users(:one)
put :update, id: model_id , params: { data: update_attributes, id: model_id }

Ah. The issue I have is that I was using different ids inside and outside the data hash

Thanks for commenting here after finding the solution. It really is a life saver.


MUI Error when trying to change the language with v5

Describe the bug
After upgrading to v5, I've noticed that switching between languages is not possible anymore, with the following error printing in the console: Warning: Use the defaultValueorvalueprops on <select> instead of settingselected on <option>.
I can't select any option in the select component.
To Reproduce
Steps to reproduce the behavior:

Configure languages with the Editor component
Try to change the language for a block -> Error!

Expected behavior
The language switches correctly.
Screenshots

Additional context
Using v5.
Thanks for looking into this!

:tada: This issue has been resolved in version 5.0.1 :tada:
The release is available on:

v5.0.1
GitHub release

Your semantic-release bot :package::rocket:


Omniauth

Opening PR from omniauth branch into master for production environment compatibility signing in with Github

Confirming this is good to go.


[Winter of Code] Roadmap for Data structures and Algorithms

🚀 Feature
As mentioned in my proposal, I would like to create a big of roadmap for helping the beginners with data structures and Algorithms and how they can go about starting with data structures
Motivation
When I first started out as a beginner it was extremely tough for me to figure out how to go about with DSA. Hence, I think there are many people searching for the same.
Pitch
When people come to NeoAlgo, they get to see DSA implemented in different languages but they might get confused as to where could they start especially if they're beginners. So this would make NeoAlgo beginner friendly and help them go about understanding the repo more easily.

Hello @joeyouss
Thanks for raising this! You can go ahead with this and make sure that your Roadmap should be in a Markdown Format and pushed to the resources/ directory, which isn't created yet. Please keep me in a Feedback Loop over Slack so that I can review it right there before you make a PR.

Thank you! I will do in the same format and submit it


Update Basic Types.md

Fixes #

Template string literals are allowed to have new lines. this one of the reasons the templates were added to JS. The sample demonstration this ability.


[Question] Unknown device issues when trying to setup as a Galaxy S21

First off let me apologize for posting this if it is a silly question, and or the wrong place.
So, I set all my device parameters;
deviceSet = {
            "app_version": "2<IP_ADDRESS>.75",
            "android_version": 30,
            "android_release": "11.0.0",
            "dpi": "480dpi",
            "resolution": "1080x2400",
            "manufacturer": "Samsung",
            "device": "Galaxy S21",
            "model": "SM-G991B",
            "cpu": "snapdragon",
            "version_code": "314665256"
        }
        user_agent = f'Instagram {deviceSet["app_version"]} Android ({deviceSet["android_version"]}/{deviceSet["android_release"]}; {deviceSet["dpi"]}; {deviceSet["resolution"]}; {deviceSet["manufacturer"]}; {deviceSet["device"]}; {deviceSet["model"]}; {deviceSet["cpu"]}; en_US; {deviceSet["version_code"]})'
        #UA Translates to: "Instagram 2<IP_ADDRESS>.75 Android (30/11.0.0; 480dpi; 1080x2400; Samsung; Galaxy S21; SM-G991B; snapdragon; en_US; 314665256)"
        cl = Client()
        cl.set_country('US')
        cl.set_country_code(1)
        cl.set_locale('en_US')
        cl.set_timezone_offset(-21600)
        cl.set_user_agent(user_agent)
        cl.set_device(device = deviceSet)

However, Instagram still registers it as an unknown device. I am trying to make the profile of a Galaxy s21, and I just cant seem to figure out what I messed up or am forgetting.
Thank you for any help! ♥

Edit: Fixed.

How did you fix this issue?

How did you fix this issue?

I believe it just dealt with my formatting for my device settings.
If i am not mistaken, I was just passing the args incorrectly.
(So if you're having this issue, take a look there, make sure everything is formatted/passed properly.)
Sorry my response isn't of much help directly.

how can I get the profile of my android device like you did?


Name conflict when a struct is called "Context"

As discussed in #1242: I'm getting the following error when a struct is called Context when calling wasm-pack test --headless --firefox:
  [3/6] Compiling tests to WASM...
     Compiling compose-area v0.1.0 (/home/danilo/Projects/compose-area)
  error: linking with `rust-lld` failed: exit code: 1
    |
    = note: "rust-lld" "-flavor" "wasm" "--no-threads" "-z" "stack-size=1048576" "--stack-first" "--allow-undefined" "--no-entry" "--fatal-warnings" "--export-dynamic" "--no-
    = note: rust-lld: error: duplicate symbol: __wbg_context_free
            >>> defined in /home/danilo/Projects/compose-area/target/wasm32-unknown-unknown/debug/deps/libcompose_area.rlib(compose_area.1zjlr9g8cstvtdrr.rcgu.o)
            >>> defined in /home/danilo/Projects/compose-area/target/wasm32-unknown-unknown/debug/deps/libwasm_bindgen_test-258015b5016fb118.rlib(wasm_bindgen_test-258015b501

Probably a name mangling issue.

Thanks again for spinning this off!
This is actually somewhat difficult to support now that I think about it. Right now all the exports from Rust are generated into a flat namespace in JS, so this is actually a legitimate namespace conflict. I'm not sure we can really support duplicate-ly named types in separate crates being linked together because we wouldn't know what to call the names in JS!
What I do think, however, is that the test framework should be using less common names :)

Do we have access to the source crate of a symbol? Maybe that could be integrated into the name?

We do! All that would end up with though is a better error message here, which is certainly an improvement! I'm nto sure we could actually allow this still

I've posted a rename for our own Context in https://github.com/rustwasm/wasm-bindgen/pull/1280

Thank you!
Feel free to close this issue, unless it should stay open to discuss further namespacing / name mangling.

Ok I think in that case I'll go ahead and close

Thank you so much for this comment --- it got me unstuck.
I found this issue when looking up "wasm_bindgen duplicate symbol".
In my particular case, I have multiple crates in a cargo workspace that followed similar strut & method naming patterns, e.g. both workspace crates "foo" and "bar" had structs with names and methods like this:
impl Consume {
   //  . . .

   #[wasm_bindgen]
   pub fn start(/* ... */) {}
}

and the solution was to re-name the structs to be "crate specific", e.g. FooConsume and BarConsume respectively. By doing this, the generated symbol names would be *_foo_consume_start & *_bar_consume_start instead of the duplicate-ly generated name *_consume_start.

I'm not sure we can really support duplicate-ly named types in separate crates being linked together because we wouldn't know what to call the names in JS!

As a complete noob: Is it possible for #[wasm_bindgen] to automatically shove-in the current crate's name into the link_name to help "mangle" the name a bit more? wasm_bindgen is still sort of voo-doo magic for me lol.