API Definition of User Service V3

We want to give our customers the possibility to define their own user schemas and create users according to those schemas. To avoid breaking changes on our current user service, we will introduce the version v3 and provider a migration path later on.
To get an impression of what the service is capable of and how this looks like in the future, we want to start with the API definition, which also gives us the possibility to collect feedback in an early state.
Acceptance Criteria

[ ] API is behind a feature flag and marked as Alpha? Or not ready.
[ ] API is defined in a new User Service V3 with proto definitions
[ ] All requests and methods are documented with examples
[ ] User Schema Requests:

[ ] ListUserSchemas: lists latests states schemas
[ ] UserSchemaByID: shows details of a specific schema
[ ] CreateUserSchema: adds a new schema
[ ] UpdateUserSchema: changes an existing schema.
[ ] DeactivateUserSchema: moves the schema to a read only state. Users of this schema cannot be updated anymore. Users are still able to log in
[ ] ReactivateUserSchema: moves the schema state to active
[ ] DeleteUserSchema: Removes an exisiting schema. This operation is only allowed if there are no associated users
[ ] ValidateUser: Verifies if the user satisfies a schema

[ ] User Request:

[ ] ListUsers: lists users
[ ] UserByID: shows details of a specific user
[ ] CreateUser: adds a new user
[ ] UpdateUser: changes an existing user. The type of a user cannot be changed
[ ] DeleteUser: Removes an exisiting user.
[ ] AddAuthenticator: Adds an authenticator
[ ] RemoveAuthenticator: Removes an existing atuehticator
[ ] LockUser: changes the state of the user to locked
[ ] UnlockUser: changes the state of the user to active after it was locked
[ ] DeactivateUser: changes the state of the user to inactive
[ ] ReactivateUser: changes the state of the user to active after it was locked

Additional Information
User Object

id: unchangeble, unique identifier of the system
schema:

type: schema type of the user (e.g. human/machine)
revision: read only, revision of the schema the user was stored the last time (e.g. 4)

authenticators: list of possibilites how a user can authenticate itself. there can be multiple authenticators of the same type
contact: email and phone of the user if provided
state: one of Active, Inactive, Locked, Deleted
data/profile/traits: custom fields of a user based on the type (whats the best name for this?)

Authenticators

username: human readable identifier of the user

username
isOrgSpecific: if set to true, login with this authenticator is only possible if org is defined TODO: add to LoginPolicy (Authenticator list as proposal)

password
webAuthNKey
one time password

email
phone

time based one time password
jwt profile key
identity provider

identity provider
isOrgSpecific: if set to true, login with this authenticator is only possible if org is defined TODO: add to LoginPolicy (Authenticator list as proposal)

FUTURE: recovery code

User Schema

id: read only unique identifier
state: state of the schema
type: single word which describes the schema
revision: read only version of the schema, each update increases the revision
schema: describes the user
possibleAuthenticators: defines the possible types of authenticators. this allows creating different user types like human/machine without usage of actions to validate possible authenticators. Removal of an authenticator does not remove the authenticator on a user

TODO: move user.isOrgSpecific to here (on username and idp)

permissions
permission
Following permissions are allowed r (4), w (2), rw (6)

admin: defines the permissions of the admin
user: defines the user of the user

oidcClaim / samlMapping
It is needed to define the oidc Claim and saml mapping so ZITADEL knows how to map the user infromations to the oidc / saml responses/tokens
additional formats

phone: possibility to add declare phone numbers
language: language tag analog to ietf bcp 47

@stebenz @eliobischof Can you please estimate this issue?

Technical estimation:

service definition: 1d
proto message definition: 2d
permission checks definition: 1d
examples: 1d
documentation in proto: 1d


Check if editing node templates causes all dropdowns to reset to defaults in 2.5.x/2.6.x

Internal reference: SURE-2978
Reported in 2.4.15
Issue description:
When navigating to node templates and choosing EDIT it seems the dropdowns for Datacenter, Datastore, Network, etc all get reset from the previous setting to the defaults. This requires changing every single setting back to the correct setting each and every time a node template is modified.
We should check if this happens in 2.5.x/2.6.x though as noted, repro is tricky.
Business impact:
Resetting all of the values for the vmware nodepool is quite annoying. Can also result in the wrong values being entered (manually) resulting in downtime or any other imaginable scenario
Troubleshooting steps:
We checked to make sure that the original values still existed in the mode template via the API and they did. So it appears that this is a UI issue of some sort not gathering the data and placing it into the form.
Repro steps:
Unknown. This appears to only happen with vmware downstream clusters from the reported user.
Actual behavior:
All values from the dropdown menus being set to "defaults" or the top value
Expected behavior:
Dropdowns being populated with the values from the node template object

Related SURE is closed, so getting reproduction steps is out the window for now.
Closing as a result.


Error: Oops, we were not able to find the service on dashboard after 200 simulated deivces added (standard RM)

Type of issue

[x] Bug
[ ] New feature
[ ] Improvement

Description
This issue is originally reported from the customer.
(Oleksandr Shkabura<EMAIL_ADDRESS>Create standard RM via PCS CLI and then add 200 simulated devices.
Dashboard shows "Error: Oops, we were not able to find the service." due to below error.
HTTP404: NOT FOUND - The server has not found anything matching the requested URI (Uniform Resource Identifier).
(XHR)GET - https://saki-standard-1.azurewebsites.net/telemetry/v1/alarms?devices=chiller-01.46%2Cengine-01.42%2Cengine-01.18%2Cdelivery-truck-01.19%2Cdelivery-truck-01.9%2Cengine-01.47%2Cengine-01.27%2Cdelivery-truck-01.37%2Cchiller-01.32%2Cdelivery-truck-01.42%2Cchiller-01.44%2Cengine-01.8%2Cengine-01.26%2Cengine-01.6%2Cengine-01.7%2Cchiller-01.4%2Cengine-01.9%2Cdelivery-truck-01.13%2Cchiller-01.3%2Cdelivery-truck-02.0%2Cengine-01.44%2Cchiller-01.37%2Cdelivery-truck-01.4%2Cchiller-01.39%2Cdelivery-truck-01.35%2Cengine-01.12%2Cchiller-01.11%2Cchiller-01.16%2Cengine-01.22%2Cengine-01.34%2Cdelivery-truck-01.25%2Cdelivery-truck-01.33%2Cengine-01.33%2Cdelivery-truck-01.2%2Cdelivery-truck-01.23%2Cchiller-01.36%2Cprototype-02.0%2Cchiller-01.12%2Cchiller-01.23%2Cdelivery-truck-01.8%2Cchiller-01.30%2Cchiller-01.20%2Cchiller-01.27%2Cchiller-01.22%2Cchiller-01.7%2Cdelivery-truck-01.44%2Cengine-01.49%2Cdelivery-truck-
Note:

Confirmed that telemetry service was up and running.
Could be related to the issue that URL is too long (see the above request URL is not fully completed format)
There was no authorization error

Screenshot

Steps to reproduce

Create standard RM via PCS CLI
Add 200 simulated devices

Expected behavior
Dashboard displays 200 devices without an error.
Current behavior
Dashboard displays "Error: Oops, we were not able to find the service."
Known workarounds
...
Possible solution
...
Context and Environment

Operating System: ...
GitHub branch: ...
.NET Runtime: ...

Duplicate of #108. Closing this and we are already working on the original bug.


Add Archiving of Profiles

Description
This PR builds on:

https://github.com/zoe/covid-tracker-react-native/pull/404
https://github.com/zoe/covid-tracker-react-native/pull/402

It wires up an "Edit Profile" screen, where a user can Archive secondary profiles.
There's a confirmation popup modal, followed by an invitation to give a modal.
Backend PR is under review:
https://github.com/zoe/covid-app/pull/206
On what platform have you tested the change?

[ ] Android - Emulator
[x] Android - Physical device
[x] iOS - Emulator
[ ] iOS - Physical device

Screenshots

### Archive API Call

Checklist

[ ] I have updated mockServer
[ ] I have run npm test
[ ] I have run npm run test:i18n

Out of scope and potential follow-up
Follow up PR to clean up the passing of Profile vs Patient - needs discussion.

Lgtm, when backend is working lets test and then merge


Incorrect PR - closing


Espose charset of TextDocument in API

see https://github.com/Microsoft/vscode/issues/3817#issuecomment-210062644

duplicate of  #4958


Add support for onopen and onmessage

Hey there,
I like your approach a lot and therefore chose to extend it with the APIs I need.
I would love to have support for onopen and onmessage, I also added some tests.
Please be aware that there is a breaking change included, the ready state is now set to ready only after the emitOpen callback is called. I would like to have this behavior as I want to test this on a fine-granular level, I hope this is ok with you. Otherwise, we need to think of alternative APIs here.

Hi @DanielMSchmidt! You changes seem good to me, I don't mind the breaking change, we'll bump semver accordingly!

Published as 2.0.0! Thanks again for the contribution.


Wrong Twitter handle is associated with my account

My profile has the right GitHub username (sdball) but the wrong Twitter handle. It's stuck on my old Twitter username (stephenballnc) and not getting my new Twitter username (MutableBall) which is current in my GitHub profile.

You should be able to disconnect and then reconnect your Twitter handle on your profile: https://elixirstatus.com/edit_profile

Thank you for taking a look! But when I click "Disconnect twitter handle" there's a spinner but then it returns to the original state.


fix: timeout on shellcheck -V

Fix #179

@marveloo could you try with this version? You can download the vsix in the release job, under Artifacts (once the job succeeds).

Great. I will give it a try for a couple of days and let you know whether the issue is still present.


Handler not found for alert message

it looks like "alert" is a type of message: https://github.com/Rantanen/node-dtls/blob/e7410c20d839fe95dfa76552f932bea05ad8f8b8/dtls.js#L8
for which there is no handler assigned: https://github.com/Rantanen/node-dtls/blob/e7410c20d839fe95dfa76552f932bea05ad8f8b8/DtlsSocket.js#L124-L134
are consumers of this lib expected to assign one?

It's one of the missing bits mentioned in the README.md. I don't remember how the specification defines the 'alert' messages - whether they are protocol level errors that should be handled within the socket or whether they should be handled in user code.
I guess a simple addition would be to emit them as 'error' events similar to how the applicationData is emitted as 'message' events. But this would then leave the responsibility of reacting to them to the user code.
Can you give an example of an alert you're receiving?

{ type: 21,
  version: { major: -2, minor: -3 },
  epoch: 0,
  sequenceNumber: <Buffer 00 00 00 00 00 03>,
  fragment: <Buffer 02 28> }

https://tools.ietf.org/html/rfc6347#section-<IP_ADDRESS>

so it sounds like you're spec compliant from reading that section.


Test internal/private variables in contracts

When testing the attacks it is good to have access to values of internal or private variables of the vulnerable contracts because it allows us to access wether the vulnerability was exploited or not.
Currently hardhat does not allow that: https://github.com/NomicFoundation/hardhat/issues/2645

use ethers.provider.getStorage(address, slot) for reading private variables from a contract


Change description of drop_remainder argument of Dataset method window

In general, several windows are dropped when drop_remainder is set to True.

@googlebot I signed it!


Change configuration without rebuilding

Is there a way to change Airflow configuration without rebuilding the image?

Sorry but no. You've to rebuild the image or totry to mount a volume airflow.cfg file


Other transformation in surface view

What is the typical way to display a sky?
Mapping longitude and latitude to x and y looks odd.

https://stackoverflow.com/questions/19999313/star-sphere-projecting-points-from-sphere-to-2d

Apparently we define a viewport that sits at some (configurable) distance from the viewer, and just map down star directions via intercept theorem.
This should look better and also be much more performant.

Closed with #29


buildvm-maint: don't fail if dir doesn't exist

@adammhaile, @jupierce
https://buildvm.openshift.eng.bos.redhat.com:8443/job/aos-cd-builds/job/build%252Fbuildvm-maint/149/console

@jupierce, the fastest merger in the West.


Fix MiddleWare Body Bug + Small improvements

Upgrade to latest listener pkg version in skr watcher
Fix remove Read Body in Middleware, otherwise Buffer would be empty, thus event cannot be unmarshaled. It is sufficient to just check size
Include more details in error logs

Unfortunately no unit testing here. Can you add to the unit test layer as soon as you have verified that everything works please? @jeremyharisch


Fix None column comment

*Issue #1017 *
Description of changes:
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

AWS CodeBuild CI Report

CodeBuild project: GitHubCodeBuild8756EF16-pDO66x4b9gEu
Commit ID: 73fcee4d4d2c4a6ea431e88476824cd750cd6dbc
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


tests: kernel: fpu_sharing: Tests in tests/kernel/fpu_sharing fail on nrf platforms

Describe the bug
Tests in tests/kernel/fpu_sharing fail on nrf platforms (nrf52dk_nrf52840 and nrf52840dk_nrf52840)
To Reproduce
Steps to reproduce the behavior:

go to zephyr dir
have either of the platforms connected
run ./scripts/sanitycheck --device-testing -p nrf52dk_nrf52832 -T tests/kernel/fpu_sharing/ --device-serial /dev/ttyACM0 -v -v
See error

Expected behavior
Tests passes
Impact
Not clear
Logs and console output
DEBUG   - DEVICE: *** Booting Zephyr OS build zephyr-v2.4.0-1584-g92ecc7866cbe  ***
DEBUG   - DEVICE: Running test suite E: ***** MPU FAULT *****
DEBUG   - DEVICE: E:   Stacking error (context area might be not valid)
DEBUG   - DEVICE: E:   Data Access Violation
DEBUG   - DEVICE: E:   MMFAR Address: 0x20002c18
DEBUG   - DEVICE: E: r0/a1:  0x00000000  r1/a2:  0x00000000  r2/a3:  0x00000000
DEBUG   - DEVICE: E: r3/a4:  0x00000000 r12/ip:  0x00000000 r14/lr:  0x00000000
DEBUG   - DEVICE: E:  xpsr:  0x61000000
DEBUG   - DEVICE: E: s[ 0]:  0x00000000  s[ 1]:  0xffffffff  s[ 2]:  0x00000000  s[ 3]:  0x00000000
DEBUG   - DEVICE: E: s[ 4]:  0x00000000  s[ 5]:  0x00000000  s[ 6]:  0x00000000  s[ 7]:  0x00000000
DEBUG   - DEVICE: E: s[ 8]:  0x00000000  s[ 9]:  0x00000000  s[10]:  0x00000000  s[11]:  0x00000000
DEBUG   - DEVICE: E: s[12]:  0x00000000  s[13]:  0x00000000  s[14]:  0x00000000  s[15]:  0x00000000
DEBUG   - DEVICE: E: fpscr:  0x00000013
DEBUG   - DEVICE: E: Faulting instruction address (r15/pc): 0x000049c8
DEBUG   - DEVICE: E: >>> ZEPHYR FATAL ERROR 2: Stack overflow on CPU 0
DEBUG   - DEVICE: E: Current thread: 0x200004a0 (main)
DEBUG   - DEVICE: E: Halting system
DEBUG   - Timed out while monitoring serial output on nrf52840dk_nrf52840
DEBUG   - run status: timeout
INFO    - 1/1 nrf52840dk_nrf52840       tests/kernel/fpu_sharing/float_disable/kernel.fpu_sharing.float_disable.arm FAILED Timeout (device 65.977s)

Environment (please complete the following information):

OS: Linux 18.4
Toolchain Zephyr SDK
Commit SHA or Version used: 92ecc7866c

Additional context
It seems that another bug (or need for enhancement) was caught here. The test ended with timeout instead of raising an error and terminating the test earlier

@nashif (copy from the issue): It seems that another bug (or need for enhancement) was caught here. The test ended with a timeout instead of raising an error and terminating the test earlier.
Should I raise a new issue for this?

@PerMac  not sure I understand what the issue is?

The issue I asking you about is the fact that the test failed causing system halt. However, sanitycheck had to wait for 65s and ended as
DEBUG   - Timed out while monitoring serial output on nrf52840dk_nrf52840
DEBUG   - run status: timeout

I wonder if this is expected behavior? I would assume that if there is some fatal error on the board the test should be terminated earlier, instead of waiting for the timeout. Hence the question: Is it a feature (and might be improved if needed) or is it a bug?

That's something for @pabigot to comment on :) Why do we need more stack now?

FYI, Tests passes on nrf9160dk_nrf9160 and nrf5340pdk_nrf5340_cpuapp

FYI, Tests passes on nrf9160dk_nrf9160 and nrf5340pdk_nrf5340_cpuapp

This makes sense - those boards have a different architecture.

That's something for @pabigot to comment on :) Why do we need more stack now?

CONFIG_CBPRINTF_FP_SUPPORT defaults to true when CONFIG_FPU=y is selected, which causes *print* to allocate a larger buffer on the stack for formatted numeric values.  If you set it off (the test doesn't format floating point numbers) the test passes.  Alternatively the test could select CBPRINTF_NANO=y, which would have a similar effect.
I'll file a PR that fixes this and another problem identified while diagnosing it.

Thanks @pabigot


Akka.CQRS Comparison

I am more curious than anything else. Feel free to close this issue if it doesn't create constructive conversation.
Have you had an opportunity to look at the Akka.CQRS reference architecture (https://github.com/petabridge/akkadotnet-cluster-workshop) that @Aaronontheweb and the Petabridge team recently created for their Akka.NET training tour? If so, how does the CQRS architecture facilitated by the Akkatecture framework compare (and contrast) to Petabridge's example?

I don't think that there is a need for a comparison. Akka.CQRS is a reference architecture that employs all the best practices for making these kind of systems. The language used in the reference architecture for me is more aligned with how you would describe an actor system
As for Akkatecture we try to take an approach that aligns with DDD / ES / CQRS concepts and we use that language. in other words, we use terms that other domain driven design practitioners are used to.
As a big ups to the Akka.CQRS project I think that it is pretty neat. Akka.NET is already a decent framework for building CQRS & event sourced solutions, all this project does is impose some honest  defaults that I find quite sane.
There is a lot of congruence between persistent actors and event sourced aggregates e.g that IMHO some folks don't see it as obvious as I do, which is why I made this project. I also made it because, as someone who has implemented a bunch of these systems, i see myself writing the same code over and over again to enable their business requirements. This project is a distillation of all of those things that I've picked up along the way.
My shortest answer to your question might be that a framework like Lagom is built on (jvm) akka just like how this project is build on dotnet akka. In some sense, Lagom is the the same animal on the JVM Akka as Akkatecture tries to be on dotnet.
If i could play a little bit of devils advocate on my own behalf, using this project to realize your goals is more restrictive with respect what you can do in the actor model but it does put you down on the right path to designing systems that conform to the principles of domain driven design. It is up to you or not if you want to be.
My personal opinion though, if you wanted to see how a business system built on akka should look like, go to the reference project. it ticks all the boxes.

@Lutando Thank you for taking the time to write a thorough response. I appreciate it.
Akkatecture seems like a valuable, opinionated framework for the Akka.NET community.

Issue is closed but anyone is welcome to reopen and discuss


Unit of ‘focal length’?

What is the unit of ‘focal length’? (mm or pixel)
and other pose estimations?

I mean in the TCO and K matrix outputs?

It should be in pixels in both cases.


UIDS-70 Add Input component

Closes #70
See in action here
I tried to mimic all the varieties found in this Figma

Minor tweak, but would you be able to change (Required) to the regular weight?


createSubWindow Function Windows Support

Are you planning to create multi window support for windows?

同问

The createSubWindow method is just an experimental method and will not support windows for the time being

You can try https://pub.dev/packages/desktop_multi_window


ERROR: pull access denied for lagoon/api-db,

Executing make lagoon-kickstart I get this error.
IMAGE_REPO=lagoon CI_USE_OPENSHIFT_REGISTRY=false docker-compose -p lagoon up -d rabbitmq openshiftremove openshiftbuilddeploy openshiftbuilddeploymonitor logs2slack api ssh auth-server local-git local-api-data-watcher-pusher local-es-kibana-watcher-pusher tests rest2tasks
Pulling api-db (lagoon/api-db:latest)...
ERROR: pull access denied for lagoon/api-db, repository does not exist or may require 'docker login'

The solution was add it at the end of deployment-test-services-main definition in Makefile

Hi @aldibier thanks for the bugreport. Can you create a pull request with that change. That would be fabulous!

+1
Pulling api-db (lagoon/api-db:)...
ERROR: pull access denied for lagoon/api-db, repository does not exist or may require 'docker login'

Would love to help here, but it's clearly not something I'm physically allowed to help with...

did you build the images locally first?

I didn't actually, but I don't know where I can find the dockerfile for it.
Perhaps you could tell me - I'm simply using the kickstart process.

make build will kick off the build process for all the images

Excellent - I'll try that and get back to you. Thank you.

you can also always get to us at https://amazeeio.rocket.chat

I'm already there :P
I did assume this issue was linked to the OP, so that was my mistake.


Add brief , default and html summary NUnit 3 transforms

Add brief and default transforms compatible with nunit v3 xml output reports. These try to match the Nunit3 console output as closely as possible.
This is the first 2 of the 4 transforms for #5
@CharliePoole / @rprouse - as this is my first PR into the nunit.org, I wasn't sure if there was any additional steps I need to take for this, so please feel free to point me in the right direction if there is!
The brief output provides a summary like this:
brief-v3.txt
Runtime Environment
   OS Version: Microsoft Windows NT 10.0.10586.0
  CLR Version: 4.0.30319.42000

NUnit Version: <IP_ADDRESS>

Test Run Summary
 Overall result: Failed
 Test Count: 14, Passed: 2, Failed: 6, Inconclusive: 2, Skipped: 4
   Failed Tests - Failures: 2, Errors: 2, Invalid: 2
   Skipped Tests - Ignored: 2, Explicit: 2, Other: 0
 Start time: 2016-09-08 07:40:49Z
   End time: 2016-09-08 07:40:49Z
   Duration: 0.146 seconds

whilst the Default provides a summary like so:
default-v3.txt
Runtime Environment
   OS Version: Microsoft Windows NT 10.0.10586.0
  CLR Version: 4.0.30319.42000

NUnit Version: <IP_ADDRESS>

Test Files
    C:\Users\mark.harrison\Documents\visual studio 2015\Projects\ClassLibrary1\bin\Debug\ClassLibrary1.dll
    C:\Users\mark.harrison\Documents\visual studio 2015\Projects\ClassLibrary1\bin\Debug\ClassLibrary1.dll

Tests Not Run

1) Explicit : ClassLibrary1.ExplicitTests.The_TEST
Some Explicit reason

2) Ignored : ClassLibrary1.NormalTests.IgnoreMe
Some Ignore reason

3) Explicit : ClassLibrary1.ExplicitTests.The_TEST
Some Explicit reason

4) Ignored : ClassLibrary1.NormalTests.IgnoreMe
Some Ignore reason

Errors and Failures

1) Invalid : ClassLibrary1.BadFixture.SomeTest
OneTimeSetUp: No suitable constructor was found

2) Error : ClassLibrary1.NormalTests.Error_Case
System.ApplicationException : error happened
   at ClassLibrary1.NormalTests.Error_Case() in C:\Users\mark.harrison\Documents\Visual Studio 2015\Projects\ClassLibrary1\Class1.cs:line 46

3) Failed : ClassLibrary1.NormalTests.Failure_Case
Intentional Failure
at ClassLibrary1.NormalTests.Failure_Case() in C:\Users\mark.harrison\Documents\Visual Studio 2015\Projects\ClassLibrary1\Class1.cs:line 40

4) Invalid : ClassLibrary1.BadFixture.SomeTest
OneTimeSetUp: No suitable constructor was found

5) Error : ClassLibrary1.NormalTests.Error_Case
System.ApplicationException : error happened
   at ClassLibrary1.NormalTests.Error_Case() in C:\Users\mark.harrison\Documents\Visual Studio 2015\Projects\ClassLibrary1\Class1.cs:line 46

6) Failed : ClassLibrary1.NormalTests.Failure_Case
Intentional Failure
at ClassLibrary1.NormalTests.Failure_Case() in C:\Users\mark.harrison\Documents\Visual Studio 2015\Projects\ClassLibrary1\Class1.cs:line 40

Run Settings
    DefaultTimeout: 900000
    WorkDirectory: D:\TestResults
    ImageRuntimeVersion: 4.0.30319
    ImageTargetFrameworkName: .NETFramework,Version=v4.5
    ImageRequiresX86: False
    ImageRequiresDefaultAppDomainAssemblyResolver: False
    NumberOfTestWorkers: 8

Test Run Summary
 Overall result: Failed
 Test Count: 14, Passed: 2, Failed: 6, Inconclusive: 2, Skipped: 4
   Failed Tests - Failures: 2, Errors: 2, Invalid: 2
   Skipped Tests - Ignored: 2, Explicit: 2, Other: 0
 Start time: 2016-09-08 07:40:49Z
   End time: 2016-09-08 07:40:49Z
   Duration: 0.146 seconds

Also, here is the nunit3 result output in case its useful for comparison
testing-nunit3.zip

@harrisonmeister thanks for the contribution. :+1:
Before I review, you asked about how to do an NUnit PR. What you submitted is fine, but I made some edits to make your next PR easier.

You don't need to use the title that GitHub suggests, especially if it ends up split. Edit it to make it short and concise so we can quickly understand it when looking at the PR list.
You referenced the issue, that is great, it creates cross links
If a PR fixes an issue, add Fixes #nnn to the PR so the issue will be automatically closed. Not needed here.
I copied the results inline to make it easier for reviewers. Downloading and opening files slows us down and we often do quick reviews on our mobile devices which is difficult with downloads.

Now, to the review... :smile:

Thanks for the feedback @rprouse - will keep it in mind for the next PR

The code changes look good to me. It would be nice to add unit tests soon before there are too many transforms to catch up on, but I am okay with finishing these four transforms first. I also need to download this project and familiarize myself with the code and features :smile:
I don't like that the brief summary does not show the errors, but I assume that it is functionally equivalent to the NUnit 2 version?
@CharliePoole can you do a second review.

@rprouse - correct, the v2 xslt didnt include details of tests which didnt run, but I can add that to the v3 ones (its essentially just the same info from the default v3 one) so let me know if you want that added too.

I also assume that detecting the transform to use based on the v2/v3 output will be done later? Looking at the code, I see that the transforms are hard coded.

I also wonder if we should rename the existing transforms to -v2 (and update the code) to make them consistent?

I made auto detection a separate issue because it requires a bit of reorganizing in the code. Currently the transform to use is based only on the options so it is constructed once before any input is read. It will have to be done now for each input file.
BTW, the list of issues I created serves as a sort of plan of attack.

@harrisonmeister One addition to @rprouse's advice for doing PRs...
Since you have write access to the repository, you should create the branches there. That gets you a build just by pushing the branch, even before you create a PR and also makes it easier for collaboration when it's needed. Our practice is still to use a PR and get another committer to approve the merge before doing it.
If you should end up working on any other nunit projects, like the framework or console runner, then you will be back to using your fork at least initially but because of your interest in the project I gave you write access to nunit-summary from the start.

I had some minor questions about format, but I think it will be better to do a final review after all eight transforms are in place and the autoselect bit is done.

Ah ok, thanks @CharliePoole for the review. I knew it was one of them :)


[9.x] Use match expression where possible

Laravel 9 will require PHP 8.0 as the minimum version, so we are free to use match expression. In this PR, I try to convert a few switch statements into match expressions that makes code easier to read.
Please review the code changes and see how nice the match expression is.

@seriquynh this seem to have broken the default null return in a couple of places: https://github.com/laravel/framework/pull/40880

@driesvints I'm sorry for missing your mention because I haven't check github for a while. I see a fixed PR has been merged. Thanks for you guys!


om release not working on osx

om release: 0.23.0
osx version: 10.12.5 (16F73)
ops manager: <IP_ADDRESS>
when making calls using the om cli, the calls just hang.
I've compiled locally using commit: 383848e17fc25e128c6657c9190c84bae287bc69 (current HEAD on master). This works fine, calls to ops manager complete successfully.
Not sure what the issue with the release binary is.

Hi @xchapter7x,
We've attempted to reproduce the issue on our machine which is running the same OSX version, om downloaded from https://github.com/pivotal-cf/om/releases/tag/0.23.0, and targeting the same ops manager however we did not run into the same issue.
$ ~/Downloads/om-darwin -t <target> -u <user> -p <password> -k curl -p "/api/v0/available_products"
Status: 200 OK
Cache-Control: max-age=0, private, must-revalidate
Connection: keep-alive
Content-Type: application/json; charset=utf-8
Date: Thu, 06 Jul 2017 21:34:49 GMT
Etag: W/"d751713988987e9331980363e24189ce"
Server: nginx/1.4.6 (Ubuntu)
Strict-Transport-Security: max-age=31536000
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Request-Id: 4f730367-212b-4fe7-8c95-dee83fa91815
X-Runtime: 0.124931
X-Xss-Protection: 1; mode=block
[]

Can you confirm that you downloaded the om darwin binary?
Thanks,
@kkallday & @joshzarrabi

just looked back in my history:
$ wget https://github.com/pivotal-cf/om/releases/download/0.23.0/om-darwin

the version and help commands worked, just not any actual calls to ops man using the downloaded binary.
i just re-wget'd it and all is good. really strange, probably error on my part somewhere.
anyways, ill close this out for now.
Thanks.

@xchapter7x Okay cool, thanks for submitting the issue and let us know if you see any other problems.

Thanks for looking into this @joshzarrabi and @kkallday!


Allow a decimal rating

Hello,
is it possible to rate an item with decimal precision?
A new configuration proprerty, something like "step" or similiar will be awesome, e.g.
step: 0.1 // 0.1 deciamal precision
step: 0.5 // half int precision
Apologize for my English :)

Hi,
Thanks for your input.
We currently don't support this, but since you have an interest in this I think it will not be so hard to implement. I will take a look at this.

This is fixed in latest version


Cosmetic build adjustments

among other things, the governance task will run only on 'msft-' prefixed branches to eliminated governance data noise generated for the development branches should developers decide to run the build manually.

I'm still part of this repository.. just got an email about the PR. Please remove my access! Thanks.

@erictraut see request from @matinehshaker above. Thanks.


Add hover on text

Is your feature request related to a problem? Please describe.
Currently, when viewing a example project there's a option to fork and embed project but there is no information about what does it means.
Describe the solution you'd like
Add hover on text for fork and embed. If no so a short description can be added below and a detailed information about this what does it means can be added to the documentation.
Additional context

Are you working on this? (Yes/No)
No, this task is for GCI students.

I guess you are talking about tooltip feature. You can change the issue heading for better understandability.

@aayush-05 Thanks for the review. Sure, I will do that.


Skipping full serialization of Keras layer because it is not built.

i trained SSD-MobileNet on custom dataset and now I want to convert the model to TensorRT model for improve performance and for Inference on Nvidia Jetson TX1 Board.
I Have:
-Ubuntu 20.14 LTS
-Tensorflow 2.11.0
-Python 3.8.10
when I run the:
import tensorflow as tf
tf.saved_model.save(model_dir , out_dir)

I wil get this error:
WARNING:tensorflow:Skipping full serialization of Keras layer <object_detection.meta_architectures.ssd_meta_arch.SSDMetaArch object at 0x7f45581d7340>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <object_detection.predictors.convolutional_keras_box_predictor.WeightSharedConvolutionalBoxPredictor object at 0x7f4558265760>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <object_detection.models.ssd_mobilenet_v2_fpn_keras_feature_extractor.SSDMobileNetV2FpnKerasFeatureExtractor object at 0x7f4561e31130>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <object_detection.predictors.heads.keras_box_head.WeightSharedConvolutionalBoxHead object at 0x7f45582562b0>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <object_detection.predictors.heads.keras_class_head.WeightSharedConvolutionalClassHead object at 0x7f4558256ca0>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <keras.layers.convolutional.separable_conv2d.SeparableConv2D object at 0x7f4558256460>, because it is not built.
WARNING:tensorflow:Skipping full serialization of Keras layer <keras.layers.convolutional.separable_conv2d.SeparableConv2D object at 0x7f45582650d0>, because it is not built.
WARNING:absl:Found untraced functions such as _jit_compiled_convolution_op, _jit_compiled_convolution_op while saving (showing 2 of 2). These functions will not be directly callable after loading.

how can I fix that ?

Hi @SJavad, I have tried to load the ssd_mobilenet and saved it (without training) and i did not get any warnings. please refer to this (gist)[https://gist.github.com/kiransair/08cb954b8be930f630f26c228dd573d9]. Also could you please try with the latest version of tensorflow. Thank You.


This is giga chad level

This is giga chad level

https://www.youtube.com/watch?v=8B5InWGdims


Harvester unit

For #139

Coverage decreased (-0.02%) to 83.886% when pulling 0d53b01687285c500f0741ed728e17165152ae0b on feature/harvester-unit into c7f958ea66728e8c4a094c4d1120e9f17a142254 on master.

Coverage decreased (-0.2%) to 83.673% when pulling 3d18f6cab7aac9df6ae5a555a3fac16b08d9a8cc on feature/harvester-unit into c7f958ea66728e8c4a094c4d1120e9f17a142254 on master.

Coverage increased (+0.4%) to 84.296% when pulling 3f39623a85bcbc646637914e433afbc775c12eba on feature/harvester-unit into c7f958ea66728e8c4a094c4d1120e9f17a142254 on master.

Coverage decreased (-0.2%) to 83.673% when pulling e23f2a86f5c0afa92afa202d77c76a88eb22ad4e on feature/harvester-unit into c7f958ea66728e8c4a094c4d1120e9f17a142254 on master.

Coverage decreased (-0.2%) to 83.673% when pulling f6cb7f3d6d4324f5facb6daeb53c30a456efef49 on feature/harvester-unit into c7f958ea66728e8c4a094c4d1120e9f17a142254 on master.


Remove unneeded asmdefs to avoid errors

The asmdefs should be added later, only when actual content is in there. otherwise there's Unity warnings about empty asmdefs.

I agree that we need to get rid of the warnings. This PR results in empty directories and we need to remove the .meta files for them (due to git not following directories).
I'll amend this PR later this week and make sure there are no dangling files. See https://github.com/Halodi/halodi-unity-package-creator/issues/12 for tracking this issue.
The asmdefs are auto-generated by our internal tooling (halodi-unity-package-creator) and setup the cross-references between the assemblies correctly (which can be a bit annoying).

Oops sorry. I removed them.

I fixed the issue in this commit: https://github.com/Halodi/halodi-unity-package-registry-manager/commit/8acaab30e6e8d2a5340725c9ab1cda46c8da6008
I removed the runtime folders and added an AssemblyInfo.cs file in the Tests folder. I want to keep the Tests folder around to shame myself in one day making tests.


[BUG] memory during fine tuning grows until MacBook crashes

Describe the bug
The required memory seems to increase continuously during fine-tuning. I train 5 epochs. Shortly after starting the 2nd epoch, my MacBook crashes because there is no more memory available. My data set contains 2058 entries and I train 10290 iterations. The peak memory increases in the meantime. With a similar configuration, I was able to perform fine-tuning a month ago without crashing. I terminate all programms before starting to train. I have a MacBook Pro with 128GB memory and M3 Max processor.
dartgen % mlx_lm.lora --config "fine_tuning/mlx/llama3_1_8B_instruct_lora_config.yaml"
Loading configuration file fine_tuning/mlx/llama3_1_8B_instruct_lora_config.yaml
Loading pretrained model
/Users/admin/.cache/huggingface/hub/models--meta-llama--Meta-Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659
Loading datasets
Training
Trainable parameters: 1.045% (83.886M/8030.261M)
Starting training..., iters: 10290
Iter 1: Val loss 0.655, Val took 87.790s
[WARNING] Some sequences are longer than 8192 tokens. The longest sentence 19487 will be truncated to 8192. Consider pre-splitting your data to save memory.
Iter 10: Train loss 0.784, Learning Rate 2.913e-07, It/sec 0.037, Tokens/sec 81.695, Trained Tokens 22075, Peak mem 121.550 GB
Iter 20: Train loss 0.841, Learning Rate 6.149e-07, It/sec 0.084, Tokens/sec 115.019, Trained Tokens 35799, Peak mem 121.550 GB
Iter 30: Train loss 0.812, Learning Rate 9.385e-07, It/sec 0.097, Tokens/sec 115.662, Trained Tokens 47775, Peak mem 121.550 GB
Iter 40: Train loss 0.728, Learning Rate 1.262e-06, It/sec 0.075, Tokens/sec 113.291, Trained Tokens 62897, Peak mem 121.550 GB
Iter 50: Train loss 0.629, Learning Rate 1.586e-06, It/sec 0.092, Tokens/sec 122.670, Trained Tokens 76177, Peak mem 121.550 GB
Iter 60: Train loss 0.604, Learning Rate 1.909e-06, It/sec 0.071, Tokens/sec 119.333, Trained Tokens 93038, Peak mem 121.550 GB
Iter 70: Train loss 0.544, Learning Rate 2.233e-06, It/sec 0.072, Tokens/sec 126.830, Trained Tokens 110709, Peak mem 121.550 GB
Iter 80: Train loss 0.583, Learning Rate 2.557e-06, It/sec 0.120, Tokens/sec 129.292, Trained Tokens 121482, Peak mem 121.550 GB
Iter 90: Train loss 0.617, Learning Rate 2.880e-06, It/sec 0.107, Tokens/sec 129.724, Trained Tokens 133588, Peak mem 121.550 GB
Iter 100: Train loss 0.574, Learning Rate 3.204e-06, It/sec 0.128, Tokens/sec 129.965, Trained Tokens 143762, Peak mem 121.550 GB
...
 Peak mem 236.340 GB
Iter 2300: Saved adapter weights to results/llama3_1_8B_instruct_lora/tuning_15/adapters/adapters.safetensors and results/llama3_1_8B_instruct_lora/tuning_15/adapters/0002300_adapters.safetensors.
Iter 2310: Train loss 0.235, Learning Rate 9.042e-06, It/sec 0.050, Tokens/sec 83.557, Trained Tokens 3243107, Peak mem 236.340 GB
Iter 2320: Train loss 0.205, Learning Rate 9.033e-06, It/sec 0.075, Tokens/sec 81.691, Trained Tokens 3254004, Peak mem 236.340 GB
Iter 2330: Train loss 0.248, Learning Rate 9.024e-06, It/sec 0.062, Tokens/sec 82.444, Trained Tokens 3267278, Peak mem 236.340 GB
Iter 2340: Train loss 0.218, Learning Rate 9.015e-06, It/sec 0.065, Tokens/sec 82.367, Trained Tokens 3279931, Peak mem 236.340 GB
Iter 2350: Train loss 0.196, Learning Rate 9.005e-06, It/sec 0.065, Tokens/sec 76.131, Trained Tokens 3291607, Peak mem 236.340 GB
Iter 2360: Train loss 0.230, Learning Rate 8.996e-06, It/sec 0.052, Tokens/sec 81.457, Trained Tokens 3307229, Peak mem 236.340 GB
Iter 2370: Train loss 0.265, Learning Rate 8.986e-06, It/sec 0.058, Tokens/sec 82.938, Trained Tokens 3321615, Peak mem 236.340 GB

  [30.10.2024, 21:51:48 Wiederhergestellt]
Last login: Wed Oct 30 21:51:45 on console

To Reproduce
`mlx_lm.lora --config "fine_tuning/mlx/llama3_1_8B_instruct_lora_config.yaml"``
model: "meta-llama/Meta-Llama-3.1-8B-Instruct"
train: true
# The fine-tuning method: "lora", "dora", or "full".
fine_tune_type: lora
data: "fine_tuning/data/rev2"
seed: 0
num_layers: 32
batch_size: 1
iters: 10290
val_batches: 25
learning_rate: 1e-5
steps_per_report: 10
steps_per_eval: 200
resume_adapter_file: null
adapter_path: "results/llama3_1_8B_instruct_lora/tuning_15/adapters"
save_every: 100
test: true
test_batches: -1
max_seq_length: 8192
grad_checkpoint: true
lora_parameters:
  keys: [
    "self_attn.q_proj",
    "self_attn.v_proj",
    "self_attn.k_proj",
    "self_attn.o_proj",
    "mlp.gate_proj",
    "mlp.down_proj",
    "mlp.up_proj"
  ]
  scale: 1
  rank: 32
  dropout: 0.1
lr_schedule:
  name: cosine_decay
  warmup: 309 # 0 for no warmup
  warmup_init: 0
  arguments: [
    1e-5,
    9981,
    0
  ]

Expected behavior
That the memory consumption remains the same and there is no crash.
Desktop (please complete the following information):

OS Version: [MacOS 15.0.1]
MLX Version [0.19.1]
MLX-LM Version [0.19.2]

Additional context
Log from a month ago when it still worked with a similar configuration:
(dartgen) admin@MBPvonH dartgen % date
Fr 20 Sep 2024 17:08:10 CEST
(dartgen) admin@MBPvonH dartgen % mlx_lm.lora --config "fine_tuning/lora_config.yaml"
Loading configuration file fine_tuning/lora_config.yaml
Loading pretrained model
Fetching 11 files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 11/11 [00:00<00:00, 159094.29it/s]
Loading datasets
Training
Trainable parameters: 1.045% (83.886M/8030.261M)
Starting training..., iters: 10290
Iter 1: Val loss 0.698, Val took 88.051s
[WARNING] Some sequences are longer than 8192 tokens. The longest sentence 19453 will be truncated to 8192. Consider pre-splitting your data to save memory.
Iter 10: Train loss 0.711, Learning Rate 1.456e-06, It/sec 0.051, Tokens/sec 113.748, Trained Tokens 22358, Peak mem 121.504 GB
Iter 20: Train loss 0.691, Learning Rate 3.074e-06, It/sec 0.097, Tokens/sec 131.496, Trained Tokens 35934, Peak mem 121.504 GB
...
Iter 10220: Train loss 0.029, Learning Rate 6.419e-09, It/sec 0.142, Tokens/sec 136.709, Trained Tokens 14102462, Peak mem 121.504 GB
Iter 10230: Train loss 0.033, Learning Rate 4.758e-09, It/sec 0.127, Tokens/sec 136.872, Trained Tokens 14113259, Peak mem 121.504 GB
Iter 10240: Train loss 0.046, Learning Rate 3.347e-09, It/sec 0.058, Tokens/sec 130.574, Trained Tokens 14135730, Peak mem 121.504 GB
Iter 10250: Train loss 0.035, Learning Rate 2.185e-09, It/sec 0.086, Tokens/sec 135.763, Trained Tokens 14151482, Peak mem 121.504 GB
Iter 10260: Train loss 0.041, Learning Rate 1.268e-09, It/sec 0.086, Tokens/sec 137.544, Trained Tokens 14167447, Peak mem 121.504 GB
Iter 10270: Train loss 0.041, Learning Rate 5.975e-10, It/sec 0.046, Tokens/sec 96.664, Trained Tokens 14188391, Peak mem 121.504 GB
Iter 10280: Train loss 0.038, Learning Rate 1.803e-10, It/sec 0.100, Tokens/sec 133.021, Trained Tokens 14201681, Peak mem 121.504 GB
[WARNING] Some sequences are longer than 8192 tokens. The longest sentence 11907 will be truncated to 8192. Consider pre-splitting your data to save memory.
Iter 10290: Val loss 0.317, Val took 123.558s
Iter 10290: Train loss 0.031, Learning Rate 4.470e-12, It/sec 0.704, Tokens/sec 1178.573, Trained Tokens 14218430, Peak mem 121.504 GB
Saved final adapter weights to adapters/adapters.safetensors.
Testing
[WARNING] Some sequences are longer than 8192 tokens. The longest sentence 10044 will be truncated to 8192. Consider pre-splitting your data to save memory.
Test loss 0.243, Test ppl 1.275.

The configuration a month ago.
model: "meta-llama/Meta-Llama-3.1-8B-Instruct"
train: true
data: "fine_tuning/"
seed: 0
lora_layers: 32
batch_size: 1
iters: 10290
val_batches: 25
learning_rate: 5e-5
steps_per_report: 10
steps_per_eval: 200
resume_adapter_file: null
adapter_path: "adapters"
save_every: 100
test: true
test_batches: -1
max_seq_length: 8192
grad_checkpoint: true
use_dora: false
lora_parameters:
  keys: [
    "self_attn.q_proj",
    "self_attn.v_proj",
    "self_attn.k_proj",
    "self_attn.o_proj",
    "mlp.gate_proj",
    "mlp.down_proj",
    "mlp.up_proj"
  ]
  scale: 4
  rank: 32
  dropout: 0.05
lr_schedule:
  name: cosine_decay
  warmup: 309 # 0 for no warmup
  warmup_init: 0
  arguments: [
    5e-5,
    9981,
    0
  ]

Describe the bug The required memory seems to increase continuously during fine-tuning. I train 5 epochs. Shortly after starting the 2nd epoch, my MacBook crashes because there is no more memory available. My data set contains 2058 entries and I train 10290 iterations. The peak memory increases in the meantime. With a similar configuration (and older MLX-Version), I was able to perform fine-tuning a month ago without crashing. I terminate all programms before starting to train.
dartgen % mlx_lm.lora --config "fine_tuning/mlx/llama3_1_8B_instruct_lora_config.yaml"
Loading configuration file fine_tuning/mlx/llama3_1_8B_instruct_lora_config.yaml
Loading pretrained model
/Users/admin/.cache/huggingface/hub/models--meta-llama--Meta-Llama-3.1-8B-Instruct/snapshots/0e9e39f249a16976918f6564b8830bc894c89659
Loading datasets
Training
Trainable parameters: 1.045% (83.886M/8030.261M)
Starting training..., iters: 10290
Iter 1: Val loss 0.655, Val took 87.790s
[WARNING] Some sequences are longer than 8192 tokens. The longest sentence 19487 will be truncated to 8192. Consider pre-splitting your data to save memory.
Iter 10: Train loss 0.784, Learning Rate 2.913e-07, It/sec 0.037, Tokens/sec 81.695, Trained Tokens 22075, Peak mem 121.550 GB
Iter 20: Train loss 0.841, Learning Rate 6.149e-07, It/sec 0.084, Tokens/sec 115.019, Trained Tokens 35799, Peak mem 121.550 GB
Iter 30: Train loss 0.812, Learning Rate 9.385e-07, It/sec 0.097, Tokens/sec 115.662, Trained Tokens 47775, Peak mem 121.550 GB
Iter 40: Train loss 0.728, Learning Rate 1.262e-06, It/sec 0.075, Tokens/sec 113.291, Trained Tokens 62897, Peak mem 121.550 GB
Iter 50: Train loss 0.629, Learning Rate 1.586e-06, It/sec 0.092, Tokens/sec 122.670, Trained Tokens 76177, Peak mem 121.550 GB
Iter 60: Train loss 0.604, Learning Rate 1.909e-06, It/sec 0.071, Tokens/sec 119.333, Trained Tokens 93038, Peak mem 121.550 GB
Iter 70: Train loss 0.544, Learning Rate 2.233e-06, It/sec 0.072, Tokens/sec 126.830, Trained Tokens 110709, Peak mem 121.550 GB
Iter 80: Train loss 0.583, Learning Rate 2.557e-06, It/sec 0.120, Tokens/sec 129.292, Trained Tokens 121482, Peak mem 121.550 GB
Iter 90: Train loss 0.617, Learning Rate 2.880e-06, It/sec 0.107, Tokens/sec 129.724, Trained Tokens 133588, Peak mem 121.550 GB
Iter 100: Train loss 0.574, Learning Rate 3.204e-06, It/sec 0.128, Tokens/sec 129.965, Trained Tokens 143762, Peak mem 121.550 GB
...
 Peak mem 236.340 GB
Iter 2300: Saved adapter weights to results/llama3_1_8B_instruct_lora/tuning_15/adapters/adapters.safetensors and results/llama3_1_8B_instruct_lora/tuning_15/adapters/0002300_adapters.safetensors.
Iter 2310: Train loss 0.235, Learning Rate 9.042e-06, It/sec 0.050, Tokens/sec 83.557, Trained Tokens 3243107, Peak mem 236.340 GB
Iter 2320: Train loss 0.205, Learning Rate 9.033e-06, It/sec 0.075, Tokens/sec 81.691, Trained Tokens 3254004, Peak mem 236.340 GB
Iter 2330: Train loss 0.248, Learning Rate 9.024e-06, It/sec 0.062, Tokens/sec 82.444, Trained Tokens 3267278, Peak mem 236.340 GB
Iter 2340: Train loss 0.218, Learning Rate 9.015e-06, It/sec 0.065, Tokens/sec 82.367, Trained Tokens 3279931, Peak mem 236.340 GB
Iter 2350: Train loss 0.196, Learning Rate 9.005e-06, It/sec 0.065, Tokens/sec 76.131, Trained Tokens 3291607, Peak mem 236.340 GB
Iter 2360: Train loss 0.230, Learning Rate 8.996e-06, It/sec 0.052, Tokens/sec 81.457, Trained Tokens 3307229, Peak mem 236.340 GB
Iter 2370: Train loss 0.265, Learning Rate 8.986e-06, It/sec 0.058, Tokens/sec 82.938, Trained Tokens 3321615, Peak mem 236.340 GB

  [30.10.2024, 21:51:48 Wiederhergestellt]
Last login: Wed Oct 30 21:51:45 on console

To Reproduce
`mlx_lm.lora --config "fine_tuning/mlx/llama3_1_8B_instruct_lora_config.yaml"``
model: "meta-llama/Meta-Llama-3.1-8B-Instruct"
train: true
# The fine-tuning method: "lora", "dora", or "full".
fine_tune_type: lora
data: "fine_tuning/data/rev2"
seed: 0
num_layers: 32
batch_size: 1
iters: 10290
val_batches: 25
learning_rate: 1e-5
steps_per_report: 10
steps_per_eval: 200
resume_adapter_file: null
adapter_path: "results/llama3_1_8B_instruct_lora/tuning_15/adapters"
save_every: 100
test: true
test_batches: -1
max_seq_length: 8192
grad_checkpoint: true
lora_parameters:
  keys: [
    "self_attn.q_proj",
    "self_attn.v_proj",
    "self_attn.k_proj",
    "self_attn.o_proj",
    "mlp.gate_proj",
    "mlp.down_proj",
    "mlp.up_proj"
  ]
  scale: 1
  rank: 32
  dropout: 0.1
lr_schedule:
  name: cosine_decay
  warmup: 309 # 0 for no warmup
  warmup_init: 0
  arguments: [
    1e-5,
    9981,
    0
  ]

Expected behavior That the memory consumption remains the same and there is no crash.
Desktop (please complete the following information):

OS Version: [MacOS 15.0.1]
MLX Version [0.19.1]
MLX-LM Version [0.19.2]

Additional context
Log from a month ago when it still worked with a similar configuration:
(dartgen) admin@MBPvonHolger2 dartgen % date
Fr 20 Sep 2024 17:08:10 CEST
(dartgen) admin@MBPvonHolger2 dartgen % mlx_lm.lora --config "fine_tuning/lora_config.yaml"
Loading configuration file fine_tuning/lora_config.yaml
Loading pretrained model
Fetching 11 files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 11/11 [00:00<00:00, 159094.29it/s]
Loading datasets
Training
Trainable parameters: 1.045% (83.886M/8030.261M)
Starting training..., iters: 10290
Iter 1: Val loss 0.698, Val took 88.051s
[WARNING] Some sequences are longer than 8192 tokens. The longest sentence 19453 will be truncated to 8192. Consider pre-splitting your data to save memory.
Iter 10: Train loss 0.711, Learning Rate 1.456e-06, It/sec 0.051, Tokens/sec 113.748, Trained Tokens 22358, Peak mem 121.504 GB
Iter 20: Train loss 0.691, Learning Rate 3.074e-06, It/sec 0.097, Tokens/sec 131.496, Trained Tokens 35934, Peak mem 121.504 GB
...
Iter 10220: Train loss 0.029, Learning Rate 6.419e-09, It/sec 0.142, Tokens/sec 136.709, Trained Tokens 14102462, Peak mem 121.504 GB
Iter 10230: Train loss 0.033, Learning Rate 4.758e-09, It/sec 0.127, Tokens/sec 136.872, Trained Tokens 14113259, Peak mem 121.504 GB
Iter 10240: Train loss 0.046, Learning Rate 3.347e-09, It/sec 0.058, Tokens/sec 130.574, Trained Tokens 14135730, Peak mem 121.504 GB
Iter 10250: Train loss 0.035, Learning Rate 2.185e-09, It/sec 0.086, Tokens/sec 135.763, Trained Tokens 14151482, Peak mem 121.504 GB
Iter 10260: Train loss 0.041, Learning Rate 1.268e-09, It/sec 0.086, Tokens/sec 137.544, Trained Tokens 14167447, Peak mem 121.504 GB
Iter 10270: Train loss 0.041, Learning Rate 5.975e-10, It/sec 0.046, Tokens/sec 96.664, Trained Tokens 14188391, Peak mem 121.504 GB
Iter 10280: Train loss 0.038, Learning Rate 1.803e-10, It/sec 0.100, Tokens/sec 133.021, Trained Tokens 14201681, Peak mem 121.504 GB
[WARNING] Some sequences are longer than 8192 tokens. The longest sentence 11907 will be truncated to 8192. Consider pre-splitting your data to save memory.
Iter 10290: Val loss 0.317, Val took 123.558s
Iter 10290: Train loss 0.031, Learning Rate 4.470e-12, It/sec 0.704, Tokens/sec 1178.573, Trained Tokens 14218430, Peak mem 121.504 GB
Saved final adapter weights to adapters/adapters.safetensors.
Testing
[WARNING] Some sequences are longer than 8192 tokens. The longest sentence 10044 will be truncated to 8192. Consider pre-splitting your data to save memory.
Test loss 0.243, Test ppl 1.275.

The configuration a month ago.
model: "meta-llama/Meta-Llama-3.1-8B-Instruct"
train: true
data: "fine_tuning/"
seed: 0
lora_layers: 32
batch_size: 1
iters: 10290
val_batches: 25
learning_rate: 5e-5
steps_per_report: 10
steps_per_eval: 200
resume_adapter_file: null
adapter_path: "adapters"
save_every: 100
test: true
test_batches: -1
max_seq_length: 8192
grad_checkpoint: true
use_dora: false
lora_parameters:
  keys: [
    "self_attn.q_proj",
    "self_attn.v_proj",
    "self_attn.k_proj",
    "self_attn.o_proj",
    "mlp.gate_proj",
    "mlp.down_proj",
    "mlp.up_proj"
  ]
  scale: 4
  rank: 32
  dropout: 0.05
lr_schedule:
  name: cosine_decay
  warmup: 309 # 0 for no warmup
  warmup_init: 0
  arguments: [
    5e-5,
    9981,
    0
  ]

I am currently testing it with a downgrade of the MLX version to the version of 20 Sep. The current peak mem seems to be stable at 121,504 GB.  But I'm only at Iter 710, so I'll report more tomorrow.
mlx = "0.17.3"
mlx-lm = "0.18.2"
...
transformers = "==4.44.2"

With MLX 0.17.3 and Mlx-lm 0.18.2 there is no crash. Peak Memory is stable.
Iter 2700: Train loss 0.211, Learning Rate 7.850e-06, It/sec 0.084, Tokens/sec 134.516, Trained Tokens 3806066, Peak mem 121.504 GB
Iter 2700: Saved adapter weights to results/llama3_1_8B_instruct_lora/tuning_16/adapters/adapters.safetensors and results/llama3_1_8B_instruct_lora/tuning_16/adapters/0002700_adapters.safetensors.
Iter 2710: Train loss 0.270, Learning Rate 7.834e-06, It/sec 0.116, Tokens/sec 134.561, Trained Tokens 3817627, Peak mem 121.504 GB
Iter 2720: Train loss 0.206, Learning Rate 7.817e-06, It/sec 0.089, Tokens/sec 134.335, Trained Tokens 3832672, Peak mem 121.504 GB
Iter 2730: Train loss 0.320, Learning Rate 7.801e-06, It/sec 0.164, Tokens/sec 136.103, Trained Tokens 3840970, Peak mem 121.504 GB
Iter 2740: Train loss 0.219, Learning Rate 7.785e-06, It/sec 0.089, Tokens/sec 134.494, Trained Tokens 3856049, Peak mem 121.504 GB
Iter 2750: Train loss 0.193, Learning Rate 7.769e-06, It/sec 0.120, Tokens/sec 136.323, Trained Tokens 3867396, Peak mem 121.504 GB
Iter 2760: Train loss 0.244, Learning Rate 7.752e-06, It/sec 0.104, Tokens/sec 136.876, Trained Tokens 3880495, Peak mem 121.504 GB

Thanks for flagging. I did confirm there is a leak with gradient checkpointing enabled. Looking into it.

Thanks for flagging. I did confirm there is a leak with gradient checkpointing enabled. Looking into it.

Many thanks for the quick fix. :)


ArgumentNullException in LocalTransport.InitStreams when using MIMode = gdb

When attempting to attach to a process on macOS using "MIMode": "gdb", the following error is thrown:

Steps to reproduce in VScode:

Create a project and use the "(lldb) Attach" template
Change MIMode to gdb
Attach to a program.

{
  // Use IntelliSense to learn about possible attributes.
  // Hover to view descriptions of existing attributes.
  // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
  "version": "0.2.0",
  "configurations": [
    {
      "name": "(lldb) Attach",
      "type": "cppdbg",
      "request": "attach",
      "program": "${workspaceRoot}/build-debug/bin/clangd",
      "processId": "${command:pickProcess}",
      "MIMode": "gdb",
      "logging": { "engineLogging": true }
    }

  ]
}

I have reproduced this in both VSCode and Vimspector (another Debug Adapter Protocol client).
In the vimspector case, we're certainly sending 'cwd' key:
2019-08-10 14:53:19,915 - DEBUG - Sending Message: {"command": "attach", "arguments": {"type": "cppdbg", "request": "attach", "program": "/Users/ben/Development/llvm-project/clang-tools-extra/clangd/../../build-debug/bin/clangd", "cwd": "/Users/ben/Development/llvm-project/clang-tools-extra/clangd", "args": [], "env": {"LD_LIBRARY_PATH": "/Users/ben/Development/llvm-project/clang-tools-extra/clangd/../../build-debug/lib"}, "miMode": "gdb", "stopAtEntry": false, "logging": {"engineLogging": true}, "processId": "62648", "name": "test"}, "seq": 1, "type": "request"}

Full error including the DAP log in this gist: https://gist.github.com/puremourning/35e2340ed364576b7721e3885a923445
Seems to have been reported before: https://github.com/microsoft/vscode-cpptools/issues/548#issuecomment-324781355

FWIW this is the version of vscode-cpptools that's installed in viscose:
Name: C/C++
Id: ms-vscode.cpptools
Description: C/C++ IntelliSense, debugging, and code browsing.
Version: 0.25.0-insiders
Publisher: Microsoft
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=ms-vscode.cpptools

@puremourning The problem is that if you specify "MIMode": "gdb" you need to specify "miDebuggerPath" to your location of gdb. I'll need to debug it and figure out what changed to cause it to throw the exception instead.

The actual issue of this was because cwd is also empty and was fixed in #893. It will be in the next release.

Should be fixed with 0.25.0. @pieandcakes Can you close this?


Running a retidy test app: TypeError: not a webpackBootstrap function call AST

Hello,
I'm trying to debundle a very short react file using retidy.
This is my test.js file:
test.js
import retidy from "retidy"
import fs from "fs"
const code = fs.readFileSync("vendor-bundle.js", "utf-8")
retidy.retidy(code, { type: "webpack-jsonp", outDir: "./out/", bundleAstReferenceKeys: ["body", 0, "expression"] })

This is the file to be unbundled:
vendor-bundle.js
!function(e){function n(t){if(r[t])return r[t].exports;var o=r[t]={exports:{},id:t,loaded:!1};return e[t].call(o.exports,o,o.exports,n),o.loaded=!0,o.exports}var t=window.webpackJsonp;window.webpackJsonp=function(a,l){for(var p,c,s=0,i=[];s<a.length;s++)c=a[s],o[c]&&i.push.apply(i,o[c]),o[c]=0;for(p in l)e[p]=l[p];for(t&&t(a,l);i.length;)i.shift().call(null,n);return l[0]?(r[0]=0,n(0)):void 0};var r={},o={4:0,6:0};n.e=function(e,t){if(0===o[e])return t.call(null,n);if(void 0!==o[e])o[e].push(t);else{o[e]=[t];var r=document.getElementsByTagName("head")[0],a=document.createElement("script");a.type="text/javascript",a.charset="utf-8",a.async=!0,a.src=n.p+""+e+"."+({5:"bundle",6:"common-bundle"}[e]||e)+".js",r.appendChild(a)}},n.m=e,n.c=r,n.p=""}([]);
This is my package.json:
package.json
{
  "name": "testapp",
  "version": "1.0.0",
  "type": "module",
  "description": "",
  "main": "test.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "retidy": "^1.5.1"
  }
}

And I have a common-bundle.js file:
common-bundle.js
webpackJsonp([6,4],[]);
This is the error i get running node --trace-warnings test.js:
(node:9928) UnhandledPromiseRejectionWarning: TypeError: not a webpackBootstrap function call AST.
    at Object.exports.notWebpackBootstrapAstErr (C:\Users\Roberto\Desktop\retidy\node_modules\retidy\dist\extractor\webpack\extract-modules.js:17:58)
    at Object.exports.extractModules (C:\Users\Roberto\Desktop\retidy\node_modules\retidy\dist\extractor\webpack\extract-modules.js:41:23)
    at exports.webpackExtractor (C:\Users\Roberto\Desktop\retidy\node_modules\retidy\dist\extractor\webpack\webpack-extractor.js:6:43)
    at Object.exports.extractor (C:\Users\Roberto\Desktop\retidy\node_modules\retidy\dist\extractor\extractor.js:20:12)
    at C:\Users\Roberto\Desktop\retidy\node_modules\retidy\dist\index.js:124:46
    at step (C:\Users\Roberto\Desktop\retidy\node_modules\retidy\dist\index.js:33:23)
    at Object.next (C:\Users\Roberto\Desktop\retidy\node_modules\retidy\dist\index.js:14:53)
    at fulfilled (C:\Users\Roberto\Desktop\retidy\node_modules\retidy\dist\index.js:5:58)
    at emitUnhandledRejectionWarning (internal/process/promises.js:168:15)
    at processPromiseRejections (internal/process/promises.js:247:11)
    at processTicksAndRejections (internal/process/task_queues.js:94:32)
(node:9928) TypeError: not a webpackBootstrap function call AST.
    at Object.exports.notWebpackBootstrapAstErr (C:\Users\Roberto\Desktop\retidy\node_modules\retidy\dist\extractor\webpack\extract-modules.js:17:58)
    at Object.exports.extractModules (C:\Users\Roberto\Desktop\retidy\node_modules\retidy\dist\extractor\webpack\extract-modules.js:41:23)
    at exports.webpackExtractor (C:\Users\Roberto\Desktop\retidy\node_modules\retidy\dist\extractor\webpack\webpack-extractor.js:6:43)
    at Object.exports.extractor (C:\Users\Roberto\Desktop\retidy\node_modules\retidy\dist\extractor\extractor.js:20:12)
    at C:\Users\Roberto\Desktop\retidy\node_modules\retidy\dist\index.js:124:46
    at step (C:\Users\Roberto\Desktop\retidy\node_modules\retidy\dist\index.js:33:23)
    at Object.next (C:\Users\Roberto\Desktop\retidy\node_modules\retidy\dist\index.js:14:53)
    at fulfilled (C:\Users\Roberto\Desktop\retidy\node_modules\retidy\dist\index.js:5:58)
(node:9928) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
    at emitDeprecationWarning (internal/process/promises.js:180:11)
    at processPromiseRejections (internal/process/promises.js:249:13)
    at processTicksAndRejections (internal/process/task_queues.js:94:32)

I'm doing something wrong for sure, but I don't know what.
And I have no idea of what to write in the bundleAstReferenceKeys (and maybe in entryPoint option too)
Could someone help me?
Thanks

ok, I changed the test.js script  bundleAstReferenceKeys to an empty array [] and it worked.

Could someone answer please please? 🙏🏻

Sorry I was busy.
The files you've given are not complete. There should be more things there.
!function(e){function n(t){if(r[t])return r[t].exports;var o=r[t]={exports:{},id:t,loaded:!1};return e[t].call(o.exports,o,o.exports,n),o.loaded=!0,o.exports}var t=window.webpackJsonp;window.webpackJsonp=function(a,l){for(var p,c,s=0,i=[];s<a.length;s++)c=a[s],o[c]&&i.push.apply(i,o[c]),o[c]=0;for(p in l)e[p]=l[p];for(t&&t(a,l);i.length;)i.shift().call(null,n);return l[0]?(r[0]=0,n(0)):void 0};var r={},o={4:0,6:0};n.e=function(e,t){if(0===o[e])return t.call(null,n);if(void 0!==o[e])o[e].push(t);else{o[e]=[t];var r=document.getElementsByTagName("head")[0],a=document.createElement("script");a.type="text/javascript",a.charset="utf-8",a.async=!0,a.src=n.p+""+e+"."+({5:"bundle",6:"common-bundle"}[e]||e)+".js",r.appendChild(a)}},n.m=e,n.c=r,n.p=""}([
/** more code should be here */
]);

webpackJsonp([6,4],[/** more code should be here */]);


Iterate on leaderboards to bring us to v1.2

This will be mostly architectural, may post additional details but a new PR will be made.

Ended up bringing us to v1.3 but done


Link to codesandbox under playground doesn't work

Have you read the Contributing Guidelines on issues?

[x] I have read the Contributing Guidelines on issues.

Description
Hi. The link to codesandbox here: playground doesn't work. It gives error:
-1 - Error: UNKNOWN_ERROR - 500 - {"error":"Error","message":"authorize error: pitcher-host-container didn't respond in 5s"} (500)

Version: 0df28d8
Environment: production
Event: PITCHER:ERROR
State: CONNECTING
Branch: docusaurus

(Stackblitz works)
Self-service

[ ] I'd be willing to address this documentation request myself.

Now it works.

I also think it works :)


Add custom 404 page

Changes

add custom 404 page
merge 2 links from the header into a single one (a11y improvement)
improve SEO title

I've made some changes to the original design, so the banner image is not so big now. This way, the main message about the page wasn't found is readable without scrolling and everything fits better:

Context

Closes #258

I can't find the way to test the custom 404 page with Cypress, I always get this:

I'll investigate more about it. Otherwise, it might be the perfect case to add a jest + Testing Library test.

I might need to check it by asserting the request body like here: https://www.cypress.io/blog/2019/09/04/guest-post-the-concept-of-monitoring-tests/#what-else-monitoring-tests-could-check
If that's the case, I definitely will test this with Testing Library instead.

I can't find the way to test the custom 404 page with Cypress, I always get this:

I'll investigate more about it. Otherwise, it might be the perfect case to add a jest + Testing Library test.

Hmm, it looks like Cypress goes: "Received 404 status code, abort test" instead of going "404, follow redirect, check page content".
It says in the error message that you can ignore status codes, but that sounds like a bad idea, because for existing pages we want the test to fail on 404 status code... 😄

Hmm, it looks like Cypress goes: "Received 404 status code, abort test" instead of going "404, follow redirect, check page content".
It says in the error message that you can ignore status codes, but that sounds like a bad idea, because for existing pages we want the test to fail on 404 status code... 😄

I see! It would be interesting to ignore the 404 code in this very specific test for testing purposes tho. I'll check that possibility.

We do run the risk of "painting ourselves into a corner here" as we need a way to switch between light and dark mode UI text colors programatically. But I think you already had ideas on that?

Yes, I have a bare idea of how to do it. I expect to figure out more about it on #515, but basically, the idea will be that just using the token "primaryText" gets the corresponding color for either light or dark mode (same for secondary and tertiary variants).

We do run the risk of "painting ourselves into a corner here" as we need a way to switch between light and dark mode UI text colors programatically. But I think you already had ideas on that?

Yes, I have a bare idea of how to do it. I expect to figure out more about it on #515, but basically, the idea will be that just using the token "primaryText" gets the corresponding color for either light or dark mode (same for secondary and tertiary variants).

I like this idea a lot! Easy to write, easy to think about, easy to maintain. 🥳


Jonas regressor

train_ridge_regressor and correspondign tests

@jon-kri  I think the test is failing because you have to export the train ridge regressor function otherwise the test class can't find the function


How to convert kryo5 object to string in redis function?

I have a redis function that takes a list of arguments and executes HGET. The problem that I have is that the first argument is a time to live. This argument gets encoded by Kryo5 and sent to the redis function as an argument. I am not sure how to convert it in my redis function from the encoded string to an integer.
Object[] args = new Object[3];
args[0] = 123456;
args[1] = "some key";
args[2] = "the key value";
redisson.call(FunctionMode.WRITE, "test", FunctionResult.Value, List.of("map"), args);

...

local function do_something(keys, args)
    redis.log(redis.LOG_WARNING, args[1]) -- this is not an integer. This is an encoded kryo5 representation of the integer.
end

You need to use org.redisson.client.codec.ByteArrayCodec#INSTANCE instead and encode each param manually to byte array.

Thank you @mrniko . Just to be clear there is no way to convert the Kryo5 object in redis to an integer? In order to do what I am trying to do, I would have to convert the object into a byte array and then decode it back into a string?

Just to be clear there is no way to convert the Kryo5 object in redis to an integer?

You can do it only on Java side


fix(android): Disconnect listeners now only accept disconnect events from matching devices.

Description
When connecting to multiple devices on Android, the first one to disconnect ends up calling the disconnect handler for all attached devices, leaving some devices in a limbo state where they are connected to the phone, but the app no longer knows about it. This also helps to alleviate unexpected disconnect events for devices that should remain connected.

@facetious so i understand that was an issue which appeared after your refactoring from the last PR, right?
I dont see that issue with the production release which right now does not use that last PR

@facetious so i understand that was an issue which appeared after your refactoring from the last PR, right?
I dont see that issue with the production release which right now does not use that last PR

@farfromrefug No, this error exists on the production release as well. Specifically, this line has no guard for device matching and the disconnectListeners array is a list of all listeners.

@farfromrefug No, this error exists on the production release as well. Specifically, this line has no guard for device matching and the disconnectListeners array is a list of all listeners.

@facetious indeed!!!! thanks a lot for this one!!

@facetious indeed!!!! thanks a lot for this one!!

@facetious finally had the time to test this. Just released it. Thanks a lot for your PRs


Run without a webserver

If I try to use pagefind currently loaded from a simple HTML file, it will fail with
pagefind-ui.js:1 Uncaught (in promise) TypeError: Failed to resolve module specifier '[...]/_pagefind/pagefind.js'. The base URL is about:blank because import() is called from a CORS-cross-origin script.
    at HTMLInputElement.ut (pagefind-ui.js:1:37346)

This seems like a strange thing to fail on, because I'd expect pagefind to just work with completely static files - since everything needed is already there. It does work if being served via any webserver.
Would it be possible to fix this, as it would allow completely static backups of a site to be searchable too without having to run pagefind --serve?

Hi @karalabe 👋
Unfortunately Pagefind as it stands is fairly incompatible without being behind a webserver. It relies on JS modules, which I don't think have any support in the file:// protocol, and similarly I don't think webassembly works in this realm either 😕
Sorry to not have better news, but at this point using some static webserver is the only real solution (I usually use python3 -m http.server to spin something quick up on a machine without installs)

That’s ok! Unfortunate and annoying, but I’m mostly clueless on the .js
department so I’ll take your word for it. Sweet project nonetheless!
On Tue, 24 Jan 2023 at 03:06, Liam Bigelow @.***> wrote:

Closed #202 https://github.com/CloudCannon/pagefind/issues/202 as not
planned.
—
Reply to this email directly, view it on GitHub
https://github.com/CloudCannon/pagefind/issues/202#event-8340007082, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAA7UGJZXTYOQEUU7GL4RFDWT42KZANCNFSM6AAAAAAUAHQIJU
.
You are receiving this because you were mentioned.Message ID:
@.***>


Fix README.md's scanCode issues.

master is broken due to trailing whitespace.
@mrutkows did scanCode's config change without updating the main repo?

@markusthoemmes it was a pr that had passed travis before the scan code changes.


Glob syntax not supported for Windows out of the box [with esbuild]

Currently, the build script involves: esbuild app/javascript/*.* --bundle --outdir=app/assets/builds
On Windows, this results in an error: Could not resolve "'app/javascript/*.*'" (glob syntax must be expanded first before passing the paths to esbuild).
From what I've found troubleshooting, it appears that the glob syntax is a shell-specific feature. esbuild does not interpret it on its own. With PowerShell or CMD on Windows not supporting the glob syntax properly, the build commands doesn't work out of the box in these shell environments.
I'm not sure how to go on from here for a general solution. For now, in my project I changed the build script to reference the only .js file I have: esbuild app/javascript/application.js --bundle --outdir=app/assets/builds

Would be nice to have a section in the README about Windows considerations 👍

We could actually also detect Windows in the install script and change the default build script proposed. Feel free to have a look at that if you'd like!

Yeah, the only problem is that I'm not sure which command could do the trick on Windows, without specifying all the js files.
I'm not sure it is possible without an esbuild config file and something like an external glob package...

Documented it via https://github.com/rails/jsbundling-rails/commit/95e6b15d02153e852712212c666af17a45f45d15


Workaround double load of each URL path

Work around Dash double firing of routes : https://github.com/plotly/dash/issues/1049 that is causing the DNN Keras model to be loaded twice

Workaround created on the main_network_view in 23600bbd0cd8d21a2b1861bb4bf42eb0fefc7698

Follow up issue in #38


build: improve test runner (refs #7)

set exit code
add test coverage(c8)
small refactor(cli)

chore: small refactor test
chore: add test/cli to make testing working
build: add coverage

Note that I extended this printer as seen here so the printer tests can be run correctly. That code need to be ported back into this code :)
And can you try to get rid of the constants and make a .md template instead?

@aladdin-add Need to make sure non-ascii chars and escaped chars get correctly printed in .md files. I discovered some issues regarding this.

All bullet points in https://github.com/kataw/kataw/issues/7 solved or later PR's?

Maybe you should update the snaps?

for better review, will update in next commit


fix: align concurrency groups

Fixing a issue when workflow github.ref_name was causing undesired cancellation of other running workflows.
This condition is too broad.
GitHub docs: https://docs.github.com/en/actions/using-jobs/using-concurrency#example-using-a-fallback-value

:tada: This PR is included in version 4.8.4 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Migrate SES integration to AWS SDK 2

Migrate SES integration from AWS SDK v1 to AWS SDK v2 in 3.0.x branch.
SES should be moved completely from the core/context module and all SES related classes should be in spring-cloud-aws-v3-ses module.
Consider the following guidelines:

Create package io.awspring.cloud.autoconfigure.ses at spring-cloud-autoconfigure which contains SesProperties, SesAutoConfiguration
Create module spring-cloud-aws-ses
Create module spring-cloud-starter-aws-ses which contains a pom.xml with spring-cloud-autoconfigure and spring-cloud-aws-ses dependencies
Prefix should be spring.cloud.aws.ses

Shouldn't we migrate the spring-cloud-aws-core first ?

@driverpt core will not be directly migrated as quite a few classes will be either removed or moved to separate modules. Integrations definitely need autoconfigure module to be there so that they can use credentials and region providers.
It will become clearer once we have autoconfigure and at least single integration migrated.

@MatejNedic @maciejwalkowiak Can I give this a shot?

Description updated with guidelines to build the module

Description updated with guidelines to build the module

All guidelines have been factored in #84.

Closing in favor of #84


:seedling: Fix failing e2e tests

Please check if the PR fulfills these requirements

[ ] Tests for the changes have been added (for bug fixes / features)
[x] PR title follows the guidelines defined in  https://github.com/ossf/scorecard/blob/main/CONTRIBUTING.md#pr-process

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
Bug Fix

What is the current behavior? (You can also link to an open issue here)
Failing e2e tests

What is the new behavior (if this is a feature change)?
The packaging docker image for scorecard has been removed from github
workflow to gcr.io.
This was causing the e2e check failing.
This fix will remove that check and address the failing e2e.

Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)
None

Other information:
None

Slightly worried here, so will let @azeemsgoogle @azeemshaikh38 review this. we might forget to reenable this, so lets see if we can fix it somehow.

We will need to have a GitHub workflow with docker push. TBH I find that not a valid check. As a project, we don't use GitHub action to do any packaging.

Cannot merge any PR's till we make a decision for this.


Instruction "get off the bike" is missing

Please look at this route: https://www.openstreetmap.org/directions?engine=graphhopper_bicycle&route=51.25745%2C7.14858%3B51.25727%2C7.14617
The cyclists are routed over a pedestrians way here, which has no bicycle=yes or any allowing value (I mean this: https://www.openstreetmap.org/way/180247410). But the default bicycle-Value for highway=pedestrian is "no", isn't it?

Bicycle is not forbidden as we can force it into "get off the bike", so we assume a speed of 6km/h.
But still an information about this in the instructions is missing.

is RacingBikeFlagEncoder.java the file which needs to be changed?

This is fixed (GraphHopper Maps)

It is no longer part of the instruction text, but you can fetch it via the path detail get_off_bike.

I tried to find it also in the path detail but I was not successful. Could you please show me an URL which returns it? Thank you!

details=get_off_bike should do this for the API or GH Maps:
https://graphhopper.com/maps/?point=51.439644%2C14.245191&point=51.439387%2C14.244938&vehicle=bike&elevation=true&details=get_off_bike


jellyfin: remove assertion if emby enabled: no emby module exists

Motivation for this change
Jellyfin replaced the emby module, however the used PR expected the emby module to exist, and therefore had an assertion when !config.services.emby.enable, which prevents NixOS from rebuilding because there's no emby attribute in config.services.
This commit removes such assertion, allowing jellyfin to be used again.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[x] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

If I'm running the tests correctly, the jellyfin test didn't run before this commit:
nix-build --no-out-link '<nixpkgs/nixos/tests/jellyfin.nix>'
error: attribute 'emby' missing, at /nix/var/nix/profiles/per-user/root/channels/nixos/nixos/modules/services/misc/jellyfin.nix:53:22
(use '--show-trace' to show detailed location information)```

Wheras now, it doesn't report explicit success, but it appears to run successfully.

@DIzFer Thanks!


Do not change user's environment, even during testing

Test in xt-bad is still failing..

Test in xt-bad is still failing.

Fixed in version 0.11.1.


fix: #8997 - auth trigger detached after push

Description of changes
This PR fixes #8997. A Lambda trigger was detached after subsequent updates to the auth resource.
Issue #, if available
fixes: #8997
Description of how you validated changes
Manually and new e2e test is added
Checklist

[X] PR description included
[X] yarn test passes
[X] Tests are changed or added
[ ] Relevant documentation is changed or added (and PR referenced)
[ ] New AWS SDK calls or CloudFormation actions have been added to relevant test and service IAM policies

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report

Merging #9312 (3b4dbca) into master (4c12eb1) will increase coverage by 0.08%.
The diff coverage is 88.00%.

@@            Coverage Diff             @@
##           master    #9312      +/-   ##
==========================================
+ Coverage   55.36%   55.44%   +0.08%
==========================================
  Files         887      887
  Lines       50331    50343      +12
  Branches    10869    10873       +4
==========================================
+ Hits        27866    27913      +47
+ Misses      20393    20361      -32
+ Partials     2072     2069       -3

Impacted Files
Coverage Δ

...amplify-appsync-simulator/src/server/operations.ts
47.36% <25.00%> (-0.78%)
:arrow_down:

packages/amplify-appsync-simulator/src/index.ts
73.52% <100.00%> (+0.26%)
:arrow_up:

...s/amplify-appsync-simulator/src/type-definition.ts
100.00% <100.00%> (ø)

...ppsync-simulator/src/utils/auth-helpers/helpers.ts
86.20% <100.00%> (+5.25%)
:arrow_up:

...dformation/utils/generate-auth-trigger-template.ts
100.00% <100.00%> (ø)

...y-provider-awscloudformation/src/push-resources.ts
17.30% <100.00%> (+2.54%)
:arrow_up:

...ormation/src/resource-package/resource-packager.ts
90.34% <100.00%> (+3.82%)
:arrow_up:

...dformation/src/resource-package/resource-export.ts
80.90% <0.00%> (+5.90%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update db0f8ac...3b4dbca. Read the comment docs.


DB RESTORE DOES NOT PROCEED

Hi,
i am using python3.6 and django-dbbackup==3.3.0 and django==1.11. i am unable to restore my backups. when i run the command python manage.py dbrestore the terminal does not proceed after confirming 'Are you sure you want to continue? [Y/n]'. Please how do i go about resolving this?

Never mind, was able to resolve it by reverting to django-dbbackup==3.0.2


build: Re-enable Scala style checker and spotless

This PR re-enables Scala style checker and spotless Maven plugin. Now the CI should fail when the code style is incorrect.

cc @viirya @andygrove

Thanks! merged


Estimated development time.

Summary 💡
Can we add an extra output line where we mention an estimate of how many hours went into development?
I'm afraid there is no well-maintained library that implements this, but I believe any simple formula with simple assumptions will do.

Fun idea! First we'll probably want a utility in place to detect files that are generated or vendored, so that committing e.g. dist/* or bootstrap.min.js won't incorrectly increase the calculated work time.
After that, I'd assume we'd get work from an estimated lines of code per hour or bytes per hour (there are pros and cons to each). The simplest way would be to check the current files, but this won't take deletions into account. If a project is rewritten, we'd miss a lot of the work hours! The most accurate way would probably be to compare all the deltas of the commits discoverable in the current branch, but this will be more expensive and time-consuming.

Thanks for expressing interest. That's a lot of ideas for first comment. We can start simple by hardcoding the exclusion of cache files and generated code. There are good tools that identify such files for all types of projects, e.g., kondo.
This repo has implementation. The README refers to 30 lines of code algorithm with decent explanation.
git-hours
I tried installing it but it didn't work.
This one worked but it vastly overestimated the time.
git-estimate
The unreasonable numbers are probably due to the caveats that you discerned in your first line of reasoning.

:thinking: As I understand it, both of those use the time between commits to estimate the time worked. TBH I think this is a mistake. For example, I will often use git add -p to select the changes I want and ignore the changes I don't want for a commit, commit them, then repeat until I've committed all my changes. This practice means that I might have spent a significant amount of time, but my commits would all be grouped into the span of a few minutes. This could lead to a significant underestimation of time spent. One could also make these tools significantly overestimate time by simply making a small commit that took 5 minutes to write, waiting an hour, and making another small commit. The actual time spent might just be 10 minutes, but, AFAIK, these tools would consider that an hour of work.
IMO any tool that estimates time worked in a git repository would need to take code size into account (lines of code, file size, or something else). A quick Google search told me that the average developer writes 20 lines of code per hour, so that might be a good starting point.

I propose a different factor that incorporate the size of the repo. The factor of 20 lines per hour could be reasonable the project is small. But when the project grows in complexity, you spend way more time observing tthe consequences of any small change on the overall behaviour.

I'm glad you're thinking about this! For the sake of a reasonable estimate, I wonder if "development time" should be reimagined as "coding time" -- i.e. the actual time spent writing code for the project. "Development" is a broad subject, and can include responding to issues, reviews, planning, etc.

:thinking: I think that calculating time spent can be a serious undertaking with a lot of code complexity. Additionally, it would have a lot of utility outside of this project. For this reason, I believe it shouldn't necessarily be part of the onefetch project, but a completely separate crate/project that maybe onefetch can add as a dependency. @o2sh Thoughts?

I love the idea, thanks for the suggestion @thisismygitrepo.
Finding the "right" algorithm will be a trade off between complexity-accuracy and the solution will inherently be opiniated.
Personally, I'm okay with it being a rough estimate as long as the logic is sound and well-documented.

I believe it shouldn't necessarily be part of the onefetch project, but a completely separate crate/project that maybe onefetch can add as a dependency.

For the sake of simplicity in review, integration, and testing, we could start by incubating it within onefetch for the first iteration, and then later on extract it into its own crate. Alternatively, we could begin with it as a separate crate from the outset. Both approaches are feasible.
@thisismygitrepo If you're up for it, feel free to give it a try!


Log/uid and user information for garm for better traceability

Description
Added a debugging log for running user & its userid

Don't mind about the fatal log: 2024-10-28 03:20:57    [FATAL] │ listen tcp :443: bind: permission denied
Assignees

[x] Assignees is set

Type of changes

[x] Apply one or more labels of the following that fits:

bug: Bug fix
dependencies: Dependency upgrades
documentation: Documentation changes
enhancement: New Feature
good first issue: First contribution
logging: Log changes
refactor: Refactoring (no functional changes, no api changes)

Flags
- [ ] Breaks backward compatibility
- [ ] Requires a documentation update
- [ ] Has untestable code

Checklist
- [ ] Followed the guidelines in the CONTRIBUTING document
- [ ] Added prefix `[skip ci]`/`[ci skip]`/`[no ci]`/`[skip actions]`/`[actions skip]` in the PR title if necessary
- [ ] Tested and linted the code
- [ ] Commented the code
- [ ] Made corresponding changes to the documentation

Checklist for maintainer
- [ ] Use `Squash and merge`
- [ ] Double-confirm the merge message has prefix `[skip ci]`/`[ci skip]`/`[no ci]`/`[skip actions]`/`[actions skip]`
- [ ] Delete the branch after merge

LGTM


Word Change

Changed suitable to available in sentence. The sentence as is gives the impression that there IS an existing table at NCBI.  There will be cases where the table does not exist at NCBI but may exist at another database.  So we need to make it clear that NCBI is not assumed to be the only place a table might be.

Thank you @maxglycine! We can commit directly to master to keep admin processing steps to a minimum


Add ASCII filter

@barnabwhy wanna give it a look? <3
Seems to be working fine so far in prod.

Bad idea this is ngl. Would stop many non-latin alphabetical characters being used

Hmmm, given that the set of all Titanfall maps and gamemodes is known, we could actually even go more restrictive for those and limit those to pre-defined values.
(Will require master server update though if we add new maps or modes)
For server names I agree that we should allow valid UTF-8 text. Issue is homoglyph attacks to get around word filter etc.
Maybe there's a library that handles that?

Modded gamemodes exist though

Modded gamemodes exist though

True. So I guess keep ASCII requirement for modded gamemodes? Doubt they'll ever include UTF-8.

Potential candidate for UTF-8 filter: https://www.npmjs.com/package/utf-8-validate

That doesn't actually do what we want it to do. Running that with the data from before, it returns valid. It is checking for a valid utf-8 string from a bytearray.

After some more digging, i found the following regex:
/[\p{Cc}\p{Cn}\p{Cs}]+/gu
from this stackoverflow thread.
I ran it against a capture of earlier data and it seems to perfectly match all bad servers, while not including any legit servers. Also tried this with some chinese and japanese glyphs, and those seems to work fine too.

Not necessary at the moment, will revisit if an issue arises again.


configs,scons: Update scripts and build_opts to make GPU-FS simulations more configurable

This PR adds support for command line arguments in GPU-FS runs to allow the user to configure several parts of the GPU. It also increases the bits per set in the build_opts/VEGA_X86 file to enable GPU-FS simulations to use 64 directories or more.

@v-ramadas is this ready for review again?  Or do you need to resolve the bitset issue first (not sure if that is a blocker @abmerop?)?

@v-ramadas is this ready for review again? Or do you need to resolve the bitset issue first (not sure if that is a blocker @abmerop?)?

Definitely not a blocker. That should be a separate PR in fact

Would it be better if I remove the NUM_BITS_PER_SET=128 change from this PR and make it separate (after looking into the suggestions made)?

Would it be better if I remove the NUM_BITS_PER_SET=128 change from this PR and make it separate (after looking into the suggestions made)?

No, my sense from @abmerop 's reply above is we should leave it and fix it separately / after.


columns_mobile is not recognized as property in Python API

Description
columns_mobile is not recognized as property in Python API. This makes it hard for people to find it or understand it. I had the issue with a customer.
Acceptance Criteria

[ ] Ensure new code is unit tested, and check code coverage is at least 90%.
[ ] Propagate any change on the demos and run all of them to ensure there is no breaking change.
[ ] Ensure any change is well documented.

Code of Conduct

[X] I have checked the existing issues.
[ ] I am willing to work on this issue (optional)

you need to use columns__mobile (note the 2 _)


Increase tolerance for slow tests

Nothing was wrong, just our new karma/mocha setup being intolerant of occasional slowness or browser disconnects in Circle.
cc @hankim813 @ndhoule to review

Hmm, actually maybe some things were broken after all.


Save FaceEncoding

Is there a way to save the FaceEncoding as a math operation for later use again?

@danijerez
You can serialize method you want to use because FaceEncoding has serializable attribute.
Or you can call GetRawEncoding method to get float[].


Keyboard & Mouse Freeze on Exception

Hello, i use your dll for checking inputs and playing commands (hotkey for game) and when i debug my program in VS, everytime i get an exception my mouse and keyboard will freeze for good 1-2 minutes and i must wait (or restart PC which is faster than waiting)... It happens also when my main form where i check it disappears.. (I always unhook when it disappeard, that remove issue when running)
Any advice?

have freez too, its not cool

Anyone figure out how to prevent this?


fix(release): temporarily shift mac sys deps to custom build setup

fixes #366

Thank you so much!


feat: kafka ingester

https://github.com/seznam/slo-exporter/pull/48/commits/0c79a428ff00769aeadc80aa4e3e80b7569b34f5 adds few more configuration opts. This should be just enough to keep configuration simple and minimalist while covering the most probable use-cases.
As discussed offline with @FUSAKLA, https://github.com/seznam/slo-exporter/pull/48/commits/6b6a987a9eeca9608125732059491de160ea2403 introduces versioning of received data and defaults (missing) Quantity to 1.

Failing test (TestOffsetPersistence), yet another of https://github.com/seznam/slo-exporter/issues/35
Its already quite annoying, I'm going to address this asap.... :/


Scrutinizer Auto-Fixes

@jarednova requested this pull request.
It consists of patches automatically generated for this project on Scrutinizer:
https://scrutinizer-ci.com/g/timber/timber/

Coverage increased (+3.2%) to 91.697% when pulling ed71dd76055c7fd96c7b8dab70dbe06ae2cc9f4c on scrutinizer-patch-1 into f6bd4bd450c12f743f82195046a6f4d9c71536ae on master.


fix: disable cas idp not working

这个 PR 实现什么功能/修复什么问题:
修正：禁用cas后login应该不显示cas登录
是否需要 backport 到之前的 release 分支:

release/2.11
release/2.12

/area apigateway

Codecov Report

Merging #3030 into master will increase coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master   #3030      +/-   ##
=========================================
+ Coverage    7.17%   7.17%   +<.01%
=========================================
  Files         478     478
  Lines       76742   76742
=========================================
+ Hits         5504    5506       +2
+ Misses      70775   70774       -1
+ Partials      463     462       -1

Flag
Coverage Δ

#aFlag
7.17% <ø> (ø)
:arrow_up:

Impacted Files
Coverage Δ

pkg/util/seclib2/seclib.go
95.74% <0%> (+4.25%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update be416cb...d80c8fb. Read the comment docs.

/approve
/lgtm


Add metrics docs to cli reference

This adds the cli docs for the metrics-addr flag and a small description on how it can be used.
We can add full api docs after this feature is removed from experimental as metrics and metric names can be modified or removed before then.
Fixes #27843
Signed-off-by: Michael Crosby<EMAIL_ADDRESS>

@thaJeztah this has been updated


Browser tab hangs when (async) refreshing streamlit pages

Streamlit doesn't have a nice built in way of refreshing the pages.
There's streamlit-autorefresh but it is a very hacky way.
I like to use the following async event loop based pattern:
import asyncio

import pandas as pd
import streamlit as st
from st_aggrid import AgGrid

def main():
    try:
        asyncio.run(run_async())
    except Exception as e:
        print(f"error...{type(e)}")
        raise

async def run_async():
    while True:
        # Assume this line could be something like:
        # df = await refresh_data_from_source()
        df = pd.DataFrame({"a": [1,2,3], "b": [1,2,3]})

        # This works well!
        # st.dataframe(df)

        # This does not: AgGrid never shows up!
        AgGrid(df)

        await asyncio.sleep(10)

if __name__ == "__main__":
    main()

Though it works well with the built in dataframe viewer. It doesn't work with this AgGrid setup.
What ends up happening is that the browser tab gets hanged / stuck. I ran a browser profile which pointed me to the JsCode object - so essentially, we resend the code object on every refresh and browser just stays busy in loading that 5 MB JS object!

Try setting a height to the grid.
Did it work when not async?
I've never saw async calls on streamlit before. I always used @cache decorators

Yes, it does work when not async, but we don't get the refresh behavior we need. I'll try setting the height
Is it not possible to load the JS object only once? because that is not gonna change!
And only send the data to the UI that is actually changing? Like what happens for the default streamlit.dataframe class.

To avoid grid reloading, set a fixed key when calling aggrid. e.g AgGrid(...,key="grid1")
This the reload behavior comes from streamlit itself and how it defines component keys when no key parameter is defined.


Parsing error in table.load_sql() can be handled more intelligently

Hi, team
if possible, could you kindly stop parsing of sql recordset after the first row if there're too many columns
At the moment, if the data schema definition doesn't match the sql recordset we are trying to load, then there'll be no exceptions in the Jupyter Lab, and the server will go through all the records and send a message to the logs saying "I can't parse the record"  - this is slow and not user friendly. After every record was processed, then through the table.loading report we'll see that none of the records were loaded.
It would be nice, if right after atoti received the response from the SQL Server, it would be able to stop processing if the columns count doesn't match datastore columns count.
Thank you
Anastasia

Hello,
All sources are currently working this way because they are not strict. We currently don't want the load to stop if one or more rows are failing. I think on SQL it is possible for you to check that the loading is working properly easily :

First try to run your query with a very small LIMIT to limit the number of rows returned by your query.
If you have a mismatch in types between atoti and sql table you can use CAST in your query

Did you create your table using read_sql or not ? It is strange to have all the rows failing if you have first done a read_sql


There is inconsistent or insufficient error handling in asynchronous operations.

There is inconsistent or insufficient error handling in asynchronous operations, which may lead to unhandled promise rejections and application crashes.
Implement standardized error handling throughout the codebase, such as using try/catch blocks in async functions and ensuring that errors are properly logged and returned to clients.    add gssoc-ext and hackoctober labels to it
I wants to work on this issue . can you plwase assign this issue to me.

i would like to work on this since there is not much progress on this issue. Pls assign me this issue