Pulseaudio fails to restore original sample settings

Hello, i'm having problems with pulseaudio on the hassio-audio image, every time I try to use the voice assist (wyoming protocol+ assist microphone+openwakeword+piper+whisper addons) or just piper using the Text-to-speech (TTS): Speak service trough the VLC-TELNET entity, the audio logs get spammed with this message until the memory runs out.
PaltaHass hassio_audio[1543]: W: [alsa-sink-ff880000.i2s-ES8316 HiFi ES8316 HiFi-0] alsa-sink.c: Resume failed, couldn't restore original sample settings.
what could be causing this?
thanks in advance
ha audio info.txt
ha audio logs (before and after killing pulseaudio).txt
ha info.txt
pa-info (in hassio-audio)

looking at module-alsa-sink.c i've found:
        pa_log_warn("Resume failed, couldn't restore original sample settings.");
        goto fail;
    }

And the PA wiki says "This tests if the sample format specifications are identical."
Should I change somethink in order to make the hassio_audio container work properly?

Most likely the driver or your platform doesn't support the sample rate TTS is trying to use :cry:
Not sure what sample rate TTS is using. Have you tried playing sound directly with various sample rates on Debian on this platform (maybe you have to shutdown the Supervisor service and all containers to avoid conflicts).
Did you try Home Assistant OS on that same platform? The ODROID-C2 is not all that well tested, so I am not sure if audio works on it :cry:

Thanks for confirming. So yeah, this is a platform specific driver issue then :cry:
I'd report this to the Orangepi 4 folks.

I'm not so sure, mostly because the TTS part works correctly if Spotify connect is playing music inside HA, and audio also works fine in baremetal, it's just when pulseaudio is connected through the hassio audio sock that fails to resume the sink.

I guess it works correctly when Spotify is playing because then Pulseaudio mixes the two audio streams, and resamples TTS to whatever Spotify is using. To me it seems that the driver is advertising support for a sample rate (and Pulseaudio is trying to use it when only TTS is running) which it can't handle when trying to actually play it.

You were right all along (go figure!) after further testing with speaker-test, audio plays correctly at 44100Hz but breaks at 48000Hz.
Is there a way to configure pulseaudio inside the container/image to use a fixed sample rate? (non volatile config files?)
thanks for your patience

No, currently we don't have such a "force baudrate option". In fact, i am not even sure if Pulseaudio has that. There is default-sample-rate, but I'd assume that Pulseaudio would also pick a higher sample rate if needed.
The problem here is really the audio driver advertises it is capable of 48000Hz. And Pulseaudio (correclty) makes use of it. As a simple fix, you probably can just delete that from the audio driver, but it really would need a kernel change.

In the end it was the Assist Microphone add-on who was causing all the playback issues, once deactivated, every sample rate works perfectly. Now i'm trying to know what's causing such problem, I'll post again if I find something.

Maybe Assist Microphone plug-in uses an audio input which influences clocks of your audio output such that it can't run on those two different rates? :thinking:  What are your ALSA audio inputs (arecord -l)?

arecord -l
**** List of CAPTURE Hardware Devices ****                                                                                card 0: rockchipes8316c [rockchip-es8316c], device 0: ff880000.i2s-ES8316 HiFi ES8316 HiFi-0 [ff880000.i2s-ES8316 HiFi ES8316 HiFi-0]
Subdevices: 1/1                                                                                Subdevice #0: subdevice #0

arecord -L
null                                                                                Discard all samples (playback) or generate zero samples (capture)                                                                                hw:CARD=rockchipes8316c,DEV=0                                                                                rockchip-es8316c, ff880000.i2s-ES8316 HiFi ES8316 HiFi-0                                                                                Direct hardware device without any conversions                                                                                plughw:CARD=rockchipes8316c,DEV=0                                                                                rockchip-es8316c, ff880000.i2s-ES8316 HiFi ES8316 HiFi-0                                                                                Hardware device with all software conversions                                                                                default:CARD=rockchipes8316c                                                                                rockchip-es8316c, ff880000.i2s-ES8316 HiFi ES8316 HiFi-0                                                                                Default Audio Device                                                                                sysdefault:CARD=rockchipes8316c                                                                                rockchip-es8316c, ff880000.i2s-ES8316 HiFi ES8316 HiFi-0                                                                                Default Audio Device                                                                                dsnoop:CARD=rockchipes8316c,DEV=0                                                                                rockchip-es8316c, ff880000.i2s-ES8316 HiFi ES8316 HiFi-0                                                                                Direct sample snooping device`


ERROR: When trying to generate audio from text

Steps to reproduce

run npm start
Use the Main branch for testing
select a voice
input text
click play  button

Error
Server is listening on port 3000
Error writing audio to file: [Error: ENOENT: no such file or directory, open 'audio/audio.wav'] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: 'audio/audio.wav'
}
[Error: ENOENT: no such file or directory, open 'audio/audio.wav'] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: 'audio/audio.wav'
}
Error writing audio to file: [Error: ENOENT: no such file or directory, open 'audio/audio.wav'] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: 'audio/audio.wav'
}
[Error: ENOENT: no such file or directory, open 'audio/audio.wav'] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: 'audio/audio.wav'
}

@SandraRodgers 👋

I made the changes to fix this in #4 @SandraRodgers


backing up state to s3

Im quite new to terraform. Is it possible to modify tack to backup tf state to s3?

Yes, it's possible. You can modify the Makefile to do it. Check the part of code that upload the keypair to s3 and you Will know how upload the tf state to s3.


Do not parse charges on Record parser

Context
The record parser extracts the cases from the search results page. It also extracts the charges associated to the case, but this is not necessary since the charges can be extracted from the case details page in more detail.
Tasks

[ ] Update tests to check the parser skips the charge column.
[ ] Remove the sections responsible for parsing the charge columns for each case.
[ ] Ensure all existing functionality does not change.

@nwalton125 Hey! This task hasn't been updated in a few months so I just wanted to check in and see how it's going. Are you still working on this task? If not, or if I don't hear back, I will open it up for someone else to work on.

Hello!
Sorry! I am not, best to open it up for someone else.
On Wed, Mar 27, 2019 at 7:44 PM Max Wallace<EMAIL_ADDRESS>wrote:

@nwalton125 https://github.com/nwalton125 Hey! This task hasn't been
updated in a few months so I just wanted to check in and see how it's
going. Are you still working on this task? If not, or if I don't hear back,
I will open it up for someone else to work on.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/CodeForPortland/recordexpungPDX/issues/34#issuecomment-477423938,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHk63lEd84NMqqIZKXXQflBk0p0GcNHkks5vbCyQgaJpZM4ZGLHP
.

@nwalton125 no worries! Thanks for the reply :)

@maxkwallace I added a temporary fix to this in another commit. It still parses the charges, but it doesn't save the results. We could probably close this issue?
It's not so important, but what I would actually like to see done is a complete re-write or refactor of the record parser so that it is easier to understand. What do you think?

@NickSchimek sounds good, I'll close the issue.
Regarding the refactor I'd have to read the parser code myself in order to comment. I trust your judgment. There are a lot of factors that go into deciding whether a refactor makes sense but IMO the main ones are (1) whether the code is functioning well now and (2) whether or not it will need to be edited many times in the future.

@maxkwallace yeah, I completely agree. It most likely will not have to be edited in the future, and it currently works so.... It was just something that would be nice, but is not currently necessary.


Line Lens 'never' doesn't get rid of Line lens

Describe the bug
When the Line Lens setting gets set to 'never' the type hints don't disappear on functions.
File extension is '.fsx'
Expected behaviour
When set to 'never' the type hint disappears completely.
Screenshots
The setting:

When set to 'never'

When set to 'replaceCodeLens'

When set to 'always'

Machine info

OS: Windows
.NET SDK version: 5.0.301
Ionide version: 5.10.1

@Degubi You need to disable codelenses in VSCode if you don't want any type hints. This setting does get rid of the line lenses, but it doesn't get rid of code lenses.

Disabling Editor:Code Lens helped, thank you @cartermp !

One thing that might be worth looking into (I might just snipe myself into doing this) is having a separate setting for disabling type hints in code lenses independently of the global code lenses setting. Happy this worked for you though!


libcamera: fix binary reproduciblity

Description of changes
libcamera is a quite important package as it's used for pipewire.
Hence if we want a binary reproducible desktop, we need to ensure that
libcamera also is able to build reproducible.

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Might need to move this staging. Let's see.


k8s 1.12.x

Hi, do u plan to upgrade this wonderful code to k8s version 1.12.x?
really appreciate your sharing this code. help me on setup k8s with HA.
thank you very much

You are welcome, I will upgrade later.


Add AsyncDrain

This PR is for adding AsyncDrain that has similar apis to Drain.
And it is required for https://github.com/open-swift/S4/pull/52
Please review this!

You've got a thumbs up from me. But @paulofaria knows more about this stuff so let's wait for his thumbs up as well.
I think we can merge this today since it's been neglected for so long.

@paulofaria @tannernelson I fixed some issues.  When this is merged and released with next tag then I'm gonna resolve conflicts for https://github.com/open-swift/S4/pull/52.
Then the test will be passed.


Feature Request: Ability to override DNS TTL

I'm currently using Kong with Docker Swarm + the internal docker engine DNS server.
Docker Swarm serves out DNS records with a TTL of 600 and currently offers no way of configuring otherwise. This causes issues when Docker services are updated because they get new IPs within their docker network and if the TTL hasn't expired, the cached entry is used ... causing requests to fail since the service is no longer listening at that location.
It would be great if Docker Swarm offered configurable TTLs but it would also be really nice if there was the ability to override how long a DNS record is valid for, ignoring what is sent by the DNS server.
This feature would be very similar to the NGINX valid option that you can add when configuring a resolver.

So a setting that would imply a maximum TTL? (any received TTL being longer, would be set to the maximum)
Or overriding the TTL always?

Overriding the TTL always
On Thu, Jul 12, 2018 at 8:37 PM Thijs Schreijer<EMAIL_ADDRESS>wrote:

So a setting that would imply a maximum TTL? (any received TTL being
longer, would be set to the maximum)
Or overriding the TTL always?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Kong/lua-resty-dns-client/issues/48#issuecomment-404708076,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABG4GBKKNBfWDhO5PXdubkwbIjxWLFtLks5uGAgHgaJpZM4VLWWV
.

This is indeed similar to the valid option of the Nginx resolver for which I advocated its implementation around last year or so.

Is this something that could be implemented in the plugin framework? I'm not very experienced with Lua but based on what I was reading here: https://github.com/Kong/lua-resty-dns-client/blob/master/src/resty/dns/client.lua ... it didn't look like it. I wanted to confirm though.

no, it cnnot be implemented in the Kong plugin framework, it must be implemented in this library first. And then add the extra config property to Kong.


Need help on interpreting the output of TF-Slim example(train_image_classifier & eval_image_classifier.py)

I have succusfully run TF-Slim example with inception model as given on tensor flow on spark repositotry.
Below are few points I would like to mention.
a) created small subset of 25 images for each of the classes from huge flowers dataset.
I am not worried about about model accuracy, just want to run the example in short time to see the reults.
b) first i ran train_image_classifier.py with inception model and flower dataset and script generated output as per
--train_dir hdfs://default/user/${USER}/slim_train
c) then I ran eval_image_classifier.py as per
--checkpoint_path hdfs://default/user/${USER}/slim_train
In my case both scripts ran fine without any error and got the outputs generated in given path.
Here I have couple of doubts/questions,
1- How to validate the outputs of above scripts ?
I was expecting to see some information on tensorboard, graph,summary events so I ran tensorboard pointing to logdir to
both the outputs but tensorboard doesn't display any information
2- My assumption is eval_image_classifier.py is used to make predictions on new dataset.
is it correct or it just evaluates the model and provides accuracy and for prediction we have to write our own code ?
whatever the case is I need to know how to get meaningfull information from the output about evaluation/predictions.
In my case output of eval_image_classifier has two files which are not text files,
events.out.tfevents.1508841834.tensorflow6
events.out.tfevents.1508841868.tensorflow5
Any help is highly appreciated.

@deepak-2017 that example (like many others) has an "eval" job which only tries to evaluate the overall model accuracy.  From that code, you can see that the raw 'predictions' are being calculated, so you'll just need to add some code to extract/use them as you need.
As for tensorboard, it's mostly for training time only.  In TFoS, you will need to supply the --tensorboard argument and then look in the driver logs for the URL to use.

Thanks for the response.
I had already tried with -- tensorboard option but nowhere in driver logs,i find information about url.
with or with out the option,logs are the same.
Second question is still the same as in my first post.I am not getting how to see the accuracy after running eval script.output of the script contains two event files, events.out.tfevents.1508841834.tensorflow6
events.out.tfevents.1508841868.tensorflow5
And in logs also, I don't see accuracy printed.
here are my worker logs snippet.
INFO:tensorflow:Evaluating hdfs://xtrserv:8020/user/tflow/slim_train_test1/model.ckpt-2
2017-10-25 03:28:16,399 INFO (MainThread-6197) Evaluating hdfs://xtrserv:8020/user/tflow/slim_train_test1/model.ckpt-2
INFO:tensorflow:Starting evaluation at 2017-10-25-03:28:17
2017-10-25 03:28:17,133 INFO (MainThread-6197) Starting evaluation at 2017-10-25-03:28:17
INFO:tensorflow:Restoring parameters from hdfs://xtrserv:8020/user/tflow/slim_train_test1/model.ckpt-2
2017-10-25 03:28:17,536 INFO (MainThread-6197) Restoring parameters from hdfs://xtrserv:8020/user/tflow/slim_train_test1/model.ckpt-2
INFO:tensorflow:Evaluation [1/4]
2017-10-25 03:29:14,880 INFO (MainThread-6197) Evaluation [1/4]
INFO:tensorflow:Evaluation [2/4]
2017-10-25 03:29:40,637 INFO (MainThread-6197) Evaluation [2/4]
INFO:tensorflow:Evaluation [3/4]
2017-10-25 03:30:05,868 INFO (MainThread-6197) Evaluation [3/4]
INFO:tensorflow:Evaluation [4/4]
2017-10-25 03:30:31,185 INFO (MainThread-6197) Evaluation [4/4]
2017-10-25 03:30:31.210033: I tensorflow/core/kernels/logging_ops.cc:79] eval/Recall_5[1]
2017-10-25 03:30:31.210137: I tensorflow/core/kernels/logging_ops.cc:79] eval/Accuracy[0]
INFO:tensorflow:Finished evaluation at 2017-10-25-03:30:32
2017-10-25 03:30:32,649 INFO (MainThread-6197) Finished evaluation at 2017-10-25-03:30:32
2017-10-25 03:30:32,650 INFO (MainThread-6197) Finished TensorFlow worker:0 on cluster node 0
Plz help me in figuring out the model accuracy after running the script.

If you supply the --tensorboard option, you should see something like the following in your driver's stdout logs:
2017-10-25 16:49:55,815 INFO (MainThread-51289) ========================================================================================
2017-10-25 16:49:55,815 INFO (MainThread-51289)
2017-10-25 16:49:55,815 INFO (MainThread-51289) TensorBoard running at:       http://<IP_ADDRESS>:36239
2017-10-25 16:49:55,815 INFO (MainThread-51289)
2017-10-25 16:49:55,815 INFO (MainThread-51289) ========================================================================================

As for the model accuracy, I think it's actually there in your logs (but maybe "nonsensical" due to the fact that you're training on a tiny dataset).  For example, here's what I see running the eval_image_classifier.py after training the inception_v3 model for 1000 steps:
2017-10-25 16:47:02,360 INFO (MainThread-7075) Evaluation [500/500]
2017-10-25 16:47:02.406177: I tensorflow/core/kernels/logging_ops.cc:79] eval/Accuracy[0.001]
2017-10-25 16:47:02.406190: I tensorflow/core/kernels/logging_ops.cc:79] eval/Recall_5[0.00502]
INFO:tensorflow:Finished evaluation at 2017-10-25-16:47:02

Thanks @leewyang
Got it.

There is one more question around summaries/metrics being logged for training script(train_image_classifier.py).
In the logs and on tensorbord graphs, I could  see only the values of loss/total loss  but no information logged for accuracy.
I am wondering that why that's not been included in training script because how well your model has performed is decided by it's accuracy.
My requirement is to extract accuracy info during model training and I am not sure how to do it.
Could you plz help me.

@deepak-2017 Again, this is because we were just converting code from an existing example, which used a separate eval_image_classifier.py job to evaluate the job in parallel with the training job.  Unfortunately, that evaluation job is currently uses slim.evaluation.evaluate_once, so it only provides a single "point-in-time" accuracy/loss number.  You MIGHT be able to convert it to use slim.evalution.evalute_loop, and configure TFoS to run/use TensorBoard, since the evaluation job does seem to write the metrics as tf.summary events already, although I'm not sure if you can grab the --steps for your x-axis.  (Worse case, I think you can use wall-clock).


feat: show battery on bluetooth device list

For connected devices only

Very keen for this. Would be awesome to monitor device battery levels instead of just the system!

How do we handle scenarios where battery is not available? I believe not all devices provide a battery status.

Let me handle the scenerio too

Is null a response that we get back? I have a bluetooth device which doesn't have battery levels but it's now showing null... Rather it shows battery-level: 0 when no battery is available or battery-level: 1 when a battery can be fetched. Could you verify with devices if you get the same thing between ones with battery status and those without?

I dont have a device without a battery so cannot test this properly.

I would like it to have a toggle option.
Additionally, I think we should use a Widget.Separator and set the spacing between the status and the battery separator via margins. I've used spaces in the past and based on the font you're using it can be very inconsistent.

Let's also separate the battery icon and the percentage into their own Widget.Labels. Reason is that we can then assign a txt-icon class to the battery icon (txt-icon is built to properly align nerdfont icons) to center/align it properly.

@Jas-SinghFSU Added a dedicated settings menu for bluetooth module. From there user can toggle the bluetooth battery as well as configure the icon to use

I dont have a device without a battery so cannot test this properly. @Jas-SinghFSU if you have a device perhaps you can test and tell me the op. I will adjust the code accordingly
Also do we want it to have a toggle option?

I will leave the testing to you for device without battery as I cannot test it out due to this

Sounds good, thank you! I'll try to get to it tomorrow.

All my devices have the battery percentage showable, else I'd test it :(. Might just be up to you sorry @Jas-SinghFSU.


(feat) add windows ci: appveyor.yml

Unable to create new account
https://github.com/appveyor/ci/issues/2001


Please add information on how to upgrade from v1

The release notes list changes but not API changes. I wanted to upgrade from v1 and noticed a lot of compiling issues I have no idea how to fix them so I get the same result as before:
DataTarget.AttachToProcess => AttachFlag is gone, as well as the timeout parameter. Why? And what should I use now when I was previously using AttachFlag.Passive?
ClrInfo.LocalMatchingDac => is gone. What's the new API for this?
ClrRuntime.Flush => is gone. What's the new API for this?
ClrRuntime.GetMethodByAddress => is gone. What's the new API for this?
Please update the docs and list up all API changes and alternatives when upgrading from v1.

I was not able to find any docs on migrating from v1 to v2 but I was able to migrate BenchmarkDotNet in https://github.com/dotnet/BenchmarkDotNet/pull/2040.
Short summary:

DataTarget does not support timeout anymore, the AttachFlag enum has been replaced with boolean:

using (var dataTarget = DataTarget.AttachToProcess(
    settings.ProcessId,
-    (uint)TimeSpan.FromSeconds(5).TotalMilliseconds,
-    AttachFlag.Passive))
+    suspend: false))

There is no need to call dataTarget.DataReader.Flush(); to workaround https://github.com/microsoft/clrmd/issues/303 as the bug has been solved.
Runtime.Heap does not expose GetTypeByName anymore, instead we need to iterate over the modules and use module.GetTypeByName

+ var typeWithBenchmark = state.Runtime.EnumerateModules().Select(module => module.GetTypeByName(settings.TypeName)).First(type => type != null);
- var typeWithBenchmark = state.Runtime.Heap.GetTypeByName(settings.TypeName);

We can still configure symbol path, but in a way more easy way:

private static void ConfigureSymbols(DataTarget dataTarget)
{
-    // code copied from https://github.com/Microsoft/clrmd/issues/34#issuecomment-161926535
-    var symbols = dataTarget.DebuggerInterface as IDebugSymbols;
-    symbols?.SetSymbolPath("http://msdl.microsoft.com/download/symbols");
-    var control = dataTarget.DebuggerInterface as IDebugControl;
-    control?.Execute(DEBUG_OUTCTL.NOT_LOGGED, ".reload", DEBUG_EXECUTE.NOT_LOGGED);
+    dataTarget.SetSymbolPath("http://msdl.microsoft.com/download/symbols");
}

Few properties like ILOffsetMap are no longer using an array, but ImmutableArray instead. Since ImmutableArray is a struct all null checks emit a warning.

method.GetFullSignature() has been replaced with method.Signature property

Runtime.DataTarget.ReadProcessMemory has been replaced with Runtime.DataTarget.DataReader.Read

- Runtime.DataTarget.ReadProcessMemory(startAddress, code, code.Length, out int bytesRead)
+ bytesRead = Runtime.DataTarget.DataReader.Read(startAddress, code)

Runtime.PointerSize has been replaced with Runtime.DataTarget.DataReader.PointerSize
runtime.GetMethodTableName has been replaced with runtime.DacLibrary.SOSDacInterface.GetMethodTableName
runtime.GetMethodByAddress has been replaced with runtime.GetMethodByInstructionPointer
PdbReader type no longer exists, but we can use SymbolReader from TraceEvent to read the symbols

We need to create SymbolReader and configure it to use MS symbol server:
SymbolReader symbolReader = new SymbolReader(TextWriter.Null) { SymbolPath = SymbolPath.MicrosoftSymbolServerPath }

PdbInfo exposes Path property which can be used to open symbol file (there is no need to use Runtime.DataTarget.SymbolLocator):
+ reader = symbolReader.OpenSymbolFile(pdbPath);
- reader = new PdbReader(pdbPath);

There were few other things I had to do to be able to repalce SymbolReader, you can take a look at this source file to get the overall idea


unit tests with mock i2c

Adds tests over a mock i2c connection.
The tests are factored into common tests for any transport, plus transport-specific tests for mocked i2c and mocked serial.
DRY's up the python commands in github actions by reusing the Makefile.

@m-mcgowan I'm having a hard time seeing what has actually changed in this pr so I am inclined to reject it until it's either simplified or broken into multiple PRs. Rather than taking on a giant refactor of how the tests work, why not add the I2C mocks and tests and we can worry about a refactor later, if at all?

I didn't think it was that big a refactor. All I did was factor out the notecard connection from the tests by moving them in to a base class which calls get_port() to retrieve the port with an expected response, and this is used by two subclasses, one for i2c and one for serial that defines get_port().
It's not presently building on github actions because that's a linux environment and the notecard communications code has the I2C setup hard-coded for to be different for that. It worked on my machine because I'm on OSX - I need to change the logic to also use the same in a test environment so that it is consistent regardless of runtime platform.
When we have HIL testing for this, all we have to do is define another class that uses a real I2C/serial connection running the same tests.

 mock tests for note-python and I2C

I didn't build this specifically to target HIL, I just wanted to factor out the communications channel, which is orthogonal to most of the tests (i.e. the tests are comms-neutral - they don't care about what channel they are using.) It just happens that with that done, adding HIL later will be easier because we can reuse the same mechanism.
We want a simple way to run all the comms-neutral tests over both mocked serial and i2c, and IMHO this was the simplest way to do it without causing maintenance headaches. The only other way I can think of is to duplicate (copy and paste) all the existing serial tests, and rework them to use i2c, but then we have a lot of duplication of code, which will only become worse when we add more comms channels to the test suite.

My apologies @m-mcgowan, the combination of an impenetrable diff and mention of HiL caused me to (wrongly) assume that more was going on here than mocking I2C for testing.

No problem! I was going to suggest you view the files side-by-side because then it's much easier to see the diff than what github gives you


airline/lightline support

The title says it all. Iceberg is an amazing theme, it would be more complete if it supported airline and lightline plugins which are quite popular among vim users.
You don't even have to start from scratch: apparently, neodark theme already has themes for these plugins that uses colors set by the vim colorscheme. I know one could use it from neodark, but it would have been more convenient if the airline/lightline themes were included.

Done! Thank you for the advice and helpful information!


[Java support] Exception in command handler results in malformed gRPC response

If an exception is thrown in a command handler of a user function written in Java (including FailInvoked thrown by CommandContext.fail(), the caller (gRPC client) does not receive a gRPC error. Instead the gRPC response seems to be truncated.
Output from grpcurl client:
Error invoking method "com.example.inventory.ProductInventory/ReserveProduct": grpc call for "com.example.inventory.ProductInventory.ReserveProduct" failed: EOF

@jroper @pvlugter Any thoughts here?

I guess it's just a bug.

Discussed at the last contributor's call. May be a regression from changes at some point. Would be useful to have it covered by tests...


Fixes to Master

I encountered two issues when attempting to get the master branch of Batfish to run on example test rigs. These commits fix the problems.

Hi there! The master branch is FAR behind the current development branch. As such, I am not going to apply any external pull requests at this time.


Run randomized testing and benchmarks in Azure

Summary
Azure pipelines gives us much faster node start up time and much more
capacity than travis, so we are moving jobs over to azure to speed up
our CI time. This commit moves the randomized testing and comparison
benchmarks over to run in azure pipelines instead of in travis. After
this the only jobs left in travis are release wheel build automation
(see #3009) and the coverage job.
Details and comments
The coverage job will need extra setup because coveralls (the service we
use for coverage aggregation) doesn't natively support appveyor, so we'll
have to generate tokens and do extra manual setup before that migration
is possible.
Fixes #2921

So the jobs don't work yet, but this was actually a good test, the master azure check Qiskit.qiskit-terra passes. Which verifies the continueOnError job attribute works as expected, which is what we wanted. Having the benchmarks or randomized testing fail is not something we want to block

On looking at this in real time I'm not sure that we want to merge it. It looks like it holds up the check until the jobs finish running (even though they're no fatal). For the benchmark tests this adds a ~20 min job to the end of the run which make the jobs take ~40mins. I'm not sure we want to double our time for CI because of this.

I opened a feature request on azure pipelines for a fast_finish flag: https://developercommunity.visualstudio.com/idea/728163/fast-finish-flag-for-stage.html that's what we leveraged in travis to not block the CI return on the slow running jobs.

The feature request process for azure seems to be slow. Shall we close this PR in the meanwhile? We can track the status in #2921

Yeah I'm just going to close this now that we have a Travis subscription for all of Qiskit there isn't as much of a pressing need for this. If the need arises we can always reopen this


Ldap password

resolves #310

Description
Add variable for password in profile.
Pass password to connection.
Pass exceptions message to user if connection problem is.
Checklist

[v] I have signed the CLA
[v] I have run this code in development and it appears to resolve the stated issue
[ ] This PR includes tests, or tests are not required/relevant for this PR
[ ] I have updated the CHANGELOG.md and added information about my change to the "dbt-spark next" section.

@jtcohen6  I'm begging your pardon. Could you review the state of this merge? Could it be fixed or, maybe it is better to do a new merge fixing LDAP?

Sorry, we missed this a few months ago! The changes look straightforward, although ldap authentication isn't something we have automated testing for today.
@barloc It looks like the commits in this PR aren't properly associated with your GitHub user. Any chance you could update those, and sign the CLA, so that the CLA check passes? If so, we can queue this PR for review.

Would it be possible to set it up so that CUSTOM auth also accepts the password argument, and not just LDAP?

Would it be possible to set it up so that CUSTOM auth also accepts the password argument, and not just LDAP?

Nevermind, looks like this is addressed in #396.

Would it be possible to set it up so that CUSTOM auth also accepts the password argument, and not just LDAP?

Nevermind, looks like this is addressed in #396.

@nousot-cloud-guy Yes, since auth and password are both will be passed to hive.connect() as is, it will allow to use CUSTOM as a bonus.

@barloc feel free to disagree but it looks like @VShkaberda fixed the issue in this PR so closing this PR out as already implemented


nixos/github-runners: clean workDir as root

Purge contents of workDir as root to also allow the removal of files marked as read-only. It is easy to create read-only files in workDir, e.g., by copying files from the Nix store.
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Thanks team!


SOLR-9775 fixed NPEs

https://issues.apache.org/jira/browse/SOLR-9775
Description
This is brought over from https://github.com/apache/lucene-solr/pull/116/files
Solution
I feel like I am pulling a thread on a sweater.....  It feels odd that we have so much handling around nulls....   And that this whole thing needs some refactoring so there aren't nulls.
In fact, there is kind of a comment about this in QueryResultKey.unorderedCompare():
    // SOLR-5618: if we had a guarantee that the lists never contained any duplicates,
    // this logic could be a lot simpler
    //
    // (And of course: if the SolrIndexSearcher / QueryCommand was ever changed to
    // sort the filter query list, then this whole method could be eliminated).

It feels like if we simplified the types of objects passed into the QueryResultKey this would all be simpler.  It's literally just a key!
Tests
Existing tests, and then added some to demonstrate the NPE and then the fix...
Checklist
Please review the following and check all that apply:

[ ] I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
[ ] I have created a Jira issue and added the issue ID to my pull request title.
[ ] I have given Solr maintainers access to contribute to my PR branch. (optional but recommended)
[ ] I have developed this patch against the main branch.
[ ] I have run ./gradlew check.
[ ] I have added tests for my changes.
[ ] I have added documentation for the Reference Guide

Okay, look at my most recent tests...    While they pass, I am not sure I am actually making things better....

@risdenk that is sooo much better.   I guess I don't have the courage of my convictions to change an array to a List!

@cpoerschke thoughts on this PR?  I think it's ready to merge.

@kagan770 I wanted to flag that this old PR is looking ready to get merged if you want to review ;-).

offering a review and a 'looks good to me' (even if non binding)

@stillalex @risdenk I'm starting to look at various PR's that I've had open for a long time, and trying to get them merged or closed.
Are we still happy with this one?

@stillalex can you push up to this PR the changes you were thinking about?


Download from medici.tv as logged in user account

Checklist

[x] I'm asking a question
[x] I've looked through the README and FAQ for similar questions
[x ] I've searched the bugtracker for similar questions including closed ones

Question

I tried to download a video from medici.tv. The trailer video can be downloaded without any problem with youtube-dl. But I tried to download the whole video - so I logged in in my account.
With youtube-dl I gave -u USERNAME and -p PASSWORD as well. But there is still only the trailer video which is downloaded. It seems that trailer and whole video have the same URL.
Thanks.

It's a bit tricky to get the right URL.
Using developper options on Firefox, open the Network tab and refresh the page, then look for a vnd.apple.mpeg file type.
You'll find  4 of them, the first 2 are most likely the trailer, the 4th one has the right URL for the whole video.

Any updates on how to do this more simply?

To make it easier, simply copy and paste the medici tv video url to Allavsoft and click Download button


Add branch protection for app runtime platform WG repos

👋@stephanme/@rkoster - do you know when this would get merged/enforced so we can warn the WG of the actual timeline for the change going through?


Change corpora-api.yml to reflect corpora 2.0.0 changes

A few dataset attributes have been either added, changed, or removed, update our api yml to reflect those changes:
https://github.com/chanzuckerberg/corpora-data-portal/blob/44b98b49bd3a38bf19d9f73b9b75e14d2b4e0586/backend/config/corpora-api.yml#L688-L743
Details on the changes can be seen here: #1296
The frontend should also add the fields to the collection entity defined in src/common/queries/entities.ts

Collapsed this ticket onto #1296


Fix errors in test_latex_name_isnt_split for min environments

There's some error in the minimum environments (for example ubuntu-latest py37-bare-minimum) introduced in #5682 but was hidden in all other errors because of #5654:
 =========================== short test summary info ============================
ERROR xarray/tests/test_plot.py::test_latex_name_isnt_split - NameError: name...
= 7908 passed, 4163 skipped, 202 xfailed, 42 xpassed, 89 warnings, 1 error in 279.31s (0:04:39) =

But when looking further in the details it mentions a completely different function crashing because matplotlib isn't installed, which is true but the function doesn't test any mpl-specific things.
 ==================================== ERRORS ====================================
_______________ ERROR at teardown of test_latex_name_isnt_split ________________
[gw0] linux -- Python 3.7.10 /usr/share/miniconda/envs/xarray-tests/bin/python

    @pytest.fixture(scope="function", autouse=True)
    def test_all_figures_closed():
        """meta-test to ensure all figures are closed at the end of a test

        Notes:  Scope is kept to module (only invoke this function once per test
        module) else tests cannot be run in parallel (locally). Disadvantage: only
        catches one open figure per run. May still give a false positive if tests
        are run in parallel.
        """
        yield None

>       open_figs = len(plt.get_fignums())
E       NameError: name 'plt' is not defined

/home/runner/work/xarray/xarray/xarray/tests/test_plot.py:72: NameError

Might be something that pytest runs on every function. Not sure if there's a clean way of turning it off, but requiring matplotlib to be installed works of course.

[x] Passes pre-commit run --all-files

@dcherian Is this solution fine or do you have a better idea how to turn off test_all_figures_closed for this function?
CC @tomchor

On my end I think this idea is fine! Although I don't really understand why matplotlib would be needed for that test in the first place.
Sorry to have inadvertently introduced a failing test.

I only saw the sparse errors on your PR as well. It's rough when you can't trust the CI results.

Sorry, my mistake. I should've manually checked the min deps test too.

Can we skip this entire file if matplotlib is not installed? cc @keewis
If not we can just put that test in here:
https://github.com/pydata/xarray/blob/41099669c63c7dbb51b112fdec09dfae07379fe7/xarray/tests/test_plot.py#L166

Can we skip this entire file if matplotlib is not installed?

I would say we can because at the moment plotting requires matplotlib to be installed. That might change with the introduction of plotting backends (e.g. #3640), but then we'd probably have to refactor the tests, anyways.

I like the idea of moving this to test_label_from_attrs seems to be a lot of similar tests there.


Enable assigning an owner to navigation regions and links

Inspired by the concept of Shape owners in Godot Physics, this PR adds the ability to assign an object as the "owner" of a navigation region or link.  This will be used to return the NavigationRegion or NavigationLink node that an agent enters while following its path.
Right now, it's only possible to get the RID of the underlying region or link in the server, which makes the typical usage of a signal when entering a link or region rather unfriendly to the casual user.  The PR should enable those signals to return the nodes encountered on its way in a future PR, but I wanted to get this out early in order to get feedback on the approach.
Why use ObjectID?
I considered using pointers to the owning object, but I'm concerned about returning pointers to freed memory if the owning object is deleted without properly unregistering itself.  Using ObjectID protects against this by returning an ID that would no longer be valid, which ObjectDB can catch and return a null Object pointer.

CC @smix8 @lawnjelly

Thanks!


archivist:create is now broken for couchbase

Using couchbase 4.
archivist:
  vaults:
    volatileVault:
      type: couchbase
      config:
        create:
          adminUsername: admin
          adminPassword: hello
          bucketType: couchbase
          ramQuotaMB: 100
          replicaNumber: 0
          bucketType: memcached
        options:
          scheme: couchbase
          hosts:
            - localhost:8091
          bucket: volatile

When I call npm run archivist:create, I get the following error:
Failed to load MAGE application: CouchbaseError: Authentication failed.

It seems strange. Is your couchbase version 4.0.0? And what is the version of your node sdk?

The only reason I see why it could fail is that you have an username field in config.options


Consider using builtin importlib.metadata

Using pip show to collect package metadata only returns a select few metadata of a package.
Using importlib.metadata would allow accessing to all of them, such as Download-URL, Project-URL, and Maintainer. See https://packaging.python.org/en/latest/specifications/core-metadata/.
This should make the code slightly faster and more reliable.

Thanks for the suggestion.


Induction Matrix stopped working

Hi,
Before I updated to 8.1.9 my induction matrix was running fine, using Thermal Dynamics cryo-stabilised fluxducts between the fusion reactor and the matrix. I updated and now the induction ports are refusing input from the Thermal Dynamics fluxducts, both cryo-stabilised and resonant. I've also tried it with basic universal cable and Ender IO Ender Energy Conduits. The cables/conduits/ducts all show as being powered, but the Induction Matrix doesn't show any input, and the charge which was held in there is now gone.
Any ideas? What can I do to help with tracing this - logs etc.?

I can confirm this. Induction Matrix seems to retain it's information. When breaking a block and then replacing it makes it work again until the server reboots.

Strange, nothing was changed between the two versions that could cause this.  I'll do a bit more digging.

Ah, I think I found the issue- it has to do with a change in the multiblock algorithm I implemented.  I'll push out a quick fix and bump it to recommended :)

Build 265 has the fix.

That was quick! Updating asap. Thanks


Rev Identity dependency versions May 2020

ServiceBus test failures aren't due to this PR, they are happening elsewhere as well. Should be safe to disregard them here.


feat: ContactInformation

The type of this PR is: TYPE
SWA-133

This PR resolves []
Description

PR Checklist (tick all before merging)

[ ] I have included screenshots or videos to illustrate my changes, or I have not changed anything that impacts the UI.
[ ] I have tested my changes on iOS and Android.
[ ] I have added tests/stories for my changes, or my changes don't require testing/stories, or I have included a link to a separate Jira ticket covering the tests.
[ ] I have added a feature flag, or my changes don't require a feature flag. (How do I add one?)
[ ] I have documented any follow-up work that this PR will require, or it does not require any.
[ ] I have added an app state migration, or my changes do not require one. (What are migrations?)
[ ] I have added a changelog entry below or my changes do not require one.

To the reviewers 👀

[ ] I would like at least one of the reviewers to run this PR on the simulator or device.

Changelog updates
Changelog updates

Cross-platform user-facing changes

iOS user-facing changes

Android user-facing changes

Dev changes

Warnings

:warning:

❌ An error occurred while validating your changelog, please make sure you provided a valid changelog.

  Generated by :no_entry_sign: dangerJS against e1e67576abd7cdab47e4ca65f663a69ab7a3571c

@pvinis @AlicanAkyuz @iskounen @MounirDhahri
Hi all! :)
This Pull Request is now ready for review.
Please let us know if we should update something before merging something to main (behind a feature flag).
There are some open issues that you can find under "Conversation" that we are planning to resolve on a later stage.
Thanks :)


Enable security on websocket end points as well as http.

Noticed that the rest utils does not currently set up security on the websocket end points. This PR adds a puts the same auth on the websocket requests as was already on the http requests, and adds suitable tests.

@confluentinc It looks like @big-andy-coates just signed our Contributor License Agreement. :+1:
Always at your service,
clabot


Link in readme does not work

Meistertr,
The link „A long discription you can find here: (Link).“ (iOS, no jailbreak) does not work anymore. A description is needed.
Thanks

done


Add Curated Drops to profiles

Random idea 🚀
Once we launch profiles, it will be good to think about a way how to add Curated drops to the profiles of individual artists.
All Drops are curated under KodaDot Minter, we are missing a chance to have artists on the platform as users as they dont need to do anything with our platform to create a drop. It would be nice if we can push them to create their profiles and have an option to somehow link the drop in their profiles.
This will solve the whole artists identity issue as most of them would love to have their art linked to their names on KodaDot.

@exezbcz once you have more time pls can you look at this?


Viva Goals not in GCC

Can it be included in a note or something on the page that Viva Goal is not yet available for GCC, GCCH and DOD Clouds? Listed as NA here, and on all the internal Roadmaps it says TBD.
https://supportability.visualstudio.com/M365 Release Announcements/_wiki/wikis/M365-Product-Updates.wiki/573963/Viva-Goals

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 0e0ef188-7c51-588e-d5b6-94b847a625f9
Version Independent ID: 0e0ef188-7c51-588e-d5b6-94b847a625f9
Content: Introduction to Microsoft Viva Goals
Content Source: Viva/goals/intro-to-ms-viva-goals.md
Service: viva
Sub-service: viva-goals
GitHub Login: @RaSanders-MSFT
Microsoft Alias: rasanders

Hello @Matt-Mueller-MSFT How are you?
I am afraid I can not access the link you added here, can you include the exact link to the roadmap item you are talking about? I couldn't find the exact item that talks solely about Goals. Thank you!
About where to add that, probable here is the right article https://learn.microsoft.com/en-us/office365/servicedescriptions/microsoft-viva-service-description

Hello @Matt-Mueller-MSFT could you see my comment here? Thanks

@scanum closing, for now, it can be re-opened.


fix: fix previous release references

Description

Currently the previous pointer is not available under the unreleased group when running git cliff. Also when running git cliff --unreleased the preivous release is not correct and is pointing to the release before last, instead of the last tagged release.
This PR deals with this by doing the following. For tackling the first problem we simply check if we produced more then one release at the end of the loop and add the missing pointers to the previous release.
For solving the other problem we just adapt the checked_sub argument to take the difference into account and fix it on the unreleased scenario, when it must be 1 instead of 2.
Motivation and Context

Currently the previous pointer is not available under the unreleased group when running git cliff. Also when running git cliff --unreleased the preivous release is not correct and is pointing to the release before last, instead of the last tagged release. This PR deals with these problems.
This is particularly useful for example on a Keep A Changelog format where we need the correct previous pointers so that we can add the proper link references.
How Has This Been Tested?

I've added an extra fixture with a keep a changelog format example to test the scenario.
Screenshots / Logs (if applicable)
Types of Changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Documentation (no code change)
[ ] Refactor (refactoring production code)
[ ] Other

Checklist:

[x] My code follows the code style of this project.
[x] I have updated the documentation accordingly.
[x] I have formatted the code with rustfmt.
[x] I checked the lints with clippy.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.

Codecov Report
Patch coverage has no change and project coverage change: -0.30 :warning:

Comparison is base (ff7f47c) 44.32% compared to head (fbb605e) 44.01%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #200      +/-   ##
==========================================
- Coverage   44.32%   44.01%   -0.30%
==========================================
  Files          12       12
  Lines         580      584       +4
==========================================
  Hits          257      257
- Misses        323      327       +4

Flag
Coverage Δ

unit-tests
44.01% <0.00%> (-0.30%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

git-cliff/src/lib.rs
0.00% <0.00%> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

Oh btw, can you comment on this issue if you haven't already? #165


mGBA emulator update

Verification

[X] This issue's title and/or description do not reference a single formula e.g. brew install wget. If they do, open an issue at https://github.com/Homebrew/homebrew-core/issues/new/choose instead.

Provide a detailed description of the proposed feature
mGBA 0.10.3 has been released Jan 8 with a few bug fixes. brew's mGBA version is still at 0.10.2. It would be appreciated if the latest version could be featured on brew.

What is the motivation for the feature?
brew is by far the most comfortable way to get mgba on macs.
How will the feature be relevant to at least 90% of Homebrew users?
i doubt there are so much people using mGBA. but it would still be appreciated by mGBA users :)
What alternatives to the feature have been considered?
none

It would be appreciated if the latest version could be featured on brew.

You can follow the instructions at https://docs.brew.sh/How-To-Open-a-Homebrew-Pull-Request on how to update it.


Error during plugin dependency resolution

Hi,
I need to add JSON export to you plugin, but when I try to run scriped, I've the following error :

> scripted
Running sbt-dependency-graph / ignoreScalaLibrary
[info] Getting org.scala-sbt sbt 0.13.0 ...
[info] :: retrieving :: org.scala-sbt#boot-app
[info] 	confs: [default]
[info] 	43 artifacts copied, 0 already retrieved (12440kB/40ms)
[info] Getting Scala 2.10.2 (for sbt)...
[info] :: retrieving :: org.scala-sbt#boot-scala
[info] 	confs: [default]
[info] 	5 artifacts copied, 0 already retrieved (24390kB/19ms)
[info] [info] Loading project definition from /tmp/sbt_fd43f870/ignoreScalaLibrary/project
[info] [info] Updating {file:/tmp/sbt_fd43f870/ignoreScalaLibrary/project/}ignorescalalibrary-build...
[info] [info] Resolving net.virtual-void#sbt-dependency-graph;0.7.5-SNAPSHOT ...
[info] [warn] 	module not found: net.virtual-void#sbt-dependency-graph;0.7.5-SNAPSHOT
[info] [warn] ==== typesafe-ivy-releases: tried
[info] [warn]   http://repo.typesafe.com/typesafe/ivy-releases/net.virtual-void/sbt-dependency-graph/scala_2.10/sbt_0.13/0.7.5-SNAPSHOT/ivys/ivy.xml
[info] [warn] ==== sbt-plugin-releases: tried
[info] [warn]   http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/net.virtual-void/sbt-dependency-graph/scala_2.10/sbt_0.13/0.7.5-SNAPSHOT/ivys/ivy.xml
[info] [warn] ==== local: tried
[info] [warn]   /home/pogam-_g/.ivy2/local/net.virtual-void/sbt-dependency-graph/scala_2.10/sbt_0.13/0.7.5-SNAPSHOT/ivys/ivy.xml
[info] [warn] ==== public: tried
[info] [warn]   http://repo1.maven.org/maven2/net/virtual-void/sbt-dependency-graph_2.10_0.13/0.7.5-SNAPSHOT/sbt-dependency-graph-0.7.5-SNAPSHOT.pom
[info] [info] Resolving org.scala-lang#scala-library;2.10.2 ...
[info] Resolving org.scala-sbt#sbt;0.13.0 ...
[info] Resolving org.scala-sbt#main;0.13.0 ...
[info] Resolving org.scala-sbt#actions;0.13.0 ...
[info] Resolving org.scala-sbt#classpath;0.13.0 ...
[info] Resolving org.scala-sbt#launcher-interface;0.13.0 ...
[info] Resolving org.scala-sbt#interface;0.13.0 ...
[info] Resolving org.scala-sbt#io;0.13.0 ...
[info] Resolving org.scala-sbt#control;0.13.0 ...
[info] Resolving org.scala-lang#scala-compiler;2.10.2 ...
[info] Resolving org.scala-lang#scala-reflect;2.10.2 ...
[info] Resolving org.scala-sbt#completion;0.13.0 ...
[info] Resolving org.scala-sbt#collections;0.13.0 ...
[info] Resolving jline#jline;2.11 ...
[info] Resolving org.scala-sbt#api;0.13.0 ...
[info] Resolving org.scala-sbt#compiler-integration;0.13.0 ...
[info] Resolving org.scala-sbt#incremental-compiler;0.13.0 ...
[info] Resolving org.scala-sbt#logging;0.13.0 ...
[info] Resolving org.scala-sbt#process;0.13.0 ...
[info] Resolving org.scala-sbt#relation;0.13.0 ...
[info] Resolving org.scala-sbt#compile;0.13.0 ...
[info] Resolving org.scala-sbt#persist;0.13.0 ...
[info] Resolving org.scala-tools.sbinary#sbinary_2.10;0.4.2 ...
[info] Resolving org.scala-sbt#classfile;0.13.0 ...
[info] Resolving org.scala-sbt#compiler-ivy-integration;0.13.0 ...
[info] Resolving org.scala-sbt#ivy;0.13.0 ...
[info] Resolving org.scala-sbt#cross;0.13.0 ...
[info] Resolving org.apache.ivy#ivy;2.3.0-rc1 ...
[info] Resolving com.jcraft#jsch;0.1.46 ...
[info] Resolving org.scala-sbt#run;0.13.0 ...
[info] Resolving org.scala-sbt#task-system;0.13.0 ...
[info] Resolving org.scala-sbt#tasks;0.13.0 ...
[info] Resolving org.scala-sbt#tracking;0.13.0 ...
[info] Resolving org.scala-sbt#cache;0.13.0 ...
[info] Resolving org.scala-sbt#testing;0.13.0 ...
[info] Resolving org.scala-sbt#test-agent;0.13.0 ...
[info] Resolving org.scala-sbt#test-interface;1.0 ...
[info] Resolving org.scala-sbt#main-settings;0.13.0 ...
[info] Resolving org.scala-sbt#apply-macro;0.13.0 ...
[info] Resolving org.scala-sbt#command;0.13.0 ...
[info] Resolving org.scala-sbt#compiler-interface;0.13.0 ...
[info] Resolving org.scala-sbt#precompiled-2_8_2;0.13.0 ...
[info] Resolving org.scala-sbt#precompiled-2_9_2;0.13.0 ...
[info] Resolving org.scala-sbt#precompiled-2_9_3;0.13.0 ...
[info] Resolving org.scala-lang#jline;2.10.2 ...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] [warn] 	::::::::::::::::::::::::::::::::::::::::::::::
[info] [warn] 	::          UNRESOLVED DEPENDENCIES         ::
[info] [warn] 	::::::::::::::::::::::::::::::::::::::::::::::
[info] [warn] 	:: net.virtual-void#sbt-dependency-graph;0.7.5-SNAPSHOT: not found
[info] [warn] 	::::::::::::::::::::::::::::::::::::::::::::::
[info] [warn]
[info] [warn] 	Note: Some unresolved dependencies have extra attributes.  Check that these dependencies exist with the requested attributes.
[info] [warn] 		net.virtual-void:sbt-dependency-graph:0.7.5-SNAPSHOT (scalaVersion=2.10, sbtVersion=0.13)
[info] [warn]
[info] sbt.ResolveException: unresolved dependency: net.virtual-void#sbt-dependency-graph;0.7.5-SNAPSHOT: not found
[info] 	at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:213)
[info] 	at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:122)
[info] 	at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:121)
[info] 	at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:116)
[info] 	at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:116)
[info] 	at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:104)
[info] 	at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:51)
[info] 	at sbt.IvySbt$$anon$3.call(Ivy.scala:60)
[info] 	at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:98)
[info] 	at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:81)
[info] 	at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:102)
[info] 	at xsbt.boot.Using$.withResource(Using.scala:11)
[info] 	at xsbt.boot.Using$.apply(Using.scala:10)
[info] 	at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:62)
[info] 	at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:52)
[info] 	at xsbt.boot.Locks$.apply0(Locks.scala:31)
[info] 	at xsbt.boot.Locks$.apply(Locks.scala:28)
[info] 	at sbt.IvySbt.withDefaultLogger(Ivy.scala:60)
[info] 	at sbt.IvySbt.withIvy(Ivy.scala:101)
[info] 	at sbt.IvySbt.withIvy(Ivy.scala:97)
[info] 	at sbt.IvySbt$Module.withModule(Ivy.scala:116)
[info] 	at sbt.IvyActions$.update(IvyActions.scala:121)
[info] 	at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1144)
[info] 	at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1142)
[info] 	at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$73.apply(Defaults.scala:1165)
[info] 	at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$73.apply(Defaults.scala:1163)
[info] 	at sbt.Tracked$$anonfun$lastOutput$1.apply(Tracked.scala:35)
[info] 	at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1167)
[info] 	at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1162)
[info] 	at sbt.Tracked$$anonfun$inputChanged$1.apply(Tracked.scala:45)
[info] 	at sbt.Classpaths$.cachedUpdate(Defaults.scala:1170)
[info] 	at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1135)
[info] 	at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1113)
[info] 	at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
[info] 	at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:42)
[info] 	at sbt.std.Transform$$anon$4.work(System.scala:64)
[info] 	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
[info] 	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
[info] 	at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:18)
[info] 	at sbt.Execute.work(Execute.scala:244)
[info] 	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
[info] 	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
[info] 	at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:160)
[info] 	at sbt.CompletionService$$anon$2.call(CompletionService.scala:30)
[info] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[info] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[info] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[info] 	at java.lang.Thread.run(Thread.java:745)
[info] [error] (*:update) sbt.ResolveException: unresolved dependency: net.virtual-void#sbt-dependency-graph;0.7.5-SNAPSHOT: not found
[info] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore?
[error] x sbt-dependency-graph / ignoreScalaLibrary
[trace] Stack trace suppressed: run last :scripted for the full output.
[error] (:scripted) Remote sbt initialization failed
[error] Total time: 7 s, completed Aug 24, 2015 4:33:35 PM

How can I solve it ? I'm sorry, I'm a totally Scala/SBT beginner (It's the first time I use them).
Other question, how can I run the plugin to test it on my computer ?
Thank you for your answer.

Hi @Kendos-Kenlen, try running publish-local in sbt first which will publish the artifact to your local ivy repository with the configured version which currently is 0.7.5-SNAPSHOT. You can then just follow the instructions about how to use the plugin replacing the released version with the snapshot version.
scripted also expects the current version to be available locally so running publish-local first should make it work.

Everything works, thank you ! :)

Nice :)

HI,
Just, I've a new question : is the plugin available for SBT 0.11 ?

@Kendos-Kenlen in which way was it unsuccessful?

And yes it should work :)

I'm trying to crawl the dependencies from this project :
But I've a dependency resolution error with SBT 0.11 :https://github.com/chrislewis/hazelbox

[info] Loading global plugins from /home/pogam-_g/.sbt/plugins
[info] Updating {file:/home/pogam-_g/.sbt/plugins/}default-607b65...
[warn]  module not found: net.virtual-void#sbt-dependency-graph;0.7.5
[warn] ==== typesafe-ivy-releases: tried
[warn]   http://repo.typesafe.com/typesafe/ivy-releases/net.virtual-void/sbt-dependency-graph/scala_2.9.1/sbt_0.11.0/0.7.5/ivys/ivy.xml
[warn]   -- artifact net.virtual-void#sbt-dependency-graph;0.7.5!sbt-dependency-graph.jar:
[warn]   http://repo.typesafe.com/typesafe/ivy-releases/net.virtual-void/sbt-dependency-graph/scala_2.9.1/sbt_0.11.0/0.7.5/jars/sbt-dependency-graph.jar
[warn] ==== local: tried
[warn]   /home/pogam-_g/.ivy2/local/net.virtual-void/sbt-dependency-graph/scala_2.9.1/sbt_0.11.0/0.7.5/ivys/ivy.xml
[warn]   -- artifact net.virtual-void#sbt-dependency-graph;0.7.5!sbt-dependency-graph.jar:
[warn]   /home/pogam-_g/.ivy2/local/net.virtual-void/sbt-dependency-graph/scala_2.9.1/sbt_0.11.0/0.7.5/jars/sbt-dependency-graph.jar
[warn] ==== public: tried
[warn]   http://repo1.maven.org/maven2/net/virtual-void/sbt-dependency-graph_2.9.1_0.11.0/0.7.5/sbt-dependency-graph-0.7.5.pom
[warn]   -- artifact net.virtual-void#sbt-dependency-graph;0.7.5!sbt-dependency-graph.jar:
[warn]   http://repo1.maven.org/maven2/net/virtual-void/sbt-dependency-graph_2.9.1_0.11.0/0.7.5/sbt-dependency-graph-0.7.5.jar
[warn] ==== Scala-Tools Maven2 Repository: tried
[warn]   http://scala-tools.org/repo-releases/net/virtual-void/sbt-dependency-graph_2.9.1_0.11.0/0.7.5/sbt-dependency-graph-0.7.5.pom
[warn]   -- artifact net.virtual-void#sbt-dependency-graph;0.7.5!sbt-dependency-graph.jar:
[warn]   http://scala-tools.org/repo-releases/net/virtual-void/sbt-dependency-graph_2.9.1_0.11.0/0.7.5/sbt-dependency-graph-0.7.5.jar
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: net.virtual-void#sbt-dependency-graph;0.7.5: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn]  Note: Some unresolved dependencies have extra attributes.  Check that these dependencies exist with the requested attributes.
[warn]          net.virtual-void:sbt-dependency-graph:0.7.5 (sbtVersion=0.11.0, scalaVersion=2.9.1)
[warn]
[error] {file:/home/pogam-_g/.sbt/plugins/}default-607b65/*:update: sbt.ResolveException: unresolved dependency: net.virtual-void#sbt-dependency-graph;0.7.5: not found
Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? q

No problem with SBT 13.
Maybe I don't require the plugin with the good way. I followed the README but instead of creating plugin in ~/.sbt/0.13, I created them in ~/.sbt (SBT 0.11 look to not care about ~/.sbt/0.11).

If you look here, it seems that only sbt 0.11.0 specifically is not supported. Would it be possible to update to sbt 0.11.3?

Owh I see ... Maybe it could be a good idea to specify it in the readme.
Thank you !


[neuralnet] Update model_file to save_path in neuralnet properties

Update model_file to save_path in neuralnet properties
Resolves #488
Self evaluation:

Build test: [x]Passed [ ]Failed [ ]Skipped
Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor<EMAIL_ADDRESS>

:memo: TAOS-CI Version: 1.4.20191203. Thank you for submitting PR #489. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://nnsuite.mooo.com/.


How to have the weights saved as .safetensors??

Reminder

[X] I have read the README and searched the existing issues.

Reproduction
I was able to fine-tune mistral 7x8B, however, when I tried to run it on multiple GPU, it didn't work. Here is my code:
from transformers import (
AutoModelForCausalLM,
AutoTokenizer,
pipeline
)
import torch
#device = torch.cuda.set_device(0) #use the first gpu
#model with non_nlp input
model_path = "saves/Mixtral-8x7B/lora/trained_model_4k_fen_input"
model_mixtral = AutoModelForCausalLM.from_pretrained(model_path, device_map="auto", low_cpu_mem_usage=True)
tokenizer_mixtral = AutoTokenizer.from_pretrained(model_path)
print(model_mixtral.hf_device_map)
fen_state = "2b1k1n1/r2p1pbr/nq5p/1pp1pPp1/2NPP1PP/2P4R/RB2B2K/6N1 w - - 0 26"
text = """
Instruction:
You are a chess grandmaster. you understand and are able to locate chess pieces on the board given its FEN state. Your task is to find the best next move in the chess game given the FEN state. You will describe the best next move by giving a piece starting square and ending square using algebraic notation describing both coordinates. Take extra care to only use legal and valid move.
The board is in the following position: """ + fen_state + """
What next move would produce the best centipawn improvement for the active player
Response:"""
inputs = tokenizer_mixtral(text, return_tensors="pt").to("cuda")
outputs = model_mixtral.generate(**inputs, max_new_tokens=10000)
print(tokenizer_mixtral.decode(outputs[0], skip_special_tokens=True))
And here is the error I have:
File "/home/knguyen/LLaMA-Factory/env/lib/python3.10/site-packages/accelerate/hooks.py", line 166, in new_forward
output = module._old_forward(*args, **kwargs)
File "/home/knguyen/LLaMA-Factory/env/lib/python3.10/site-packages/peft/tuners/lora/layer.py", line 509, in forward
result = result + lora_B(lora_A(dropout(x))) * scaling
File "/home/knguyen/LLaMA-Factory/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/knguyen/LLaMA-Factory/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/knguyen/LLaMA-Factory/env/lib/python3.10/site-packages/accelerate/hooks.py", line 161, in new_forward
args, kwargs = module._hf_hook.pre_forward(module, *args, **kwargs)
File "/home/knguyen/LLaMA-Factory/env/lib/python3.10/site-packages/accelerate/hooks.py", line 347, in pre_forward
set_module_tensor_to_device(
File "/home/knguyen/LLaMA-Factory/env/lib/python3.10/site-packages/accelerate/utils/modeling.py", line 443, in set_module_tensor_to_device
torch.cuda.empty_cache()
File "/home/knguyen/LLaMA-Factory/env/lib/python3.10/site-packages/torch/cuda/memory.py", line 162, in empty_cache
torch._C._cuda_emptyCache()
RuntimeError: CUDA error: unspecified launch failure
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
Expected behavior
No response
System Info
No response
Others
When I test the same way like that with original mistral 7x8b model from hugging face and it works, So I assume that  it is not mapping the weights to the GPUs because of the different way the model existed is saved. I wonder how can I convert the weights to .safetensors. or if I fine tune again, how can I have the weights saved as .safetensors???

Could you provide your training script? if deepspeed zero3 was used, did you enable the option to save the 16bit weights?

Here is my training script:
{
"train_batch_size": "auto",
"train_micro_batch_size_per_gpu": "auto",
"gradient_accumulation_steps": "auto",
"gradient_clipping": "auto",
"zero_allow_untested_optimizer": true,
"fp16": {
"enabled": "auto",
"loss_scale": 0,
"loss_scale_window": 1000,
"initial_scale_power": 16,
"hysteresis": 2,
"min_loss_scale": 1
},
"bf16": {
"enabled": "auto"
},
"zero_optimization": {
"stage": 3,
"overlap_comm": true,
"contiguous_gradients": true,
"sub_group_size": 1e9,
"reduce_bucket_size": "auto",
"stage3_prefetch_bucket_size": "auto",
"stage3_param_persistence_threshold": "auto",
"stage3_max_live_parameters": 1e9,
"stage3_max_reuse_distance": 1e9,
"stage3_gather_16bit_weights_on_model_save": true
}
}
and here is the command I use to run :
#!/bin/bash
deepspeed --num_gpus 3 src/train_bash.py
--deepspeed ds_config.json
--stage sft
--do_train
--model_name_or_path mistralai/Mixtral-8x7B-v0.1
--dataset Fen_data_nl_input_4k
--dataset_dir data
--template default
--finetuning_type lora
--output_dir saves/Mixtral-8x7B/lora/trained_model_4k_nl_input
--overwrite_cache
--overwrite_output_dir
--cutoff_len 1024
--preprocessing_num_workers 16
--per_device_train_batch_size 1
--per_device_eval_batch_size 1
--gradient_accumulation_steps 2
--lr_scheduler_type cosine
--logging_steps 10
--warmup_steps 20
--save_steps 100
--eval_steps 100
--evaluation_strategy steps
--learning_rate 5e-5
--num_train_epochs 3.0
--max_samples 6000
--val_size 0.1
--ddp_timeout 1800000
--plot_loss
--fp16
# deepspeed --num_gpus 3 src/train_bash.py \
# --deepspeed  ds_config.json\
# --ddp_timeout 180000000 \
# --stage sft \
# --do_train True \
# --model_name_or_path mistralai/Mixtral-8x7B-v0.1 \
# --finetuning_type lora \
# --template default \
# --dataset_dir data \
# --dataset Fen_chess \
# --cutoff_len 1024 \
# --learning_rate 5e-05 \
# --num_train_epochs 3.0 \
# --max_samples 100000 \
# --per_device_train_batch_size 26 \
# --gradient_accumulation_steps 8 \
# --lr_scheduler_type cosine \
# --max_grad_norm 1.0 \
# --logging_steps 5 \
# --save_steps 100 \
# --warmup_steps 0 \
# --optim adamw_torch \
# --report_to none \
# --output_dir saves/Mixtral-8x7B/full/train_2024-04-02-14-46-24 \
# --plot_loss True

Did you try using this example to infer the model? multiGPUs are supported by specifying CUDA_VISIBLE_DEVICES=0,1,2,3

So you mean, instead of use deep speed, I should try to use that example? if I use that, what type of file extension it produces? and may I know what's wrong with deepspeed because as I read, it support to train the model on multi GPU

You still need to use deepspeed to fine-tune the model on multi GPUs, it should produce adapter_model.safetensors
I think that you can try using llama factory to chat with the fine-tuned model

Thanks, btw, how can I use llama factory to chat with fine-tuned model?

You could check if there is a file named in the output path
Chat with fine-tuned model on Gradio UI can be done by https://github.com/hiyouga/LLaMA-Factory/blob/main/examples/inference/web_demo.sh

can you double check on everything is right on this?
#!/bin/bash
CUDA_VISIBLE_DEVICES=0 python src/web_demo.py
--model_name_or_path saves/Mixtral-8x7B/lora/trained_model_4k_fen_input
--adapter_name_or_path /saves/Mixtral-8x7B/lora/trained_model_4k_fen_input/adapter_model.safetensors
--template default
--finetuning_type lora

--model_name_or_path mistralai/Mixtral-8x7B-v0.1
--adapter_name_or_path saves/Mixtral-8x7B/lora/trained_model_4k_nl_input
--template default
--finetuning_type lora

Thanks a lot

No worries, thank anyway


Functional tests rely on stack and don't seem to work with new-build

After adding stack to the shell.nix dependencies and hie to the build-tool-depends I get:
Test suite func-test: RUNNING...
Warming up HIE cache...
func-test: fd:45: hGetLine: end of file
func-test: Language server unexpectedly terminated
Test suite func-test: FAIL

I did a bit of debugging on this last night: the hie instance invoked by cabal new-test :func-test isn’t able to find cabal-helper-wrapper. Oddly enough cabal new-run :func-test is.
The solution was to add cabal-helper:cabal-helper-wrapper to build-tool-depends for func-test, but then I ran into this issue with cabal https://github.com/haskell/cabal/issues/6011

hie does not longer use cabal-helper-wrapper and we are gonna to run the test suite with cabal soon: #1571


ERROR: the following packages/stacks could not have their rosdep keys resolved

When I try to build jaco-arm-pkgs I get the following error after running
rosdep install -y --from-paths . --ignore-src --rosdistro kinetic
ERROR: the following packages/stacks could not have their rosdep keys resolved
to system dependencies:
jaco_gazebo: Cannot locate rosdep definition for [gazebo_joint_control]
jaco_joints: Cannot locate rosdep definition for [joint_trajectory_execution]
jaco_on_table: Cannot locate rosdep definition for [common_sensors]
Can you help me with this?

I haven't used it with rosdep before, possibly need to add rosdistro entries. Just compile and if there are errors relating to missing dependencies, install them. If you install the packages as described in the wiki install instructions, it should be fine though.

Sorry but I'm not getting around to adding rosdistro entries and then maintaining them, the jaco packages are only to be used for tutorials etc. So I'm afraid this won't be fixed. To use jaco, you will have to add the jaco packages manually to your catkin workspace. Maybe some time in the future when I have more time (I'm praying for this day ;)) I will get around to adding the jaco packages too.


Make homeshich.sh position-independent

The current implementation of homeshick.sh contains a hard-coded path to the installation location of homeshick. This patch makes homeshick position-independent by computing the actual installation location at the time homeshick.sh is sourced (e.g., by .bashrc).
Users should never notice any difference if they install homeshick as described in the wiki. Hence, there are no specific tests to ensure the change works for arbitrary installation locations, but all existing tests show that it works for the default location.

The purpose of this change is to make packaging of homeshick possible without applying patches: [https://build.opensuse.org/package/show/home:oreinert/homeshick]

Sweet! It's great to see homeshick getting this level of attention :-)
If people are running fish or csh, they're going to have a bad time though. You didn't patch those files...

You're right about the csh and fish scripts - let me fix those.

OK, so I fixed homeshick.fish. However, I can't figure out a way to fix homeshick.csh, so I decided to leave it as-is. That should not be a problem, it will continue to work as expected for users who install homeshick by cloning.
For the RPM I will patch homeshick.csh to point at the proper installation location. That's OK because it isn't needed for running test cases.

Hence, there are no specific tests to ensure the change works for arbitrary installation locations, but all existing tests show that it works for the default location.

Well, they are also useful to ensure that I don't break your modifications in the future :-)
No matter, I'll find the time to go through it tonight, merge it and add some tests myself.

I'm having some reservations with this PR. Scripts that have to detect their own location is a bad smell, somehow - it all seems a bit too convoluted for my taste.
How about if we just define that an environment variable (e.g., HOMESHICK_DIR) defines the location of homeshick? The sourced scripts can define a default value for it, and homeshick itself can simply require that it's defined. (In a way, this is, sort of, what line 4 in bin/homeshick is all about, anyway.) That would make the things this PR tries to do much simpler.
I'm willing to code it up.

I agree about the smell, but to be honest, getting my code shipped as a package the [second time] (https://aur.archlinux.org/packages/homeshick-git/) lowered my standards a little :-)

The sourced scripts can define a default value for it, and homeshick itself can simply require that it's defined.

Why not keep the default in bin/homeshick instead? This way the executable also works when invoked directly (albeit without the cd command).
Also: You might want to ping the guys over at archlinux, not quite sure how they solved it, but this solution seems great.

So, I opened PR #138 as an alternative to this one. It's not quite complete yet, but it shows the direction I'd like it to go. Feel free to decline this PR if you agree #138 is a better approach.
It (also) keeps the default in bin/homeshick, as you suggested - that's a good idea.
The ArchLinux package does the same as my current package, it patches the scripts. Works fine, but it's still a crutch.

Completely agree :-)
Closing this one...


Highlight the location in a lobby that will be chosen for the event

https://github.com/sopra-fs23-group-07/sopra-fs23-group-07-client/commit/9469766d72c0043e82056b219fd6717e46090b8c


Add bloqs to convert classical values to quantum values for some dtypes

Adds functions which convert classical values to quantum registers using XGate()'s. Includes classical simulation testing for positive integers only.

Re: naming: these operations will initialize a constant k if the register is in the zero state. But otherwise it does a bitwise xor. Maybe something like XorInt XorUInt

Re: naming: these operations will initialize a constant k if the register is in the zero state. But otherwise it does a bitwise xor. Maybe something like XorInt XorUInt

Do you want me to add that Xor functionality for the classical simulation or is the current implementation sufficient?

Yeah, I think the classical simulation should match the actual behavior of the bloq. At the very minimum, you could check the input value and if it's anything other than zero, raise a NotImplementedError; but ideally you'd implement an xor.
If you want to mandate that the input is always zero, then you should avoid having an input. You can change the register to a RIGHT register. The decomposition can start with an allocation and then you can do your x gates. This would be redundant with IntState, though; so check that out first if you want to go that route.

Yeah, I think the classical simulation should match the actual behavior of the bloq. At the very minimum, you could check the input value and if it's anything other than zero, raise a NotImplementedError; but ideally you'd implement an xor.
If you want to mandate that the input is always zero, then you should avoid having an input. You can change the register to a RIGHT register. The decomposition can start with an allocation and then you can do your x gates. This would be redundant with IntState, though; so check that out first if you want to go that route.

Honestly I think IntState is probably better for this anyway. I just wasn't aware it already existed. I'll opt to use that instead.


chore(kubernetes): update ingress manifest to use not beta api version

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 24.451%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
3732

Relevant Lines:
16115

💛  - Coveralls


feat: Support for WAL cleanup of Chroma v 0.5.x

#16  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @tazarov and the rest of your teammates on  Graphite


toml-f: 0.4.1 -> 0.4.2

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/toml-f/toml-f/releases.
meta.description for toml-f is: TOML parser implementation for data serialization and deserialization in Fortran
meta.homepage for toml-f is: https://github.com/toml-f/toml-f
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0.4.2 with grep in /nix/store/jb2hrg0xxi9m05hllaz3p86h9ycpbw2a-toml-f-0.4.2
found 0.4.2 in filename of file in /nix/store/jb2hrg0xxi9m05hllaz3p86h9ycpbw2a-toml-f-0.4.2

Rebuild report (if merged into master) (click to expand)

5 total rebuild path(s)

5 package rebuild(s)

First fifty rebuilds by attrpath
python310Packages.tblite
python311Packages.tblite
simple-dftd3
tblite
toml-f

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/jb2hrg0xxi9m05hllaz3p86h9ycpbw2a-toml-f-0.4.2 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A toml-f https://github.com/r-ryantm/nixpkgs/archive/b07ca0897e89060aa73367bebe4ff0f078cd19f0.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/b07ca0897e89060aa73367bebe4ff0f078cd19f0#toml-f

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/jb2hrg0xxi9m05hllaz3p86h9ycpbw2a-toml-f-0.4.2
ls -la /nix/store/jb2hrg0xxi9m05hllaz3p86h9ycpbw2a-toml-f-0.4.2/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

5 packages built:

python310Packages.tblite
python311Packages.tblite
simple-dftd3
tblite
toml-f

Maintainer pings
cc @sheepforce for testing.

@GrahamcOfBorg build simple-dftd3 python3Packages.tblite


Switch rustc_codegen_spirv to static lib

Was set as a dylib but not really sure why. Having it as a static library enables dependencies, like spriv-builder, to start building directly after the frontend section of the build of the crate is done. Rather than having to wait for the codegen as well.
Saw clean ´spirv-builder` build times go from 46.3 -> 45.8 seconds with this. Have seen bigger benefits when combined with other optimizations as well.
Part of #858

Oh interesting CI failure, so we do need this as dylib for the tests?

That must have indeed been the case, as can't reproduce this building without being a dylib on a separate machine after properly clean build.

I did do a cargo clean before hand and examples did work

Heh, I don't trust tools to do this properly (for whatever reason), and I rm -rf target/ instead.


Enable test on Safari

Description
Adds missing test on Safari for our suite of visual tests.
Types of changes

[x] Chore (tooling change or documentation change)
[ ] Refactor (non-breaking change which maintains existing functionality)
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.

@jshjohnson Please review the new Safari snapshot (GitHub allows to see side-by-side, so, it's handy) - it has some visual changes regarding placeholders, not sure is as supposed to be.

I think the comparison is fine but I have noticed since the 'ch' unit PR, that placeholder widths are a lot wider than they need to be:

@jshjohnson Yeah, I know and wrote in original PR (including a link to Chris Mayer writing about it), that ch is the width of 0, so, on average it's about 20% wider than normal text. We can tune that in separate PR.
But here I've mostly referred these cases where placeholder disappears at all - I've seen your recent changes to placeholder handling and not sure whether such behavior is desired or it's a regression?

Anyway, I think it's better to merge like this (as it reflects current master state), so then any new changes will come with adjusted snapshots and easier to understand.

The placeholder change removed the ability to pass a placeholder via config so that's expected 👍

Hey - i'm not sure if it's related to this PR but visual checks seem to behaving strangely in this PR: https://github.com/jshjohnson/Choices/pull/739 which was opened after merging this one

I'm not sure neither, changes here are pretty minimal. I would prefer to wait a day to see if that's a Github Actions problem...


Link directly to BUILD.md in the readme

Minor quality-of-life improvement for first-time visitors

Thanks!


Competition entry form

Overall, this is excellent work! If you have time, consider adding some unit tests for the entry form. Nice one! :)

Nice improvements! I think that it would be good to add some unit tests for the CompetitionEntryForm.tsx file. Unless @KrisBoxAND spots anything that I might have missed. Great work! :)

Thanks @sxpydo
I need to wait for other tickets to finish to add tests, as they are altering and adding things to this file.
Once all of the components fit together, there will be tests for the functionality


train_expansion_keras_model yields error in dimensions of logits and labels

Hello,
I have a large database of over 92k reaction templates and I'm trying to retrain the expansion model. I ran the pre-process expansion step and moved to train, but am encountering the following error:
InvalidArgumentError                      Traceback (most recent call last)
/tmp/ipykernel_5701/843677716.py in <module>
      9 )
     10
---> 11 train_expansion_keras_model(config)
     12
     13 # def main(optional_args: Optional[Sequence[str]] = None) -> None:

~/anaconda3/envs/retro/lib/python3.7/site-packages/aizynthfinder/training/keras_models.py in train_expansion_keras_model(config)
    197         "categorical_crossentropy",
    198         ["accuracy", "top_k_categorical_accuracy", top10_acc, top50_acc],
--> 199         config,
    200     )
    201

~/anaconda3/envs/retro/lib/python3.7/site-packages/aizynthfinder/training/keras_models.py in _train_keras_model(model, train_seq, valid_seq, loss, metrics, config)
    166         use_multiprocessing=False,
    167         shuffle=True,
--> 168         initial_epoch=0,
    169     )
    170

~/anaconda3/envs/retro/lib/python3.7/site-packages/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, validation_freq, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
   2221         use_multiprocessing=use_multiprocessing,
...
    File "/home/christian/anaconda3/envs/retro/lib/python3.7/site-packages/keras/backend.py", line 5099, in categorical_crossentropy
      labels=target, logits=output, axis=axis)
Node: 'categorical_crossentropy/softmax_cross_entropy_with_logits'
logits and labels must be broadcastable: logits_size=[0,92366] labels_size=[256,92366]
	 [[{{node categorical_crossentropy/softmax_cross_entropy_with_logits}}]] [Op:__inference_train_function_798]

Examining the Keras model, it seems to be constructed correctly. Here's the summary:
Model: "sequential_4"
_________________________________________________________________
 Layer (type)                Output Shape              Param #
=================================================================
 dense_9 (Dense)             (None,<PHONE_NUMBER>

 dropout_5 (Dropout)         (None, 512)               0

 dense_10 (Dense)            (None, 92366)             47383758

=================================================================
Total params: 48,432,846
Trainable params: 48,432,846
Non-trainable params: 0
_________________________________________________________________

My only thought is that something didn't quite work as expected in the pre-processing step, which led to an issue with the train_seq or valid_seq generator. I'm unsure how to debug this and would be grateful for any advice!

I did manage to get this fixed - I don't remember what it was exactly but it was related to the pre-processing script. If I recall correctly, it was dropping molecules that couldn't be converted into fingerprints which threw off the dimensions of the data being passed.


[Query] Should Authenticate to retrieve secret pop-up open when user locks and unlock the phone after trying many wrong attempts

Frequency: 100%
Repro on build version: Android Internal Release Build V1.33.0, Android Alfajores Build V1.33.0
Repro on devices: Google Pixel 4a (12.0), Google Pixel XL(10.0), Google Pixel 2XL (11.0)
Pre-condition:
1.User must have installed the app.
2. User must be on Secure your wallet page
Repro Steps:

Tap on the “Turn on Fingerprint recognition” button.
Enter the wrong attempts 4-5 times.
Lock the phone
Now unlock the phone and tap on the “Turn on Fingerprint recognition” button and observe.

Query: Should Authenticate to retrieve secret pop-up should open when user locks and unlock the phone after trying many wrong attempts
Expected Behavior:  Authenticate to retrieve a secret pop-up should open when the user locks and unlock the phone after trying many wrong attempts.
Investigation:

Authenticate to retrieve a secret pop-up is not opening when the user locks and unlock the phone after trying many wrong attempts and tapping on the “Turn on fingerprint recognition” button user is redirected to the Phone number screen.

Attachment: Fingerprint Biometric.mp4

@kathaypacific it looks like we've failed to setup fingerprint verification; however, after completing the above reproduction steps fingerprint verification is enabled.


[1.x] Import events from 3rd-party with respect for tenancy

When it comes about importing the third-party events, the framework provides a ThirdPartyContext tool.
However, in a multi-tenant application, the imported events were dispatched in a straightforward manner, without specifying the TenantId via TenantAwareRunner.
The corresponding test was also flawed: in fact, it was dispatching the events to a single-tenant Bounded Context.
This PR addresses the issue by fixing the test and making IntegrationBroker (an intermediary between ThirdPartyContext and "domestic" Bounded Contexts) to explicitly use TenantAwareOperation during the event dispatch.

@yevhenii-nadtochii PTAL.


Language system-test fails because of returning Promise

Environment details

OS: Linux
Node.js version: v6.8.1
npm version: 3.10.8
google-cloud-node version: language / 0.3.0

Steps to reproduce

cd packages/language
npm run system-test (under proper environment variables)

Failures
  1) Language "after all" hook:
     TypeError: GCS.getBuckets(...).on is not a function
      at Context.<anonymous> (system-test/language.js:60:8)

As I quickly reviewed, GCS.getBuckets(...) seems to return a Promise, not a Stream, and the it returns an Array of buckets instead of stream.
I'm guessing recent packages/common/src/stream-router.js needs to be rewritten to support recent changes to Promise patterns.
cc: @jmdobry

@jmuk we just released changes today that moved all streaming APIs to their own methods (in this case GCS.getBuckets() became GCS.getBucketsStream()). So what you're describing is actually the expected behavior.
If you update to<EMAIL_ADDRESS>the error you're describing should disappear.

Yes, just rebase with master. The system tests depend on the latest release of @google-cloud/storage, which changed last night to promises.


Support reading directly from gcf files

That way no retarded pre-extraction of files would be needed

VPKs are supported, and GCFs seem to be no more.


Added shrinkwrap.json to files section to distribute shrinkwrap with release

Shrinkwrap was request by the community to protect from downstream deps breakages.
shrinkwrap is generated only during deployment time and does not include dev dependencies

This change was cherry-picked in 0.22.0-rc4, testing it npm2/npm3

@bestander you probably know this already...but I think your fix to "invariant" in master will fix the broken Circle test here.

Oh snap, thanks, Adam.
I actually missed it.
I am confused now, why does it break with shrinkwrap now?

That I don't know. It seems like travis is fine though, just not Circle. I'm curious what will happen if you rebase this with master.

Yeah, I'll try that

Closing this as shrinkwrap seemed to break builds on Windows


#491 Make ExtensionAppAuthenticator conditional on missing

Ticket
Closes #491
Description
Make the ExtensionAppAuthenticator and OboAuthenticator conditional
on missing bean, so that the BDK application is able to choose to
provide these customised beans.
Dependencies
N/A
Checklist

[ ] Referenced a ticket in the PR title and in the corresponding section
[ ] Filled properly the description and dependencies, if any
[ ] Unit tests updated or added
[ ] Javadoc added or updated
[ ] Updated the documentation in docs folder

@symphony-thibault , test added

@symphony-youri honestly, i cannot remember the use case, the ticket has been created long time ago. What i can recall is in one of our projects, we were not able to inject these custom beans, so we had to inject another bean at higher level as a workaround. I just tried to find some clue in the code, but nothing, maybe it is not relevant any longer. Anyway IMO, as a framework, it s still good to provide such freedom allowing inject custom beans.