MagicMirror Forum

@MichMich and I are currently setting up a forum for the MagicMirror community. What kind of categories would you guys like to see? Currently there are:

In an attempt to avoid the General Discussion to become a catch-all for everything, how about adding categories such as 'Installation Troubleshooting', or maybe 'Troubleshooting' in general? I can see where someone who has just discovered the project would get to the forum, hit the General Discussion and get overwhelmed by folks having errors, install problems, and who knows what else. Maybe all they want to ask is something like 'will this run on such-n-such hardware?', or 'hey, I forked this, and here's my design'.

Just a quick update, this is how it looks right now 🙂

Yeah, that would make more sense. Over time you can see what's being used the most, and how, and perhaps consolidate things then. It's a start.

I think that Core System Bug Reports should remain on Github. Otherwise we get two places were we should check stuff (and retire info about a bug).

Agreed. One place to report, and resolve bugs.

Maybe lock it and add a post describing on how you should fill a bug report on GitHub?

Yes, that would be a good way.

hi . here is paul . paviro knows me as paul the nagger from his blog .😉
cool idea with the forum .

Alright guys 🙂 Check out the forum!

Thanks for your hard work, @paviro!

You are welcome :) Hope it helps you to getting a bit less issues here!

I'll work on a gh-pages branch when I have time, and @paviro can add it to the domain as the root when done. Good work!

The Domain is owned by @MichMich :) Talk to him for that matter.

@nhubbard If it's sexy enough ... I'll link it to the domain! ;) Looking forward to your work.

@paviro Wasn't it something you wanted to work on, too? Maybe you guys can collaborate?

Happy to hand that over, since I haven't started anything yet anyway :)

Tried different browsers, same result:

Mhh strange... Will remove it for now give me a second :)

Go for it (should be back online any second)

I know I've ran into captcha issues before when serving an httpS:// domain when the certificate wasn't valid ... which both browsers complained about (unknown issuer).

Mhh no problem about that with my browsers. Used https://letsencrypt.org https://letsencrypt.org/ for the certificates.

Yep, I've used them too (still do in fact.)

Any idea why they are not yet in your browser? Which OS and browser are you using?

Confirmation e-mail taking the long scenic route now ...
Eh, one's Firefox (46.0 which is latest), and the other is Chrome which is the latest as well. It just updated this morning. It's possible that I have a higher security setting too. I'm at the office and I run a (very) tight ship as far as security goes for the entire building. Part of my job. We got fined before for having lax security so that's not a scenario I'm willing to go through again. So yeah, don't worry too much about it, certainly not if others can register just fine.

Forum will be back shortly!

Sorry for the downtime. It's back :)

You can now use your GitHub Account for logging :)

Is image uploading enabled?

Yes. Will check what the problem is. Can you send me the file via email so I can try with your header?

Does it work now?

Yes!

Nice! Moved the forum to another user because it was running as root when I set it up. Node could not write into the uploads folder anymore; fixed now :)

The joys of running apps on servers. :)

The one annoyance (so far) for me is the constant pop-up at the top about enabling desktop notifications. I don't want them, I blocked them in browser, I clicked the 'x' on the bar .... it continues to pop up, constantly.

@KirAsh4 Why would you not want MagicMirror Forum Notifications?! (Just kidding ... ;))

Mhh too bad this plugin does not use cookies or saves the choice in the database... Looks like other people complained about that too.. Not really sure what we should do about that, remove it?

Yes, I think we should. I don't think many users will use it.

Alright will remove it.

If it saved the user's settings, sure. But if it doesn't, to me that's a poor implementation of the feature. Particularly when others have also complained about it.

I have problem with cert. i use IE 11 and Firefox 45.0.1
Code temps :<PHONE_NUMBER>.610 (Wed, 04 May 2016 08:27:38 GMT)
URL :   	https://forum.magicmirror.builders/
Problème :   	Certificat du serveur distant non valide
Code erreur :   	INVALID_CERT, UNRECOGNIZED_ROOT_CERT, forum.magicmirror.builders)
Thanks

Looks like your System does not accept the https://letsencrypt.org certificate. Not sure why that is. Are you running the latest Firefox version? If not you can just ignore the warning (should be a button their) for now even tough that is not a god practise so you should not do that in general!

Same here (latest Chrome for Android)

@paviro Apparently there really is a certificate issue. Maybe you should contact https://letsencrypt.org?

Will do, strange that it only effects some 😯

Are you using the correct cert file?
http://letsencrypt.readthedocs.io/en/latest/using.html#where-are-my-certificates
Please note, that you must use either chain.pem or fullchain.pem. In case of webservers, using only cert.pem, will cause nasty errors served through the browsers!
Similar problems:
https://community.letsencrypt.org/t/lets-encrypt-certificate-not-working-for-andriods-google-chrome/7184

Does it work now? Thanks @fcorin!

yes very good ! Thanks :) :)

Hey, I´m not sure if this is the right place to ask this because I am navigating on my pi. Anyways, I am trying to get location for weather services modules to automatically find the pis location using google maps geolocation API. How do I go about adding this to the existing module? If I put in the url with the API inside the location field, will that return a usable lat/long coordinate that the code can use to send to OpenWeather?

Please ask this question on http://forum.magicmirror.builders 👍🏻

Can´t do anything everytime forum says invalid session.No login ,no password reset. :-(

Try deleting your cookies!

No luck in Firefox.I have installed Chrome and everything works.

I bought some glass samples, It was an one-way glass, like the interrogation room, but it's too darker than the normal glass, Can you give me some advice? about which glass is better?  thanks a lot.

@SouthDeer This GitHub issue is not a forum itself. Please check out this topic regarding your question: https://forum.magicmirror.builders/topic/22/where-to-buy-good-two-way-mirrors

Hello,
Is MagicMirror2 supposed to work on a RPI 3 ? I have an issue related to electron that is not found.
Does it sound familiar to anyone ?

@Corvis1984 Please check http://forum.magicmirror.builders for support.

Hello!
i can't login to the forum, when i open the site it spams a "invaild session" message on the right side and when i wanna log in it says "Login Unsuccessful We were unable to log you in, likely due to an expired session. Please try again". Did i do anything wrong or what is the problem ?

Does it work now?

No :-( Got the same errors... "ERROR invalid-session" on the right side in a message box and the "Login Unsuccessful We were unable to log you in, likely due to an expired session. Please try again". Can't even make a new account, got the same errors... at home and even at work on phone, mac, pc and even raspberry :-/

Please clear your cookies so you have a fresh start after what I changed.

That was the problem, now everything works great again! Thanks for your help :-)

Forum has a little outage, working on it :)

Back online, make sure to read: https://forum.magicmirror.builders/topic/420/forum-outage-and-restore-from-backup :)

Almost, but no cigar ... dead again.

Old Server just came back... Going to restore database.

Put a leash on it ... it's not allowed to just go on a walk in the park like that.

I'm getting:
ERROR
invalid-session
From login with github.

@jtyle6 did you remove your cookies yet?

It didn't  work,   I'm trying to login with the Alternative Logins.
I will try again tomorrow morning when mac is back on.

Right now, I'm lacking a Confirmation Email to be able to use the Forum, after registering via Github.
Hmm.
I'm in Berlin with a 42" FullHD MagicTouchMirror, but would want a slightly different setup to what I've seen - I'd rather want to have something in the borders and corners instead of using screen real estate in the middle ...

It probably sends the confirmation Email to your non-reply github email.

@berlincount I've manually validated your account.

This thread is about the forum not the magic mirror software! If you need help go to said forum :) http://forum.magicmirror.builders

Hello. I have a problem with my confirmation email. I haven't received it. I resend it a couple of times i even changed my email to another one but still havent received anything.


stuck on cmd

🧐 Have you read the FAQ and viewed similar issues?

[X] I have read the FAQ
[X] I have checked older issues, open and closed
[X] I did a clean install of SpotX as described in the FAQ

Сountry of your account
finland
What is your Spotify Plan?
Free
Do you have Spicetify installed?
Yes
ℹ Computer information
- Spotify:
- Windows: 11
- PowerShell:

📝 Description

when i download its just goes to the cmd screen and doesnt do anything

...
📸 Screenshots
No response

when i download its

downloading what?  launch what? screenshot ?

I may have found a fix.
When trying to run the run script using PowerShell, cmd, Ect. It appears for a split second my fix is the following.

Install Visual Studio Coding Software
Enter Extensions Marketplace And Install PowerShell From The Search Bar
After This Navigate To Your Unzipped SpotX-main Folder Inside You Sould See A run.ps1 or it will say run.
Right Click On run.ps1, navigate to Open With >, Then Click Open With Visual Studio.

You Should See The Normal Visual Studio Interface, But On The Left Click The Run And Debug Panel.

Click On Run And Debug The Big Blue Box

After This Answer The Prompts In The Terminal With Either Y or N On Your Keyboard.

I Hope This Works For You


[wine-utils] allow 32 bit setup_dxvk.sh to work alongside 64 bit

This allows the 32 bit dxvk to be installed alongside 64 bit in syswow64 in 64 bit prefixes, in case of wine cross compiled x86_64, thus allowing a 64 bit prefix to run both 64 and 32 bit dxvk

closed per discord convo


The client feign death

Rancher versions:
rancher/server: v1.14
rancher/agent: v1.0.2
1、Add the container through the rancher server （<IP_ADDRESS>）
Error (Scheduling failed: No candidates available)
2、The physical state of machine（<IP_ADDRESS>）

3、stop docker （<IP_ADDRESS>）
Active is still displayed on the rancher server

4、restart docker （<IP_ADDRESS>）
Add  container  ：Error (Scheduling failed: No candidates available)

Thanks for reporting @mariadb-Emrys , is it correct you are running v1.1.4? There have been many releases and improvements, could you try on a newer release?

{"log":"2017-05-19 01:52:40,219 ERROR [e7253fe9-f3fc-4f47-9738-cee5fbaae2be:1498596] [instance:9060] [instance.start-\u003e(InstanceStart)] [] [orService-12596] [i.c.p.process.instance.InstanceStart] Failed to Scheduling for instance [9060] \n","stream":"stdout","time":"2017-05-19T01:52:40.22069707Z"}

@superseb  How to restore

@mariadb-Emrys You can look in this issue to recover: https://github.com/rancher/rancher/issues/5873.
As stated, there is a fix in later releases for this situation. My advice is to upgrade as soon as possible to prevent this.

@superseb  thx


Issue 253

Pull request to address #253, which aims to provide enhanced support for callback types at the DAG, default_args, Task and TaskGroup level. Unit-tests have been added and updated appropriately.

Codecov Report
Attention: Patch coverage is 92.59259% with 2 lines in your changes missing coverage. Please review.

Project coverage is 93.14%. Comparing base (017bc30) to head (ca9d1cb).

Files with missing lines
Patch %
Lines

dagfactory/dagbuilder.py
92.59%
2 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #322      +/-   ##
==========================================
- Coverage   93.29%   93.14%   -0.16%
==========================================
  Files          10       10
  Lines         776      759      -17
==========================================
- Hits          724      707      -17
  Misses         52       52

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@pankajastro, do you mind reviewing this, open to feedback?
cc: @tatiana


TextEdit - add tests for spaces and empty string

Need to add tests for spaces at front and back and replacement with an empty string.

I added the tests.  Close this issue.


Need Swift Version or a Swift Tutorial

Guys,, i need a swift version or a swift tutorial to use this calendar in my project since i am not an expert in objective C. Anyone have used this library in their swift project? if it is, i need help to implement this library in my project

have you found any solution? i also want to implement in swift

@HPRaval no. not yet bro :(

Hi guys! a swift version has already been asked here, check out other questions. There is a link for swift version of the files outside of the library. here is the link -> https://github.com/jumartin/Calendar/files/371341/TestCalendarLibForSwift.zip
it took me a while to understand it as well, but this should help:

get the library files via cocoapods
download the swift files (url above) and copy all files into your project (you can skip the viewcontrollers you don't want like YearViewController)
create a viewcontroller and subclass the MainViewController
design the viewcontroller to have a Month (label), control switch (segmented control), and the calendar itself (container). I skipped the previous and next buttons but you can add it as well
connect the outlets and actions. make sure the segmented control is connected correctly as you will have an error if not

let me know if you still have questions :)

@AzharNaaz Could it be related to this issue: https://github.com/jumartin/Calendar/issues/68 ?

@SamDecrock Issue is when we change the page and new month arrived, while new month arrived like wise it changed to April to May and if i set my startdate of event "2017-04-14 03:56:31 +0000" and Enddate of event "2017-04-14 04:56:31 +0000" it is completely working in april month and when month change april to may delegate again exucute and index again start with the 0 and again it cause crash cause the date does not match to the current date.and give me the reason "End date earlier than start date in DateRange object!" .  do you have any solution about this?

@SamDecrock
The MonthViewController file have lots of error , is it about swift 2 and swift 3 ? Currently I am using swift 3

I'm using Swift 3.

Any swift 3 solution? I am looking for it..

please Update for swift 5


Thumbnail generation script causes games to crash, causing generation to fail

The thumbnail generation script overrides some functions with no-ops when running games. For example, setTune, for games running within the script, is set to () => {}. This causes some thumbnails to fail to generate, as seen below, when the return values of these functions are accessed (in this case: playTune().end()

Relevant file: src/integrations/thumbnail.ts

please can you link a game that crashes while a thumbnail is being generated? or provide steps to reproduce

https://sprig.hackclub.com/gallery/Hey_Hay_Cow


[WIP] Use cuda stubs in Travis CI

refs: #9

[x] Install cuda packages
[x] Run cmake with CUDA options
[x] Install cudnn
[ ] Make tests written in python pass

I've completed the building but running tests failed so closing.
This needs CUDA environment to test.
Will reuse some if there is a good idea to make tests run.


support client-side objects

See https://documentation.basis.cloud/BASISHelp/WebHelp/bbjobjects/clientobject_tutorial.htm
rem Print time and TimeZone of client machine
use java.util.Date
use java.text.DateFormat
clientDate! = new Date@()
clientFormat! = DateFormat@.getDateTimeInstance()
print clientFormat!.format(clientDate!)," ",
print clientFormat!.getTimeZone().getDisplayName()

Fixed in typefox-dev.
Follow-up: validate that @ is only used in allowed expressions.


Question: Has this been tested against the Trition Flash Attention version?

Hi, I saw your research and found it amazing particularly for the memory efficiency.

Mosaic implemented Flash Attention with Triton. This works by fusing it into the forward and backward operations for memory and performance efficiency:
https://github.com/mosaicml/llm-foundry/blob/main/llmfoundry/models/layers/flash_attn_triton.py
Specifically, I am wondering if you have performed any tests against this type of flash attention since this is now considered SOTA in terms of speed.

Good Q. We compared with Jax code instead of Triton code. Triton code is definitely more ideal and faster, since it operates kernel ops directly which is better than letting compiler optimization. We are planning to add triton based implementation but no concrete eta yet.

Just FYI, if you are interested in optimizing speed, a minimal effort way would be replacing blockwise_compute_attn with using jax-triton to call the triton flashattn; and keeping other blockwise operations including blockwise_compute_ffn and blockwise_cross_entropy as is. This should provide both the memory saving of Blockwise Parallel Transformer and the speed gain of FlashAttention.

Hi @lhao499 ,
Thank you for your profound research on BPT and CoH!
I am currently working on a PyTorch rewrite of BPT.
Do you mind clarifying the part where you say you can replace blockwise_compute_attn with Triton Flash Attention? Do you mean you can do a direct one-to-one replacement by just swapping blockwise_compute_attn with Flash Attention? Or do you mean that the Triton version of Flash Attention needs to be rewritten in a way where it follows the same structure as blockwise_compute_attn?
I will likely open up a new issue soon to further discuss the PyTorch reimplementation.
Thank you again for all of your great work!
Best,
Enrico

Hi Enrico, thanks for your interest.
Yes, I think blockwise_compute_attn can be swapped with Triton flash attention, nanoGPT has triton flash attention integrated and there are also two other Triton implementations 1 2. For the remaining blockwise operations including blockwise_compute_ffn and blockwise_cross_entropy, a simpler way would be continue using Jax/PyTorch. The reason is that it requires some efforts to fuse blockwise FFN with flash attention for best performance. In addition, with blockwise parallel compiler can do some automatic fusion (albeit not necessarily the most optimal one).

Thank you for the additional clarity. It is good to know that Flash Attention can be used as a drop-in replacement without having to do any rewrite.

Hi @lhao499, congratulations on this great piece of work!

We compared with Jax code instead of Triton code.

Can you point me to the FlashAttention implementation in JAX in this repo? It appears to me that you only implemented the memory-efficient transformer, but not FlashAttention.
https://github.com/lhao499/blockwise-parallel-transformer/blob/2f42ba1bc73e91eb6315759706029b1a67b54e6d/README.md?plain=1#L87

there are also two other Triton implementations 1 2.

In case it's relevant, here is a third one: https://github.com/jax-ml/jax-triton/blob/main/examples/fused_attention.py

Flashattention uses the same method as memory-efficient attention to reduce memory cost, but is implemented with low level kernels to achieve further speed up. In this repo, code is not implemented in low-level kernels as we mainly focus on reducing memory cost, using low-level kernels should be straightforward with Triton or Pallas.
If you would like to use low level kernels, you can use Jax-Triton's Triton or Pallas API on GPUs, or use the Pallas API with mosaic lowering[1][2] on TPUs.

Hi @lhao499, I saw you compared BPT with memory-efficient attention, and mentioned that flash-attention use the same method as memory-efficient attention.
However, in Appendice B.5 of flash-attention paper , author compared their implementation with memory-efficient attention, and point out flash-attention has 2-4x speed up and smaller total memory requirement.
So I'm curious about the real speed/memory comparison between BPT and flash-attention.
I also found that flash attention seems to conflict with BPT.
Here is the Pseudocode of flash attention:

compared with BPT:

The outer loop of BPT scan the Q blocks like memory-efficient attention,  so it can compute blockwise ffn after inner loop done . The outer loop of flash-attention scan the K,V blocks, it can reduce the total memory requirement by  incrementally updates the output instead of store copies of output, as mentioned in Appendice B.5 :

FlashAttention instead incrementally updates the output (Algorithm 1 line 12) after processing each block, so only one copy of the output is needed (instead of 𝐾 copies for 𝐾 blocks). This means that FlashAttention has smaller total memory requirement compared to Rabe and Staats [66].

Regarding scanning order:
I'm not exactly sure why FlashAttention chose to scan K,V blocks first, but FlashAttention2 chose to scan Q blocks as done in memory-efficient attention and BPT. I will need to take a further look.
Regarding speed:
FlashAttention provides an efficient CUDA implementation for memory-efficient attention, at the time of implementing BPT, there was no public kernel APIs for TPU and we were mainly using TPUs, so we compared BPT with memory-efficient attention.
BPT is compatible with FlashAttention kernel optimization, recently, Jax has added Pallas for kernel APIs, it would cool to try it out.

Essentially, the comparison is somewhat biased in that the core of Flash Attention 2 is intensive low-level optimizations (I/O, kernel fusion, recomputation,  SM scheduling) specific to the NVIDIA architecture, which is why it's much more efficient and widely adopted over MEA.
I tend to believe that your method will prevail after fusing the FFN within FA's kernel, but that will take too much effort.


Gemma 2 issue with accelerate v0.32.0

System Info
accelerate==0.32.0
python==3.10.14
pytorch==2.3.0
transformers==4.42.3

Information

[ ] The official example scripts
[X] My own modified scripts

Tasks

[ ] One of the scripts in the examples/ folder of Accelerate or an officially supported no_trainer script in the examples folder of the transformers repo (such as run_no_trainer_glue.py)
[X] My own task or dataset (give details below)

Reproduction
Running the example from the just released gemma 2 works fine:
# pip install accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tokenizer = AutoTokenizer.from_pretrained("google/gemma-2-9b-it")
model = AutoModelForCausalLM.from_pretrained(
    "google/gemma-2-9b-it",
    device_map="auto",
    torch_dtype=torch.bfloat16
)

input_text = "Write me a poem about Machine Learning."
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")

outputs = model.generate(**input_ids)
print(tokenizer.decode(outputs[0]))

Then if I try it with accelerate though I get an error:
from accelerate import Accelerator
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

accelerator = Accelerator()

tokenizer = AutoTokenizer.from_pretrained("google/gemma-2-9b-it")
model = AutoModelForCausalLM.from_pretrained(
    "google/gemma-2-9b-it",
    device_map="auto",
    torch_dtype=torch.bfloat16
)

model = accelerator.prepare(model)

input_text = "Write me a poem about Machine Learning."
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")

outputs = model.generate(**input_ids)
print(tokenizer.decode(outputs[0]))

Here's my accelerate config:
compute_environment: LOCAL_MACHINE
debug: false
distributed_type: 'NO'
downcast_bf16: 'no'
enable_cpu_affinity: false
gpu_ids: '0'
machine_rank: 0
main_training_function: main
mixed_precision: bf16
num_machines: 1
num_processes: 1
rdzv_backend: static
same_network: true
tpu_env: []
tpu_use_cluster: false
tpu_use_sudo: false
use_cpu: false

Here's the error I get:
Traceback (most recent call last):
  File "test_gemma-2.py", line 20, in <module>
    outputs = model.generate(**input_ids)
  File "/home/miniconda3/envs/hf-torch2/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/miniconda3/envs/hf-torch2/lib/python3.10/site-packages/transformers/generation/utils.py", line 1914, in generate
    result = self._sample(
  File "/home/miniconda3/envs/hf-torch2/lib/python3.10/site-packages/transformers/generation/utils.py", line 2651, in _sample
    outputs = self(
  File "/home/miniconda3/envs/hf-torch2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/miniconda3/envs/hf-torch2/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/miniconda3/envs/hf-torch2/lib/python3.10/site-packages/accelerate/utils/operations.py", line 819, in forward
    return model_forward(*args, **kwargs)
  File "/home/miniconda3/envs/hf-torch2/lib/python3.10/site-packages/accelerate/utils/operations.py", line 807, in __call__
    return convert_to_fp32(self.model_forward(*args, **kwargs))
  File "/home/miniconda3/envs/hf-torch2/lib/python3.10/site-packages/accelerate/utils/operations.py", line 786, in convert_to_fp32
    return recursively_apply(_convert_to_fp32, tensor, test_type=_is_fp16_bf16_tensor)
  File "/home/miniconda3/envs/hf-torch2/lib/python3.10/site-packages/accelerate/utils/operations.py", line 118, in recursively_apply
    {
  File "/home/miniconda3/envs/hf-torch2/lib/python3.10/site-packages/accelerate/utils/operations.py", line 119, in <dictcomp>
    k: recursively_apply(
  File "/home/miniconda3/envs/hf-torch2/lib/python3.10/site-packages/accelerate/utils/operations.py", line 126, in recursively_apply
    return func(data, *args, **kwargs)
  File "/home/miniconda3/envs/hf-torch2/lib/python3.10/site-packages/accelerate/utils/operations.py", line 778, in _convert_to_fp32
    return tensor.float()
AttributeError: 'HybridCache' object has no attribute 'float'
Traceback (most recent call last):
  File "/home/miniconda3/envs/hf-torch2/bin/accelerate", line 10, in <module>
    sys.exit(main())
  File "/home/miniconda3/envs/hf-torch2/lib/python3.10/site-packages/accelerate/commands/accelerate_cli.py", line 48, in main
    args.func(args)
  File "/home/miniconda3/envs/hf-torch2/lib/python3.10/site-packages/accelerate/commands/launch.py", line 1097, in launch_command
    simple_launcher(args)
  File "/home/miniconda3/envs/hf-torch2/lib/python3.10/site-packages/accelerate/commands/launch.py", line 703, in simple_launcher
    raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)

Expected behavior
Same code works fine if I use google/gemma-1.1-7b-it instead. Gemma 2 should also work, but I assume accelerate needs to be patched.

Is there a reason you are doing accelerate.prepare? In general this shouldn't be used for inference if you're using device_map="auto".
This likely may stem from loading the model in bf16 instead of float32

Right! I tested without accelerate.prepare and it works. Just saw in the doc: You don’t need to prepare a model if you only use it for inference without any kind of mixed precision
I tried a script with gemma 2 where I do training and inference, and thus call accelerate.prepare, and that works as well. Case closed. Sorry about that!


Add new example: squares

Useful for generating Wangview-compatible tilesets which look like unprocessed source tiles, or as a base for other tilesets with non-noisy boundaries.
Will deprecate ~TilePartitionerPerlin~TilePartitionerSquares and affect #93.
I don't know why the combiner module group needs:
    "and_t":{
      "type": "TranslatePoint",
      "Translation": [-0.5, 0.5, 0],
      "SourceModule":"and"
    },

I would have expected [0.5, 0.5, 0], because and is symmetric in x=y, but that causes the corner masks to be flipped horizontally. Either there's a fault in corner mask generation that #90 didn't fix, or I'm misunderstanding the module group I wrote.
Tested in #90 before committing, but branched from master because it probably doesn't depend on the changes in #90.

After rebase on master, running:
./Wangscape example_squares/example_options.json

causes
terminate called after throwing an instance of 'spotify::json::decode_exception'
  what():  Unexpected input
spotify::json::decode_exception encountered at 216: Unexpected input[1]    14413 abort (core dumped)  ./Wangscape example_squares/example_options.json

Could you check it?

I wonder, if we could use images (sea.png and grass.png) that are already in Wangscape's repository, setting proper path in options file. This will save us from making the repository bigger, I think.

Yes, it would make sense to put a single copy of each source tile in a folder like doc/images/tiles. Not in doc/examples/tiles, because the AppVeyor scripts would try to run Wangscape on doc/examples/tiles/example_options.json. I would prefer to do that in a separate pull request, though.
I'm checking how example_squares runs after rebasing the branch now. Do you have a Wangscape executable in doc/examples?

Wangcheck found an error:
Failed validating 'type' in schema['properties']['TileFormat']['properties']['Re
solution']:
    {'description': 'Specifies the width and height of the square tiles '
                    'Wangscape will operate on.',
     'items': {'maximum': 512,
               'minimum': 8,
               'multipleOf': 1,
               'type': 'integer'},
     'maxItems': 2,
     'minItems': 2,
     'title': 'Tile resolution',
     'type': 'array'}

On instance['TileFormat']['Resolution']:
    32

The example wasn't updated when #91 was merged. I've fixed that, and there are no more complaints from Wangcheck, but I'm still getting a crash.

Debug output was enabled. Without the fixes in #111, Wangscape crashes if there isn't a test folder in the current directory. I've turned debug output off, which fixes it.
Even with the fixes in #111, the debug output routine waits for stdin. Until we have a non-interactive way to limit the number of debug tiles produced, examples should never be committed with debug output enabled.


Recent update can not move tabs around

Issue Type
Bug
Description
I used to be able to move file tabs around to the order that I want them.  I am unable to now and there is a CPU spike when I try doing so.

Screen Reader
no

VM
0%

Extensions (4)

Extension
Author (truncated)
Version

vscode-eslint
dba
1.4.5

es7-react-js-snippets
dsz
1.3.3

prettier-vscode
esb
1.1.3

vscode-versionlens
pfl
0.19.1

(1 theme extensions excluded)

Reproduces only with extensions

@DavidKindler this works as expected for me in the stable version.
Is there an error shown in the console (Help>Toggle Developer Tools, then open the Console tab)?
Since you indicate that this only reproduces with extensions, please narrow down which extension is causing the problem. To selectively enable/disable extensions pls see https://github.com/Microsoft/vscode/wiki/Performance-Issues#run-with-extensions-disabled.

I disabled all extensions and problem persists.  No errors popping up in Console tab.

@DavidKindler please provide more information about the tab configuration when this happens. Please add some screenshots. What is the tab configuration that you want to customize, what do you want to change but doesn´t work.

Here is a simple example of how I've been able to reproduce and possibly narrow down the issue.
First disable all extensions and reload.
Create a new directory and open empty folder in VSC.  Create two new files but do not save them.  Attempt to move tabs using mouse - this works.
Create two files and save them.  In my case they were test1.js and test2.js with one commented line.
Try to move the saved files tabs around - does not work.
But I can still move the unsaved files tabs.
Hope that helps.  I can do a screencast if  you need one,

@DavidKindler followed your steps and I can move test1.js and test2. js. Please provide a screencast.

Attached screencast.  You can see that when I try to move the saved versions the editor shows "Not responding" after a few secs.
https://github.com/Microsoft/vscode/issues/10066
I think this might be related issue but I have not found out how to run in non-admin mode.

@egamma - I have been able to resolve this issue by reinstalling a fresh copy of vscode.
Closing out issues.  Thanks.


🐛 Bug Report: GitHub events support is not matching providers based on organization

📜 Description
GitHub events support is presently not matching / filtering based on the organization specified in the config. The result is that if you have a config with multiple provider IDs, the created catalog entity is created in all of them. You then get errors like
Source github-provider:another detected conflicting entityRef location:default/generated-abc123 already referenced by github-provider:default and now also github-provider:another

👍 Expected behavior
If you have a config file like:
events:
  modules:
    github:
      webhookSecret: redacted
  http:
    topics:
    - github
  providers:
    github:
      default:
        organization: org1
        catalogPath: **/catalog-info.yaml
        schedule:
          frequency: {minutes: 120}
          timeout: {minutes: 30}
      another:
        organization: org2
        catalogPath: **/catalog-info.yaml
        schedule:
          frequency: {minutes: 120}
          timeout: {minutes: 30}

and a Github push event comes in for org2/repo1, you would expect it to only match the another provider ID, not additionally the default.
To get more specific, I would expect that in GithubEntityProvider.onRepoPush(), that there would be some type of org filtering for the event, more like what we see for scheduled scans in getOrganizationRepositories. I'd expect that matchesFilters would additionally filter given the on the organization name in the config vs. what is in the full_name in the push event.
👎 Actual Behavior with Screenshots
Log errors like
Source github-provider:another detected conflicting entityRef location:default/generated-abc123 already referenced by github-provider:default and now also github-provider:another

👟 Reproduction steps
Create a config like in the expected behaviors above, like the examples you see in the documentation, where you have multiple providers.
Note the resulting conflicting entityRef location logs. Note the incorrect entity refs that occur in the paths within the browser when viewing the entity which came through a GitHub event.
📃 Provide the context for the Bug.
This means we cannot use the event mechanism, since our enterprise has multiple GitHub orgs, each with distinct provider IDs.
🖥 Your Environment
No response
👀 Have you spent some time to check if this bug has been raised before?

[X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

[X] I have read the Code of Conduct

Are you willing to submit PR?
No, I don't have time to work on this right now

Yep, makes sense! 👍 Opening this up for contributions

We have not retested this recently, but also have no reason to believe it is fixed. We are unable to use the Github events support, since our Backstage instance has multiple GitHub orgs.

not stale

@pjungermann @angeliski do you wanna take a look at this and see what's happening?


Allow more options to disable enterprise mode, fixes #6779

Any tests we can add here?

We do have lots of different areas where we have boolean options via environment variables. Should we put this in a central place somewhere in fastlane_core?

@KrauseFx
Good idea! 👍


Add "outbound" HTTPS proxy support

See here:
https://github.com/AdguardTeam/AdguardForAndroid/issues/2984
Once it's done -- add a task to Android repo

HTTPS is one of the requested proxy protocols.
https://github.com/AdguardTeam/AdguardForAndroid/issues/2111
But it's maybe sensitive for Chinese government because encrypted proxy protocols can pass Chinese GFW.
If possible, I'll appreciate that AdGuard gradually supports more other encrypted proxy protocols, ex: SSH, ShadowSocks...etc.
Best Regards.

core/pull-requests/1663


Blog page - title behavior

Jack @JiaKunUp
When I click on the button 'AmJ'  on the blog pages corner ..(top right corner ..same as Jalpc)   , it routs me to url 'ashutoshmjain.github.com' .. Since there is nothing hosted on above url , it returns a 404 error.
Is there a way, I could have it routed to ashtoshmjain.github.com/homepage where my site is hosted ..
btw - it works fine on the main landing page where clicking on it takes me to page top ..
Thanks in advance

@ashutoshmjain  Thanks for using this theme, this code is located at _includes/header.html and I use {{ site.title }} before, now it has changed to {{ site.title }}, I think it works for you.
Thanks for finding this bug.

It worked for me  ..
Thanks again for the wonderful work .
Really appreciate
Ashutosh


[Discussion] How does spleeter handle stereo files when training?

I'm creating a dataset for training my own model. At the moment my dataset is comprised of mono loops. I'm trying to figure out if I should be generating stereo but i don't know how Spleeter handles mono vs stereo. I've tried digging through the code but I'm a novice with Tensorflow and this is a complex project.
Does it train on a stereo wav or does it split the wav up into 2 mono wavs (Left & Right channels)?
If it uses both channels at once are they in context of one another? If I have sounds panned hard left & right does that matter?
If I have 1,000 stereo sets does it treat it as 2,000 examples or just 1,000?
Much appreciated..

@romi1502 thank you for that answer, very helpful!. I have started training, my test run on 5k mono drum loops ended with these results.. TBH I don't know what they mean, what does good look like?
absolute_difference = 1.7624866,
global_step = 200000,
loss = 1.7624866,
brass_perc_spectrogram = 0.3552358,
kicks_spectrogram = 0.3987537,
pitched_perc_spectrogram = 0.5570321, s
nares_claps_spectrogram = 0.45146513
If the data is only being trained on one channel do you think that could improve or reduce accuracy?  I'm training on 55k mono loops now. Hopefully that will give a good result 🤞.


Typos and grammar in german language

Fixed Typos and optimized gramma
Rules:
Objekt ID (with k and ID in capitals)
Die View (female)
Das Widget (neutrum)
tbd:
gestures

Thank you pix. Same question as always. License Agreement or may I merge myself without history?

Another fix for german language:
"letzte Änderung" with e in "letzte".


The scoreboard isn't working for some reason, specifically:...

The scoreboard isn't working for some reason, specifically:
'&8» &c%vault_eco_balance_fixed%'
'&8» &c%vault_prefix%'
'&8» &c%statistic_player_kills%'
'&8» &c%statistic_deaths%'

I'll need a bit more information on this before I can help;
What version of SpigotBoard are you using?
What type of minecraft server are you running this on? (Spigot, Paper, etc..)
and what version of minecraft are you running this on?
Are there any errors or messages in the console? Please post the whole config for your SpigotBoard.


Require size.state to be greated than memory for stateful stop/snapshot

Required information

Distribution: Ubuntu
Distribution version: 20.04 LTS
The output of "lxc info" or if that fails:

Kernel version: 5.11.0-40-generic
LXC version:
LXD version: 4.21 Snap
Storage backend in use: btrfs + ceph

   ~
root @ node1 # lxc info
config:
  cluster.https_address: <IP_ADDRESS>:8443
  core.https_address: <IP_ADDRESS>:8443
  core.trust_password: true
api_extensions:
- storage_zfs_remove_snapshots
- container_host_shutdown_timeout
- container_stop_priority
- container_syscall_filtering
- auth_pki
- container_last_used_at
- etag
- patch
- usb_devices
- https_allowed_credentials
- image_compression_algorithm
- directory_manipulation
- container_cpu_time
- storage_zfs_use_refquota
- storage_lvm_mount_options
- network
- profile_usedby
- container_push
- container_exec_recording
- certificate_update
- container_exec_signal_handling
- gpu_devices
- container_image_properties
- migration_progress
- id_map
- network_firewall_filtering
- network_routes
- storage
- file_delete
- file_append
- network_dhcp_expiry
- storage_lvm_vg_rename
- storage_lvm_thinpool_rename
- network_vlan
- image_create_aliases
- container_stateless_copy
- container_only_migration
- storage_zfs_clone_copy
- unix_device_rename
- storage_lvm_use_thinpool
- storage_rsync_bwlimit
- network_vxlan_interface
- storage_btrfs_mount_options
- entity_description
- image_force_refresh
- storage_lvm_lv_resizing
- id_map_base
- file_symlinks
- container_push_target
- network_vlan_physical
- storage_images_delete
- container_edit_metadata
- container_snapshot_stateful_migration
- storage_driver_ceph
- storage_ceph_user_name
- resource_limits
- storage_volatile_initial_source
- storage_ceph_force_osd_reuse
- storage_block_filesystem_btrfs
- resources
- kernel_limits
- storage_api_volume_rename
- macaroon_authentication
- network_sriov
- console
- restrict_devlxd
- migration_pre_copy
- infiniband
- maas_network
- devlxd_events
- proxy
- network_dhcp_gateway
- file_get_symlink
- network_leases
- unix_device_hotplug
- storage_api_local_volume_handling
- operation_description
- clustering
- event_lifecycle
- storage_api_remote_volume_handling
- nvidia_runtime
- container_mount_propagation
- container_backup
- devlxd_images
- container_local_cross_pool_handling
- proxy_unix
- proxy_udp
- clustering_join
- proxy_tcp_udp_multi_port_handling
- network_state
- proxy_unix_dac_properties
- container_protection_delete
- unix_priv_drop
- pprof_http
- proxy_haproxy_protocol
- network_hwaddr
- proxy_nat
- network_nat_order
- container_full
- candid_authentication
- backup_compression
- candid_config
- nvidia_runtime_config
- storage_api_volume_snapshots
- storage_unmapped
- projects
- candid_config_key
- network_vxlan_ttl
- container_incremental_copy
- usb_optional_vendorid
- snapshot_scheduling
- snapshot_schedule_aliases
- container_copy_project
- clustering_server_address
- clustering_image_replication
- container_protection_shift
- snapshot_expiry
- container_backup_override_pool
- snapshot_expiry_creation
- network_leases_location
- resources_cpu_socket
- resources_gpu
- resources_numa
- kernel_features
- id_map_current
- event_location
- storage_api_remote_volume_snapshots
- network_nat_address
- container_nic_routes
- rbac
- cluster_internal_copy
- seccomp_notify
- lxc_features
- container_nic_ipvlan
- network_vlan_sriov
- storage_cephfs
- container_nic_ipfilter
- resources_v2
- container_exec_user_group_cwd
- container_syscall_intercept
- container_disk_shift
- storage_shifted
- resources_infiniband
- daemon_storage
- instances
- image_types
- resources_disk_sata
- clustering_roles
- images_expiry
- resources_network_firmware
- backup_compression_algorithm
- ceph_data_pool_name
- container_syscall_intercept_mount
- compression_squashfs
- container_raw_mount
- container_nic_routed
- container_syscall_intercept_mount_fuse
- container_disk_ceph
- virtual-machines
- image_profiles
- clustering_architecture
- resources_disk_id
- storage_lvm_stripes
- vm_boot_priority
- unix_hotplug_devices
- api_filtering
- instance_nic_network
- clustering_sizing
- firewall_driver
- projects_limits
- container_syscall_intercept_hugetlbfs
- limits_hugepages
- container_nic_routed_gateway
- projects_restrictions
- custom_volume_snapshot_expiry
- volume_snapshot_scheduling
- trust_ca_certificates
- snapshot_disk_usage
- clustering_edit_roles
- container_nic_routed_host_address
- container_nic_ipvlan_gateway
- resources_usb_pci
- resources_cpu_threads_numa
- resources_cpu_core_die
- api_os
- container_nic_routed_host_table
- container_nic_ipvlan_host_table
- container_nic_ipvlan_mode
- resources_system
- images_push_relay
- network_dns_search
- container_nic_routed_limits
- instance_nic_bridged_vlan
- network_state_bond_bridge
- usedby_consistency
- custom_block_volumes
- clustering_failure_domains
- resources_gpu_mdev
- console_vga_type
- projects_limits_disk
- network_type_macvlan
- network_type_sriov
- container_syscall_intercept_bpf_devices
- network_type_ovn
- projects_networks
- projects_networks_restricted_uplinks
- custom_volume_backup
- backup_override_name
- storage_rsync_compression
- network_type_physical
- network_ovn_external_subnets
- network_ovn_nat
- network_ovn_external_routes_remove
- tpm_device_type
- storage_zfs_clone_copy_rebase
- gpu_mdev
- resources_pci_iommu
- resources_network_usb
- resources_disk_address
- network_physical_ovn_ingress_mode
- network_ovn_dhcp
- network_physical_routes_anycast
- projects_limits_instances
- network_state_vlan
- instance_nic_bridged_port_isolation
- instance_bulk_state_change
- network_gvrp
- instance_pool_move
- gpu_sriov
- pci_device_type
- storage_volume_state
- network_acl
- migration_stateful
- disk_state_quota
- storage_ceph_features
- projects_compression
- projects_images_remote_cache_expiry
- certificate_project
- network_ovn_acl
- projects_images_auto_update
- projects_restricted_cluster_target
- images_default_architecture
- network_ovn_acl_defaults
- gpu_mig
- project_usage
- network_bridge_acl
- warnings
- projects_restricted_backups_and_snapshots
- clustering_join_token
- clustering_description
- server_trusted_proxy
- clustering_update_cert
- storage_api_project
- server_instance_driver_operational
- server_supported_storage_drivers
- event_lifecycle_requestor_address
- resources_gpu_usb
- clustering_evacuation
- network_ovn_nat_address
- network_bgp
- network_forward
- custom_volume_refresh
- network_counters_errors_dropped
- metrics
- image_source_project
- clustering_config
- network_peer
- linux_sysctl
- network_dns
- ovn_nic_acceleration
- certificate_self_renewal
- instance_project_move
- storage_volume_project_move
- cloud_init
- network_dns_nat
- database_leader
- instance_all_projects
- clustering_groups
- ceph_rbd_du
api_status: stable
api_version: "1.0"
auth: trusted
public: false
auth_methods:
- tls
environment:
  addresses:
  - <IP_ADDRESS>:8443
  architectures:
  - x86_64
  - i686
  certificate: |
    -----BEGIN CERTIFICATE-----
    -----END CERTIFICATE-----
  certificate_fingerprint: 550161c2aa378270121bbe171db32f9d50219e407b9a0aa2fd191108e3616909
  driver: qemu | lxc
  driver_version: 6.2.0 | 4.0.11
  firewall: xtables
  kernel: Linux
  kernel_architecture: x86_64
  kernel_features:
    netnsid_getifaddrs: "true"
    seccomp_listener: "true"
    seccomp_listener_continue: "true"
    shiftfs: "false"
    uevent_injection: "true"
    unpriv_fscaps: "true"
  kernel_version: 5.11.0-40-generic
  lxc_features:
    cgroup2: "true"
    core_scheduling: "true"
    devpts_fd: "true"
    idmapped_mounts_v2: "true"
    mount_injection_file: "true"
    network_gateway_device_route: "true"
    network_ipvlan: "true"
    network_l2proxy: "true"
    network_phys_macvlan_mtu: "true"
    network_veth_router: "true"
    pidfd: "true"
    seccomp_allow_deny_syntax: "true"
    seccomp_notify: "true"
    seccomp_proxy_send_notify_fd: "true"
  os_name: Ubuntu
  os_version: "20.04"
  project: default
  server: lxd
  server_clustered: true
  server_name: node1
  server_pid: 3903145
  server_version: "4.21"
  storage: ceph | btrfs | dir
  storage_version: 16.2.7 | 5.4.1 | 1
  storage_supported_drivers:
  - name: zfs
    version: 2.0.2-1ubuntu5.4
    remote: false
  - name: ceph
    version: 16.2.7
    remote: true
  - name: btrfs
    version: 5.4.1
    remote: false
  - name: cephfs
    version: 16.2.7
    remote: true
  - name: dir
    version: "1"
    remote: false
  - name: lvm
    version: 2.03.07(2) (2019-11-30) / 1.02.167 (2019-11-30) / 4.43.0
    remote: false

Issue description
I evacuated a node for a reboot and restored it after it came back online.
~
root @ node1 # lxc cluster restore node3
Are you sure you want to restore cluster member "node3"? (yes/no) [default=no]: yes
Error: Failed to start instance "k8s-dev": write /var/snap/lxd/common/lxd/virtual-machines/k8s-dev/config/lxd-agent: copy_file_range: no space left on device

the VM k8s-dev lives on ceph.
~
root @ node1 # rbd info LXD/virtual-machine_k8s-dev
rbd image 'virtual-machine_k8s-dev':
        size 95 MiB in 24 objects
        order 22 (4 MiB objects)
        snapshot_count: 0
        id: 1e5d2e267b5d41
        block_name_prefix: rbd_data.1e5d2e267b5d41
        format: 2
        features: layering
        op_features:
        flags:
        create_timestamp: Tue Aug  3 15:15:33 2021
        access_timestamp: Tue Aug  3 15:15:33 2021
        modify_timestamp: Tue Aug  3 15:15:33 2021
        parent: LXD/zombie_image_b1f0967a5c36cf51627a1e99f516c4612fc7ec5595e26cfef43d3e6aca06f35f_ext4@readonly
        overlap: 95 MiB

~
root @ node1 # rbd info LXD/virtual-machine_k8s-dev.block
rbd image 'virtual-machine_k8s-dev.block':
        size 19 GiB in 4769 objects
        order 22 (4 MiB objects)
        snapshot_count: 0
        id: 1e5d284f63913a
        block_name_prefix: rbd_data.1e5d284f63913a
        format: 2
        features: layering
        op_features:
        flags:
        create_timestamp: Tue Aug  3 15:15:33 2021
        access_timestamp: Tue Aug  3 15:15:33 2021
        modify_timestamp: Tue Aug  3 15:15:33 2021
        parent: LXD/zombie_image_b1f0967a5c36cf51627a1e99f516c4612fc7ec5595e26cfef43d3e6aca06f35f_ext4.block@readonly
        overlap: 19 GiB

As you can see, the config image on ceph is 100% full because of the ./state file
~
root @ node1 # rbd -p LXD map virtual-machine_k8s-dev
/dev/rbd4

~
root @ node1 # mount /dev/rbd4 /mnt

~
root @ node1 # df -h /mnt
Filesystem      Size  Used Avail Use% Mounted on
/dev/rbd4        89M   87M     0 100% /mnt

~
root @ node1 # du -sh /mnt/*
4.0K    /mnt/agent-client.crt
4.0K    /mnt/agent-client.key
4.0K    /mnt/agent.crt
4.0K    /mnt/agent.key
4.0K    /mnt/backup.yaml
10M     /mnt/config
4.0K    /mnt/config.mount
16K     /mnt/lost+found
4.0K    /mnt/metadata.yaml
128K    /mnt/qemu.nvram
77M     /mnt/state
28K     /mnt/templates

Additional information:
this VM, k8s-dev, is the first VM on this node. 3 other instances were containers and started without issue.
Another VM that lives on this host+ceph that has not started yet only uses 12% of the config image on ceph and has no ./state file
The difference between the two VMs is that k8s-dev has migration.stateful: "true" in it's config and the other VMs/containers do not.
I used this VM as a test for the new stateful migration feature, but never got it to work. The command would just wait indefinitely. I forgot about it until now. The state file is from November 17th, a little after VM live migration came available and I started testing it.
-rw-r--r-- 1 root root 79880192 Nov 17 09:47 /mnt/state

Is LXD writing the VMs memory to the ceph config image to transfer it to another host? If so, then a 100mb quota isn't going to be enough?
Information to attach

[ ] Any relevant kernel output (dmesg)
[x] Container log (lxc info NAME --show-log)
[x] Container configuration (lxc config show NAME --expanded)
[x] Main daemon log (at /var/log/lxd/lxd.log or /var/snap/lxd/common/lxd/logs/lxd.log)
[ ] Output of the client with --debug
[ ] Output of the daemon with --debug (alternatively output of lxc monitor while reproducing the issue)

Name: k8s-dev
Status: STOPPED
Type: virtual-machine
Architecture: x86_64
Location: node3
Created: 2021/08/03 15:15 CEST
Last Used: 2021/11/24 08:42 CET
Error: open /var/snap/lxd/common/lxd/logs/k8s-dev/qemu.log: no such file or directory

architecture: x86_64
config:
  boot.autostart.delay: "3"
  boot.host_shutdown_timeout: "90"
  cluster.evacuate: stop
  image.architecture: amd64
  image.description: Ubuntu focal amd64 (20210802_07:42)
  image.os: Ubuntu
  image.release: focal
  image.serial: "20210802_07:42"
  image.type: disk-kvm.img
  image.variant: cloud
  limits.cpu: "6"
  limits.memory: 4GB
  limits.memory.enforce: soft
  migration.stateful: "true"
  user.user-data: |
    #cloud-config
    packages:
      - vim
      - htop
      - facter
      - curl
      - ssh
    users:
      - name: ansible
        groups: sudo
        ssh_authorized_keys:
          - ssh-ed25519 AAAAC3NzaC1lZDI1NYQMZN Ansible Automation 08-2019
        sudo: ALL=(ALL) NOPASSWD:ALL
  volatile.base_image: b1f0967a5c36cf51627a1e99f516c4612fc7ec5595e26cfef43d3e6aca06f35f
  volatile.eth0.hwaddr: 00:16:3e:01:21:d7
  volatile.last_state.power: RUNNING
  volatile.uuid: c312ef06-77e9-4ea2-aced-bb26bfb4afa2
  volatile.vsock_id: "156"
devices:
  eth0:
    nictype: bridged
    parent: br121
    type: nic
  root:
    path: /
    pool: ceph
    type: disk
ephemeral: false
profiles:
- default
- limits.medium
- cloud-init
stateful: false
description: ""

The last entry of /var/snap/lxd/common/lxd/logs/lxd.log is that it succesfully started the previous container from the lxc cluster recover  action, so not relevant.

For now, can I safely remove the state file to get my VMs/containers to restore using the lxc cluster restore command?

Right, so your stateful migration never failed, instead it was stuck on I/O due to lack of disk space.
Your best bet is to do lxc config device set INSTANCE root size.state 8GiB or something along those lines which will allow for enough space for stateful stop/snapshots/migration and also fix your current startup problem.

That is a critical oversight on my part :-). It even says so on your newspost! https://discuss.linuxcontainers.org/t/lxd-4-12-has-been-released/10424
Can I request we get an error + automatic clean-up for when this happens instead of a stuck process + a no space left on device at next restart of the VM?
I just changed the size.state device and the VM works now
~
root @ node1 # lxc config device override k8s-dev root
Device root overridden for k8s-dev

~
root @ node1 # lxc config device set k8s-dev root size.state 8GiB

Cheers :-). Now I can also start properly testing/using the live migration feature!

Yeah, I think it'd be reasonable for us to refuse performing stateful stop/snapshots/migration unless size.state is >= size + limits.memory.

Yes that would make it more user friendly to discover that setting.

I think we should do that on startup instead of during config validation as this needs to check:

limits.memory
migration.stateful
size.state on the root device

Mixing that in with profiles and the like could cause a lot of config update failures, so probably best to just fail startup by validating this in Start() of driver_qemu.go


Upgrade select to SDK v2

Rebase + rework of the changes in https://github.com/ahmarsuhail/hadoop/pull/12
A few issues remain open:
Configuration - Since Select is only available on the Async client, we have to instantiate a second client, which leads to some code duplication. Also we need to investigate NTLM support.
SelectInputStream - v1 implementation could simply wrap the getRecordsInputStream() result. In v2, instead, we have to consume an asynchronous event stream and expose it as an input stream. The current implementation may need further work.

@ahmarsuhail, I should have covered all comments. Let's sync up tomorrow.
If you can, have also a look at my refactor attempt here (separate, draft PR to avoid polluting this review): https://github.com/passaro/hadoop/pull/5.


chore: introduce apps/web-tools ported from common-tools

📝 Description

[x] Re-create the web project known as "common-tools" as a Next.js fresh project in apps/web-tools.
[x] Port all the JavaScript into TypeScript (including some type fixes).

Future step:

once this PR is merged, we will do a second PR that completely remove out "common-tools", at the moment it wasn't done yet because we still need Vercel deployment to happen, but we will then switch to apps/web-tools.

❓ Context

Impacted projects: none
Linked resource(s): LIVE-9317

✅ Checklist

[x] Test coverage
[x] Atomic delivery
[x] No breaking changes

📸 Demo

🚀 Expectations to reach
Please make sure you follow these Important Steps.
Pull Requests must pass the CI and be internally validated in order to be merged.

Ad discussed, documenting each tool will be done in a second step soon


Avast Premium Security 24.9.9452 + License Key Till 2050

Avast Premium Security 24.9.9452 + License Key Till 2050
Avast Premium Security License Key is the most powerful antivirus application that provides multiple functions related to internet security, privacy, and performance, designed to protect your computer PC from malware, protect your personal information, and ensure smooth system operation. With cloud-based intelligent detection, it protects your PC, home network, and passwords from zero-second threats.
Download Now

.

Download Updated Link
.
.
.
..### Download Updated Link
.
.
.
..
Download Updated Link
.
.
.
..
Download Updated Link
.
.
.
..


Bump to 2.8.8

Note to self: pushing directly to 2.8.x (or main) is a no-no.

Let's wait for final 2.8.8

@ignasi35 Do you mind if we use my pull request #134? I diffed the two PRs, they are almost identical, expect mine also upgrades to latest Scala versions (2.13.5 and 2.12.13)

@ignasi35 Do you mind if we use my pull request #134? I diffed the two PRs, they are almost identical, expect mine also upgrades to latest Scala versions (2.13.5 and 2.12.13)

Sure thing!


Final update for the release

add default level
add seedlist for building a release

The release build using this codebase was tested by @knicked
It was confirmed it works correctly


util/watch: Add user-defined indexing

From the commit message:

This allows making certain queries more efficient, but leaves the work of doing that for the user of WatchStore.
One such example is implemented in this PR, by providing agent/billing with a WatchStore with an index allowing quick lookup of the VMs for its node.

This feature was implemented for #128 in order to allow efficient "get this VM" lookups in the scheduler plugin. It's necessary there in order to not have to deal with inconsistent state in the plugin extension methods vs the resource request handler.
Of the recent util/watch PRs, this probably affects #65 the most? I'm hoping that the changes from this are not super terrible to port.
Depends on:

[x] #136
[ ] #145

@tychoish re-requested review. note that I merged #145 into here, so there's currently some other commits you might want to ignore for the purposes of reviewing this PR :)


Add past types for Pokémon

This adds the field past_types to Pokemon: https://pokeapi.co/docs/v2#pokemontypepast

This is fantastic! Could set the version to 1.0.4 instead of 1.0.5 please? I'll happy merge and create another release.

Glad to help! I've corrected the version number and will take note of it if I need to make any more pull requests. Thanks for responding to these so quickly (and for his project as a whole)!

Version 1.0.4 is published!
Thank you! I try to follow semantic versioning, though I may not be. I think technically these changes constitute as a minor change instead of a patch change (because they are non-backward compatible changes on the getters/setters). So I think going to 1.0.4 instead of 1.1.0 violates semantic versioning's specifications. I just have a hard time going from 1.0.X to 1.1.0 because of changing the signature of a few getters and setters, lol. So perhaps I'm being a bit picky/inconsistent.
Glad you're finding the project useful. Really appreciate you contributing and fixing my bugs - happy coding!

Great, thanks!
Yea, I understand that. Seems too minor for 1.1.0, even if technically it should be. (Then again, before you got a json-parsing exception on that field, so everything that worked in the previous version technically still works haha.)


Using GStreamer with h264-live-player on Raspberry Pi

Hello, has anyone been able to get GStreamer to work with this?  I would like to request an official example of how to do it, if possible.
   However when modifying raspivid.js, it does not seem to work.  It does activate the camera (the red LED comes on), and if I change the fdsink to a tcpserversink I can connect to it fine, so gst-launch-1.0 IS working, it's just h264-live-player isn't picking up the data from STDOUT (or isn't recognizing it):
get_feed() {
    //var msk = "raspivid -t 0 -o - -w %d -h %d -fps %d";
    //var cmd = util.format(msk, this.options.width, this.options.height, this.options.fps);
    var msk = "gst-launch-1.0 rpicamsrc bitrate=950000 ! video/x-h264,width=800,height=480,framerate=10/1 ! h264parse ! decodebin ! clockoverlay shaded-background=true draw-shadow=true font-desc=\"Nimbus Mono\" ! omxh264enc ! video/x-h264,width=800,height=480,framerate=10/1,profile=high ! h264parse ! queue ! fdsink";
    var cmd = msk;

    console.log(cmd);

    //    var streamer = spawn('/opt/gst/bin/gst-launch-1.0', ['', '0', '-o', '-', '-w', this.options.width, '-h', this.options.height, '-fps', this.options.fps, '-pf', 'baseline']);
    var streamer = spawn('/opt/gst/bin/gst-launch-1.0', ['rpicamsrc', 'bitrate=950000', '!', 'video/x-h264,width=800,height=480,framerate=10/1', '!', 'h264parse', '!', 'decodebin', '!', 'clockoverlay', 'shaded-background=true', 'draw-shadow=true', 'font-desc="Nimbus Mono"', '!', 'omxh264enc', '!', 'video/x-h264,width=800,height=480,framerate=10/1,profile=high', '!', 'h264parse', '!', 'queue', '!', 'fdsink']);
    streamer.on("exit", function(code){
      console.log("Failure", code);
    });

    return streamer.stdout;
  }

Using the server-tcp.js example
I have an almost identical gst pipeline, only this time it outputs the h264 stream to any clients connected to a tcp port:
gst-launch-1.0 rpicamsrc ! video/x-h264,width=800,height=480,framerate=10/1 ! h264parse ! \
decodebin ! omxh264enc ! tcpserversink port=5001 host=<IP_ADDRESS>

If I connect to this TCP stream with VLC or another instance of GStreamer (locally or remotely) it works well, however I also don't get video with h264-live-player when I try with that.

In example 1, it's using the STDOUT file-descriptor to display an h264 stream, and in the second it's using a TCP h264 stream.  I feel like this should be working just fine, but it's not.  I was able to get both examples working just fine with raspivid (as  your standard code includes), but I need to be able to use GStreamer to add overlays and custom processing plugins.
Is there any apparent reason these commands aren't working with h264-live-player?
Thanks!

Btw, if I pipe either the raspivid or the gst-launch-1.0 (using the fdsink, example #1) commands to a .h264 file and then run ffprobe on it, this is waht I get:
pi@TG-BBCAM-01:~/src/h264-live-player $ ffprobe -hide_banner gst.h264; ffprobe -hide_banner raspivid.h264
Input #0, h264, from 'gst.h264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High), yuv420p(progressive), 800x480 [SAR 1:1 DAR 5:3], 25 fps, 25 tbr, 1200k tbn, 50 tbc
Input #0, h264, from 'raspivid.h264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High), yuv420p(progressive), 800x480, 25 fps, 25 tbr, 1200k tbn, 50 tbc

The only noticeable difference is that SAR / DAR thing that Gstreamer's version has...

Solved the issue (finally) with this code.  The main thing seems to be using Profile=baseline.  High and Main would not work for me.  Also, not sure if the '-q' is important, but when I dumped the fdsink output to a file, I noticed it included the text at the top until I added it:
"Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock"
"use strict";

const util      = require('util');
const spawn     = require('child_process').spawn;
const merge     = require('mout/object/merge');

const Server    = require('./_server');

class RpiServer extends Server {

        constructor(server, opts) {
                super(server, merge({
                        fps : 10,
                        width: 800,
                        height:480,
                }, opts));
        }

        get_feed() {
                var bin = '/opt/gst/bin/gst-launch-1.0';
                var args = ['-q',
                        'rpicamsrc', 'bitrate=950000', '!',
                        'video/x-h264,width=800,height=480,framerate=10/1', '!',
                        'h264parse', '!', 'decodebin', '!',
                        'clockoverlay', 'shaded-background=true', 'draw-shadow=true', 'font-desc="Nimbus Mono"', '!',
                        'omxh264enc', '!',
                        'video/x-h264,width=800,height=480,framerate=10/1,profile=baseline', '!',
                        'fdsink'
                ];

                console.log(bin +' '+ args.join(' '));

                var streamer = spawn(bin, args);
                streamer.on("exit", function(code){
                        console.log("Failure", code);
                });

                return streamer.stdout;
        }

};

module.exports = RpiServer;

            var args = ['-q',
                    'rpicamsrc', 'bitrate=950000', '!',
                    'video/x-h264,width=800,height=480,framerate=10/1', '!',
                    'h264parse', '!', 'decodebin', '!',
                    'clockoverlay', 'shaded-background=true', 'draw-shadow=true', 'font-desc="Nimbus Mono"', '!',
                    'omxh264enc', '!',
                    'video/x-h264,width=800,height=480,framerate=10/1,profile=baseline', '!',
                    'fdsink'
            ];

Hello Adam,
do you have an example for gstreamer options that uses the "videotestsrc" as source?
My try:
    var args = [
      "-q",
      "videotestsrc", "is-live=true",
      "!",
      "x264enc",
      "!",
      "video/x-h264,width=640,height=480,framerate=15/1,profile=baseline",
      "!",
      "fdsink",
    ];

If I pipe the stream into a file  (gst-launch-1.0 ... > dump.h264) and use "ffprobe" to validate the file content I get the following output:
$ ffprobe -hide_banner dump.h264
[h264 @ 0x55de4370b120] Format h264 detected only with low score of 1, misdetection possible!
[h264 @ 0x55de4370c940] missing picture in access unit with size 39724171
[AVBSFContext @ 0x55de4370bd60] No start code is found.
dump.h264: Invalid data found when processing input

And no stream is playing with these options. The static video file example works fine.
Many thanks in advance!
Markus

I found a solution for me:
var args = [
      '-q',
      'videotestsrc', 'is-live=true',
      '!',
      'x264enc', 'speed-preset=superfast', 'key-int-max=6',
      '!',
      'video/x-h264,width=640,height=480,framerate=30/1,profile=baseline',
      '!',
      'h264parse',
      '!',
      'video/x-h264,stream-format="byte-stream"',
      '!',
      'fdsink',
    ];


problem with dataload2 chunk

🐛 Describe the bug
Dataloader2 returned DataChunk, and can not be moved to device.
def TFRLoader(path):
    record_pipe = FileLister(path)
    file_pipe = FileOpener(record_pipe, mode="b")
    return file_pipe.load_from_tfrecord().map(tfrecord_praser).batch(batch_size)

rs = MultiProcessingReadingService(num_workers=cfg.num_workers)
train_dataloader = DataLoader2(train_dataset, reading_service=rs)

for i, d in enumerate(train_dataloader):
    d = d.to(device)

AttributeError: 'DataChunk' object has no attribute 'to'
also can not input to model
TypeError: conv2d() received an invalid combination of arguments - got (DataChunk, Parameter, Parameter, tuple, tuple, tuple, int), but expected one of:
 * (Tensor input, Tensor weight, Tensor bias, tuple of ints stride, tuple of ints padding, tuple of ints dilation, int groups)
      didn't match because some of the arguments have invalid types: (DataChunk of [Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor, Tensor,

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

Currently a work around is convert chunk to tensor
d = torch.stack(d.items)

Versions
Latest

You need to make the batch of data to Tensor before sending it to device. This is expected.

You need to make the batch of data to Tensor before sending it to device. This is expected.

thanks for replying.
but according to this tutorial, it seems I don't need to convert.

datapipe = IterableWrapper(["./train1.csv", "./train2.csv"])
datapipe = datapipe.open_files(encoding="utf-8").parse_csv()
datapipe = datapipe.shuffle().sharding_filter()
datapipe = datapipe.map(fn).batch(8)

rs = MultiProcessingReadingService(num_workers=4)
dl = DataLoader2(datapipe, reading_service=rs)
for epoch in range(10):
    dl.seed(epoch)
    for d in dl:
        model(d)
dl.shutdown()

but according to this tutorial, it seems I don't need to convert.

I think it's focusing on using DataLoader (not DataLoader2) to load data from DataPipe. And, DataLoaderdoesn't need to convert because it implicitly do the conversion butDataLoader2doesn't have such ability as we want to provide the flexibility for users to define the pipeline. You can simply dodp.collate()` to do the conversion.
I think we probably need to deprecate the tutorial of loading datapipe via DataLoader.

Thanks, collate() is working.
Besides, I have found that there are many details to pay attention to.
For example, I don't know why the map function is using multiprocessing even though I haven't set it,
And having a good document would be very helpful.


Return 409 for duplicate entity label

Fixes #1200

Not sure about this now. As of rfc 7231 (since superceded by 9110), a 400 response seems correct.


new window event: EventVisible

Whenever a window has changed its visibility property, we call a user
defined callback in order to notify a window has changed its visibility.
It was initially part of PR #87.

Seems I don't have a use for this anymore, no one else has requested as well, so I'm dropping this feature.


[T3A2][T10-A1]Liu Yuheng

Ready for review

Accepting this PR without a review because another of your PRs for this tutorial was reviewed in detail.


Remove -v verbose options from blockchain data management scripts

Change summary:

Remove -v verbose options from blockchain data management scripts

Related issues:

https://github.com/ainblockchain/ain-blockchain/issues/1285

Thanks for the review! @jiyoung-an Now merging..


add serialize_structs compilation feature

this is an attempt at #1254 . I'm new to Rust so I'm open to any and all feedback here :) .  The feature name I chose is serialize_structs but maybe something more specific like serde_serialization is better?
Please help keep the CHANGELOG up to date by providing a one sentence summary of your change:
Adds serialize_structs compilation feature to serialize output structs using serde-derive.

Thanks for the pull request!
I'm unable to compile a service crate with the serialize_structs feature —
error: Package `rusoto_ecr v0.41.0 (/home/ilianaw/git/rusoto/rusoto/services/ecr)` does not have these features: `serialize_structs`

You'll need to modify the Cargo.toml generation to list the feature, alongside the existing native-tls and rustls features, and you'll also want to use the cfg(feature = "serialize_structs") syntax.

I am not a maintainer but just wanted to chime in. Other crates are often using serde as a feature name (for example uuid, chrono or num). See also Rust API guidelines for more details.
In addition serde lets you cleanly hide your own dependency on serge-derive for 3rd party crates that help to name their optional serde dependency simply "serde".
Just my 2 cents.

This feature doesn't toggle serde completely, it just toggles the serialization of some structs. For this reason I think simply serde is a bit of a misleading name.

Thanks for the PR!
I agree with the feature flag naming: since we're not toggling use of serde on or off completely, naming it something close to "enable this optional feature that works with serde" is closer to actual behavior.
Anyone have ideas on how to make this feature flag discoverable? I'm inclined to make an addition to the crate READMEs and/or documentation over writing tests demonstrating it. Doing this makes it discoverable on both Github and docs.rs API docs without incurring extra build times, which are already quite long.

I added a section to the README generation about cargo features. Feel free to chime in on verbiage.

Thanks!


Number of devices 1 must equal the product of mesh_shape (1, 8)？

 What equipment do you need?
grok-1 % python run.py
INFO:jax._src.xla_bridge:Unable to initialize backend 'cuda':
INFO:jax._src.xla_bridge:Unable to initialize backend 'rocm': module 'jaxlib.xla_extension' has no attribute 'GpuAllocatorConfig'
INFO:jax._src.xla_bridge:Unable to initialize backend 'tpu': INTERNAL: Failed to open libtpu.so: dlopen(libtpu.so, 0x0001): tried: 'libtpu.so' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibtpu.so' (no such file), '/opt/anaconda3/bin/../lib/libtpu.so' (no such file), '/usr/lib/libtpu.so' (no such file, not in dyld cache), 'libtpu.so' (no such file), '/usr/local/lib/libtpu.so' (no such file), '/usr/lib/libtpu.so' (no such file, not in dyld cache)
INFO:rank:Initializing mesh for self.local_mesh_config=(1, 8) self.between_hosts_config=(1, 1)...
INFO:rank:Detected 1 devices in mesh
Traceback (most recent call last):
  File "/Users/vjs/grok-1/run.py", line 72, in <module>
    main()
  File "/Users/vjs/grok-1/run.py", line 63, in main
    inference_runner.initialize()
  File "/Users/vjs/grok-1/runners.py", line 282, in initialize
    runner.initialize(
  File "/Users/vjs/grok-1/runners.py", line 181, in initialize
    self.mesh = make_mesh(self.local_mesh_config, self.between_hosts_config)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vjs/grok-1/runners.py", line 586, in make_mesh
    device_mesh = mesh_utils.create_hybrid_device_mesh(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/lib/python3.11/site-packages/jax/experimental/mesh_utils.py", line 373, in create_hybrid_device_mesh
    per_granule_meshes = [create_device_mesh(mesh_shape, granule)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/lib/python3.11/site-packages/jax/experimental/mesh_utils.py", line 373, in <listcomp>
    per_granule_meshes = [create_device_mesh(mesh_shape, granule)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/lib/python3.11/site-packages/jax/experimental/mesh_utils.py", line 302, in create_device_mesh
    raise ValueError(f'Number of devices {len(devices)} must equal the product '
ValueError: Number of devices 1 must equal the product of mesh_shape (1, 8)

https://github.com/xai-org/grok-1/issues/38#issuecomment-2003933119

8 GPUs are required with total VRAM that can fit the model.
Please see: https://github.com/xai-org/grok-1/discussions/183
Please close this issue and move to:
https://github.com/xai-org/grok-1/discussions
Duplicate of / Related to:
#38
Reason:
User's shell issue / Not a real issue
#69
#108


console log pretty

This adds alternative way of printing mod log logs.
This change requires https://github.com/apache/mynewt-newt/pull/551
New syscfg values that can be use to configure console printing
   # enable alternative mod log prints
    LOG_CONSOLE_PRETTIFY: 1
   # enable severity level coloring
    LOG_CONSOLE_PRETTIFY_WITH_COLORS: 1
   # disable time stamp
    LOG_CONSOLE_PRETTIFY_WITH_TIMESTAMP: 0

Old way of showing mod log logs:

New way:

why not just LOG_CONSOLE_PRETTY?
I'd add an option to skip printing log level and e.g. colorize module name instead.
and perhaps an option to print timestamp as seconds, i.e. simply "0.123456" instead of "123456us".

@andrzej-kaczmarek updated as requested


Add links/references

Just so I don't forget - copying from chat (cheers to @blankdots): https://science.sciencemag.org/content/314/5807/1856.summary original source was here: http://archive.euroscipy.org/talk/6634

More links:

https://realpython.com/python-testing/
http://pythontesting.net/start-here/

An article about a glitch in Python code affecting >100 chemistry studies:
https://arstechnica.com/information-technology/2019/10/chemists-discover-cross-platform-python-scripts-not-so-cross-platform/
Original source: https://pubs.acs.org/doi/full/10.1021/acs.orglett.9b03216
Full article: https://drive.google.com/open?id=1kaK4NMGGDwhfDzuCWDDLUjGM5qvlZOjn (CRbase/workshops/2019/hackathon-stockholm-nov-2019/repro-hack/01-juho-python-glob-in-chemistry/)


tls: modify ssl failure reason format

Before:
TLS error: 33554542:system library:OPENSSL_internal:Connection timed out 33554464:system library:OPENSSL_internal:Broken pipe
Now:
TLS error:|33554542:system library:OPENSSL_internal:Connection timed out|33554464:system library:OPENSSL_internal:Broken pipe:TLS_error_end
Commit Message:
Additional Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]

I don't have a strong opinion about this, but can you elaborate on why you're making this change? To me it doesn't seem any better (but not worse either).

Thanks @ggreenway! We are looking at the separator, like before it is only a space for multiple errors, while the error itself may have space, using a new separator looks reasonable to me.

/retest

If the tests continue to fail, you might try merging main to see if its been fixed.

In the past, I don't think we've made any guarantees about the stability of these types of logs messages. I'm worried that whatever you have consuming this is going to want a stronger guarantee than we're willing to make on the content of this message.

Right, do you have some suggestions for this kind of consuming? And that is why we added the runtime guard for this.

I don't have any good advice. More of just a concern that the overall failure reason is pieced together from all over: it comes from TLS (which itself pieces it together from bits in boringssl as well as adding it's own), other transport sockets can add to failure reason, the ConnectionImpl, quic, etc. Any of those could potentially add characters that will break your parsing schema here.

I don't have any good advice. More of just a concern that the overall failure reason is pieced together from all over: it comes from TLS (which itself pieces it together from bits in boringssl as well as adding it's own), other transport sockets can add to failure reason, the ConnectionImpl, quic, etc. Any of those could potentially add characters that will break your parsing schema here.

right, and basically we want to isolate this string from boringssl, and to mitigate the mix from others, we want to add the TLS_error:...TLS_error_end.

I don't have any good advice. More of just a concern that the overall failure reason is pieced together from all over: it comes from TLS (which itself pieces it together from bits in boringssl as well as adding it's own), other transport sockets can add to failure reason, the ConnectionImpl, quic, etc. Any of those could potentially add characters that will break your parsing schema here.

Yea, I think we are doing this in the spirit of best effort with minimum disruption to existing error message.  AFAIU, the potential breakage only happens if there will be other "|"s (that are not added by us) in existing failure_reason_ (i.e., between TLS_error and TLS_error_end) . Do we foresee this? If we think it will be the case, we probably can try to find a different delimiter.

I think it's unlikely someone will add a |. I just want to make sure we agree that this is "best effort".


errors while training

Hi, I finished procaudio.py and now trying to run main.py. But I get this error: could you help with this? Thanks in advance!
WARNING: Logging before flag parsing goes to stderr.
W0805 11:26:09.772453<PHONE_NUMBER>30240 deprecation_wrapper.py:119] From main.py:2: The name tf.set_random_seed is deprecated. Please use tf.compat.v1.set_random_seed instead.
/usr/local/lib/python3.5/dist-packages/numba/errors.py:131: UserWarning: Insufficiently recent colorama version found. Numba requires colorama >= 0.3.9
warnings.warn(msg)
W0805 11:26:10.160339<PHONE_NUMBER>30240 deprecation_wrapper.py:119] From main.py:34: The name tf.app.run is deprecated. Please use tf.compat.v1.app.run instead.
W0805 11:26:10.161276<PHONE_NUMBER>30240 deprecation_wrapper.py:119] From main.py:23: The name tf.GPUOptions is deprecated. Please use tf.compat.v1.GPUOptions instead.
W0805 11:26:10.161381<PHONE_NUMBER>30240 deprecation_wrapper.py:119] From main.py:24: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.
W0805 11:26:10.161485<PHONE_NUMBER>30240 deprecation_wrapper.py:119] From main.py:26: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.
2019-08-05 11:26:10.161679: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-08-05 11:26:10.166440: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcuda.so.1
2019-08-05 11:26:11.349611: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-08-05 11:26:11.360762: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-08-05 11:26:11.361384: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5c75250 executing computations on platform CUDA. Devices:
2019-08-05 11:26:11.361393: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): GeForce GTX 1080, Compute Capability 6.1
2019-08-05 11:26:11.361396: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (1): GeForce GTX 1080, Compute Capability 6.1
2019-08-05 11:26:11.363259: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency:<PHONE_NUMBER> Hz
2019-08-05 11:26:11.363425: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5cf2890 executing computations on platform Host. Devices:
2019-08-05 11:26:11.363436: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): ,
2019-08-05 11:26:11.364043: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-08-05 11:26:11.364481: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties:
name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate(GHz): 1.8095
pciBusID: 0000:01:00.0
2019-08-05 11:26:11.364557: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-08-05 11:26:11.364981: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 1 with properties:
name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate(GHz): 1.8095
pciBusID: 0000:02:00.0
2019-08-05 11:26:11.365079: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open shared object file: No such file or directory
2019-08-05 11:26:11.365144: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory
2019-08-05 11:26:11.365194: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcufft.so.10.0'; dlerror: libcufft.so.10.0: cannot open shared object file: No such file or directory
2019-08-05 11:26:11.365253: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcurand.so.10.0'; dlerror: libcurand.so.10.0: cannot open shared object file: No such file or directory
2019-08-05 11:26:11.365311: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcusolver.so.10.0'; dlerror: libcusolver.so.10.0: cannot open shared object file: No such file or directory
2019-08-05 11:26:11.365380: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library 'libcusparse.so.10.0'; dlerror: libcusparse.so.10.0: cannot open shared object file: No such file or directory
2019-08-05 11:26:11.367616: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
2019-08-05 11:26:11.367631: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1663] Cannot dlopen some GPU libraries. Skipping registering GPU devices...
2019-08-05 11:26:11.367666: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-08-05 11:26:11.367673: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187]      0 1
2019-08-05 11:26:11.367677: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0:   N Y
2019-08-05 11:26:11.367680: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 1:   Y N
W0805 11:26:11.368261<PHONE_NUMBER>30240 deprecation_wrapper.py:119] From /home/users/woody/waveglow/waveglow-tensorflow/src/model.py:16: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.
Time Segments of audio for training: 16384
Total number of parameters: 268294760
Total number of audio/text pair for training: 11790
W0805 11:26:22.729164<PHONE_NUMBER>30240 deprecation_wrapper.py:119] From /home/users/woody/waveglow/waveglow-tensorflow/src/model.py:61: The name tf.train.exponential_decay is deprecated. Please use tf.compat.v1.train.exponential_decay instead.
W0805 11:26:22.734666<PHONE_NUMBER>30240 deprecation_wrapper.py:119] From /home/users/woody/waveglow/waveglow-tensorflow/src/model.py:65: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.
W0805 11:26:22.906246<PHONE_NUMBER>30240 deprecation.py:323] From /home/users/woody/.local/lib/python3.5/site-packages/tensorflow/python/ops/math_grad.py:1205: add_dispatch_support..wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
W0805 11:26:46.083026<PHONE_NUMBER>30240 deprecation_wrapper.py:119] From /home/users/woody/waveglow/waveglow-tensorflow/src/model.py:69: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.
2019-08-05 11:26:55.885543: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set.  If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU.  To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
W0805 11:26:58.329224<PHONE_NUMBER>30240 deprecation_wrapper.py:119] From /home/users/woody/waveglow/waveglow-tensorflow/src/model.py:82: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.
W0805 11:26:58.330414<PHONE_NUMBER>30240 deprecation_wrapper.py:119] From /home/users/woody/waveglow/waveglow-tensorflow/src/model.py:84: The name tf.summary.scalar is deprecated. Please use tf.compat.v1.summary.scalar instead.
W0805 11:26:58.333762<PHONE_NUMBER>30240 deprecation_wrapper.py:119] From /home/users/woody/waveglow/waveglow-tensorflow/src/model.py:88: The name tf.summary.merge is deprecated. Please use tf.compat.v1.summary.merge instead.
Total number of audio/text pair for validation: 1310
Error Loading Model! Training From Initial State...
0%|          | 0/5 [00:00<?, ?it/s]2019-08-05 11:27:02.720775: E tensorflow/core/common_runtime/executor.cc:641] Executor failed to create kernel. Invalid argument: Conv2DCustomBackpropInputOp only supports NHWC.
[[{{node upsample/conv2d_transpose}}]]
Traceback (most recent call last):
File "/home/users/woody/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1356, in _do_call
return fn(*args)
File "/home/users/woody/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1341, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/home/users/woody/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1429, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Conv2DCustomBackpropInputOp only supports NHWC.
[[{{node upsample/conv2d_transpose}}]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/users/woody/waveglow/waveglow-tensorflow/src/model.py", line 159, in sample
output = self.sess.run(self.output, feed_dict={self.mels: [mels]})
File "/home/users/woody/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 950, in run
run_metadata_ptr)
File "/home/users/woody/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1173, in _run
feed_dict_tensor, options, run_metadata)
File "/home/users/woody/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1350, in _do_run
run_metadata)
File "/home/users/woody/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1370, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Conv2DCustomBackpropInputOp only supports NHWC.
[[node upsample/conv2d_transpose (defined at /home/users/woody/waveglow/waveglow-tensorflow/src/ops.py:42) ]]
Errors may have originated from an input operation.
Input Source operations connected to node upsample/conv2d_transpose:
upsample/weights/read (defined at /home/users/woody/waveglow/waveglow-tensorflow/src/ops.py:19)
ExpandDims (defined at /home/users/woody/waveglow/waveglow-tensorflow/src/ops.py:54)
Original stack trace for 'upsample/conv2d_transpose':
File "main.py", line 34, in
tf.app.run()
File "/home/users/woody/.local/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "/home/users/woody/.local/lib/python3.5/site-packages/absl/app.py", line 300, in run
_run_main(main, args)
File "/home/users/woody/.local/lib/python3.5/site-packages/absl/app.py", line 251, in _run_main
sys.exit(main(argv))
File "main.py", line 27, in main
model = WaveGlow(sess)
File "/home/users/woody/waveglow/waveglow-tensorflow/src/model.py", line 13, in init
self.build_model()
File "/home/users/woody/waveglow/waveglow-tensorflow/src/model.py", line 20, in build_model
self.conditions = mydeconv1d(self.mels, args.n_mel, filter_size=args.step_per_mel*4, stride=args.step_per_mel, scope='upsample', reuse=False)
File "/home/users/woody/waveglow/waveglow-tensorflow/src/ops.py", line 55, in mydeconv1d
ret = mydeconv2d(inputs, output_channel, filter_size, strides, scope, reuse)
File "/home/users/woody/waveglow/waveglow-tensorflow/src/ops.py", line 42, in mydeconv2d
deconv = tf.nn.conv2d_transpose(inputs, weights, output_shape=output_shape, strides=strides, data_format="NCHW")
File "/home/users/woody/.local/lib/python3.5/site-packages/tensorflow/python/ops/nn_ops.py", line 2147, in conv2d_transpose
name=name)
File "/home/users/woody/.local/lib/python3.5/site-packages/tensorflow/python/ops/nn_ops.py", line 2218, in conv2d_transpose_v2
name=name)
File "/home/users/woody/.local/lib/python3.5/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 1407, in conv2d_backprop_input
name=name)
File "/home/users/woody/.local/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
op_def=op_def)
File "/home/users/woody/.local/lib/python3.5/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/home/users/woody/.local/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 3616, in create_op
op_def=op_def)
File "/home/users/woody/.local/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2005, in init
self._traceback = tf_stack.extract_stack()
0%|          | 0/2947 [00:00<?, ?it/s]

deconv = tf.nn.conv2d_transpose(inputs, weights, output_shape=output_shape, strides=strides, data_format=“NCHW”)
InvalidArgumentError (see above for traceback): Conv2DCustomBackpropInputOp only supports NHWC
I've got the same error

 deconv = tf.nn.conv2d_transpose(inputs, weights, output_shape=output_shape, strides=strides, data_format="NCHW")         deconv2 = tf.transpose(deconv, [0, 3, 1, 2])         return tf.nn.bias_add(deconv2, biases, data_format="NCHW")

https://www.tensorflow.org/api_docs/python/tf/nn/conv2d_transpose I think updated tf version supports it .
data_format: A string. 'NHWC' and 'NCHW' are supported.


[Bug]: dependency on root //:node_modules breaks transitive deps

What happened?
See https://github.com/macourteau/aspect-tsproject-imports/tree/f008b8da3f4a5ad0e51a11da150f9171e58bf072
Run bazel run //demo/web/apps/my-web-app. The transitive deps of express are not available even though the full root node_modules is present in the runfiles. Removing this extra dep fixes it?
Version
bazel: 6.3.1
rules_js: 1.31.0
rules_ts: 1.4.5

This does seem unexpected, and it would be good for us to understand why it happens.

Removing this extra dep fixes it?

It does fix the issue, however if I then add a dep on e.g. express in my-web-app (example), then I run into the same issue again (node_modules is incomplete), and I haven't found a workaround for that yet.
Basically, it seems like depending on something, say A, that a dependency, say B, also depends on doesn't bring in the deps for A, if it has any. I've tested this with packages other than express which also have dependencies, and the outcome is the same.

Seems like commenting out these two lines fixes the other issue I'm seeing: https://github.com/macourteau/aspect-tsproject-imports/blob/d1fb7ac139d7d673a6184d86d480ed4fe0a87e44/.bazelrc#L78-L79
I guess it would probably be good to figure out why it happens, but at least I'm able to build now. :)


Limit displayed array items

Have any way to limit number of displayed array items? Example, i have an array with 5000 items, but, only need to dump first 1000. And maybe a last line with an information like "More 4000 Itens.'
Thx.

Hmm... I could add something to the blacklist plugin to set an upper limit on array length, and then make it shallow so you can dump it directly to see more. It won't show you the first 1000 items, but it will show you how many are in there

Sorry for the delay. @lordskynner I've added the array size limit to the blacklist plugin on my test branch at 27ac8a9292fc17240dad468cfa24a001eed1a875
Does this work?

Pushed to kint-php with 1k/10k defaults


💅 Style / Alert ribbon

👋 Thanks for creating a pull request! 🚀 Checkout the storybook we've created for it: chlorophyll


Add ability to mask secrets when exec'ing

You mean like the masking in Gitlab pipeline, that any output containing the secret will be replaced?

Yes, exactly that.


Removed space between '###' and section heading

Section heading wasn't getting displayed correctly due to the missing space.

This seems like a small (but important) contribution, so no Contribution License Agreement is required at this point. We will now review your pull request.
Thanks,
Microsoft Pull Request Bot

take or close

Dupe of #4762


ZQS-512 remove z-quantum-actions subtree

The error should be fixed by #59. Leaving the branch hanging for a while to let the workflows complete cc @araza6 @MSRudolph


Add a style fallback to avoid invalid inline styles

Closes #7387
This is mostly copying the behaviour of https://github.com/springload/draftjs_exporter/blob/v2.1.7/example.py#L130 - if an inline style has been added which isn't valid with the site configuration, then the style will be dropped and the contents will be kept.
To test:

Try and hack in an underline block (similar to #7387)
A couple of warnings will appear on save:

WARNING:root:Missing config for "UNDERLINE". Deleting style.
WARNING:root:Missing config for "UNDERLINE". Deleting style.
[01/Aug/2021 21:42:01] "POST /admin/pages/4/edit/ HTTP/1.1" 302 0

Content will appear without inline style:

@tomkins thanks for investigating this and providing a fix with unit tests also, really appreciate it.
I have pushed a rebased version of this to your branch (along with release notes) and will merge in shortly.


Adding Diagnose elements to Download operations

Adding Diagnostic elements to download call recording operations.

API change check
API changes are not detected in this pull request.


checkpoint markers

Not quite working, but trying to illustrate what I mean in https://github.com/observablehq/plot/pull/2094#pullrequestreview-2124039917. Could probably still be improved…

Why do you say it's not quite working? It looks good to me :) (And the implementation is much nicer.)