better table formatting for ab-benchmarking output

I found the output of the ab benchmarking output hard to read so I wrote some code to format it nicely. The output from this change looks like the following:
                   Score                     Average A (noise)   Average B (noise) Speed-up
average_frame_build_time_millis                   1.77 (1.43%)        1.85 (1.86%)   0.95x
worst_frame_build_time_millis                     5.68 (9.67%)        6.64 (5.48%)   0.86x
90th_percentile_frame_build_time_millis           2.59 (0.04%)        2.41 (0.35%)   1.07x
99th_percentile_frame_build_time_millis           4.66 (3.49%)        5.32 (1.62%)   0.88x
average_frame_rasterizer_time_millis              4.60 (0.45%)        3.79 (0.92%)   1.22x
worst_frame_rasterizer_time_millis               10.44 (18.48%)       8.98 (1.91%)   1.16x
90th_percentile_frame_rasterizer_time_millis      4.98 (0.57%)        4.04 (0.90%)   1.23x
99th_percentile_frame_rasterizer_time_millis      5.98 (3.55%)        7.67 (6.52%)   0.78x
average_vsync_transitions_missed                  0.50 (100.00%)      0.00 (0.00%) Infinityx
90th_percentile_vsync_transitions_missed          0.50 (100.00%)      0.00 (0.00%) Infinityx
99th_percentile_vsync_transitions_missed          0.50 (100.00%)      0.00 (0.00%) Infinityx

It occurs to me that it isn't clear from the output what A and B are. That could be another enhancement, but I just wanted to get the table readable...

My initial goal was to make the table copy'n'pasteable into google spreadsheets. Will it still work after this change?

My initial goal was to make the table copy'n'pasteable into google spreadsheets. Will it still work after this change?

Ah. No, it won't. I could look into making it an option. Note that some of the fields between tabs were N.NN (N.NN%). How would that look pasted into sheets?
I'll definitely wait until you write the tests. Also, I could write this as a standalone formatting utility - something like dart lib/frameworkds/abformatter <stdin>

A small hack to both satisfy human eyes and Google spreadsheet is to append a '\t' at the end of each column in addition to the aligned spaces.

I imagine reading on the command-line will be a more common use-case. So I think using @flar's formatting by default and adding a --use-spreadsheet-formatting flag as an opt-in would be good.

...the --use-spreadsheet-formatting flag could also change N.NN (N.NN%) to N.NN\tN.NN\t and remove the x in the speed-up number so you can sort and conditionally format the spreadsheet.

How about just output all of the raw data (tab separated perhaps) and then having a utility to format it that is easy to run. The output could end with a sample command line and you just have to paste the command line, hit return, then the raw data (as input), hit return, and out pops the data you want.
Ascii could be one option.
HTML with Javascript for google charts could be another.
The raw data could be pasted into a google sheet.
Is there a way to output a google sheet format (preferably ascii?) that has the chart already created with nice output options?

@flar @yjbanov : how do you think about just adding a \t by the end of spaces, for example
'AB     \t 1'
'XYZ    \t 2'

@flar my PR went in. Hack away!

@yjbanov : my concern with --use-spreadsheet-format is that one could spend one hour running the test without it, and then that person don't want to spend another hour to run the test again with that flag. Having a \t in addition to spaces is still recognizable by Google spreadsheet, and human eyes won't be able to differentiate \t or  .

Another idea is to just output it twice. Once with tabs (including separating the noise out) for spreadsheet import and another for readability.
Also, what about outputting a file with the raw data in csv or json format and then printing a human readable summary on the screen. I'm not sure if you can input json into a sheet, but with the raw data in csv you can always do whatever you want with the data, and you have it saved if you do a bunch of test runs.

(PR triage): Are there still plans for this PR?


Add support for custom tags and GOBLET_ARTIFACT_TAG env var

solves https://github.com/goblet/goblet/issues/433 and https://github.com/goblet/goblet/issues/432

Awesome! thank you!


Update integrate-with-nat-gateway.md

NAT Gateway does not, today, support zone-spanning VMSS inside of a single subnet. https://feedback.azure.com/forums/217313-networking/suggestions/40030021-zone-redundant-nat-gateway

@adstuart : Thanks for your contribution! The author(s) have been notified to review your proposed change.


xref.CrossReferences: return filtered NodeInfos for requested tickets [phab:D2105]

Created by schroederc at 2017-12-20 22:14:08:

danielmoy-google commented inline (context lost during migration) at 2017-12-20 22:21:27:
Maybe I missed something here, but where is cr.source_node ever set outside of a test?

danielmoy-google accepted this revision at 2017-12-20 22:21:27

schroederc commented inline (context lost during migration) at 2017-12-20 22:26:39:
It would be populated by whatever post-processing pipeline (e.g. write_tables) is being used to make the serving data.

schroederc closed via commit 72b05f252cd0439ed5df303df454c5cff124dcaa at 2017-12-21 17:58:26


feat: add imported RDS database secrets management workflows

Description of changes

Adds support for amplify api update-secrets customer workflow that enables them to modify the database user secrets if and when needed for the imported RDS database. Common use-cases would be to use this command to modify the user secrets after checking out a different Amplify environment in your app, if you wish to have separate users for different Amplify environments.
Adds support for Multi-environment use-cases to manage the imported RDS database user credentials.

CDK / CloudFormation Parameters Changed

NA
Issue #, if available

NA
Description of how you validated changes
unit tests and using a test JS sample app
Checklist

[x] PR description included
[x] yarn test passes
[x] Tests are changed or added
[ ] Relevant documentation is changed or added (and PR referenced)
[ ] New AWS SDK calls or CloudFormation actions have been added to relevant test and service IAM policies
[ ] Any CDK or CloudFormation parameter changes are called out explicitly

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report

Merging #1176 (eacc6a5) into feature/rds-support (eacc6a5) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head eacc6a5 differs from pull request most recent head 19583f7. Consider uploading reports for the commit 19583f7 to get more accurate results

@@                 Coverage Diff                  @@
##           feature/rds-support    #1176   +/-   ##
====================================================
  Coverage                63.51%   63.51%
====================================================
  Files                      296      296
  Lines                    18863    18863
  Branches                  4573     4573
====================================================
  Hits                     11981    11981
  Misses                    6269     6269
  Partials                   613      613

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


fix activation dtype

What does this PR do?
Fix the activation type for quantizing for tensorrt
Fixes
ValueError: Invalid combination of use_static_quantization = False and activations_dtype = QuantType.QInt8. OnnxRuntime dynamic quantization requires activations_dtype = QuantType.QUInt8

Before submitting

[ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
[ ] Did you make sure to update the documentation with your changes?
[ ] Did you write any new necessary tests?

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

@philschmid @JingyaHuang should we put this in? Feels like a bug


Calculate code complexity for PhoneNumberUtil::formatNumberForMobileDialing

lizard gives NLOC=48, CCN=24, length=79

I hand-calculated the complexity as 24. Refer to commit 911451e0d934f09d741d13021fa1b2cd0f1ae5a8.
Notes on calculating complexity:

Ternary return and assignment both count as 1
A boolean assignment also add to final value when there's more than 1 predicate and linked with boolean operators
Simply count the predicates in the if condition

Answers to 3.4:

I got 24 as the result, which is different from other's. The criteria for calculating the complexity is not so clear, as I didn't find an official document for this. My result is the same as lizard.
Not necessarily. This function only has 48 lines of code.
The function is for formatting a phone number in a way that it can be dialed from a mobile phone in a specific region. Phone numbers in different regions may have different formats, and there are some special cases like political factors. These rules are hard to be generalized, so the function can easily has high CC by writing a lot of if statement.
This function does not throw any exceptions. We can view try...catch as special if statements, which can either lead to early return of continuation of the code. The CC will be higher as dealing with exceptions increase the logical complexity. (lizard treat the exception handling by counting the number of catch, or the types of exceptions)
The function doc describes its general usage, and what will be returned on success and failure. In each branch, there are comments on when will the branch be executed, and what will be applied then.

I got 22 when doing by hand.

I'm getting 25 when doing it by hand

24


Why are sidebar icons hardcoded as unloaded

Checklist

[x] I have read the newlest tutorials and know the correct effect of the functional design.
[x] There is no similar question on existing issues (including closed ones).
[x] I have tried to find the answer on Jekyll Forum and StackOverflow.
[x] My question is based on the latest code of the master branch.

Description
It seems like Chirpy is ready to display icons next to tabs in sidebar by default

But those icons have a hardcoded class "unloaded" that just gives them display: none !important;
https://github.com/cotes2020/jekyll-theme-chirpy/blob/master/_includes/sidebar.html
{% for tab in site.tabs %}

    {% capture tab_name %}{{ tab.url | split: '/' }}{% endcapture %}

    {{ site.data.locales[site.lang].tabs.[tab_name] | default: tab.title | upcase }}

{% endfor %}

Can this be controllable from the configuration?
Or maybe show the icon only when tab.icon is explicitly set?

I think they show up depending on screen resolution / how big the screen is? On my laptop, I don't see the icons unless I zoom out on the page, but on my monitor, I do.

But those icons have a hardcoded class "unloaded" that just gives them display: none !important;

Not always. It depends on your screen resolution.
https://github.com/cotes2020/jekyll-theme-chirpy/blob/1fd665bf4990c26ae23635c511c5abc9640184d1/_sass/addon/commons.scss#L1532
https://github.com/cotes2020/jekyll-theme-chirpy/blob/1fd665bf4990c26ae23635c511c5abc9640184d1/_sass/addon/commons.scss#L1617-L1630


Fixes preconditioner Benchmark

This PR fixes the preconditioner benchmark by storeing only encoded preconditioner names. The benchmark can run successfully in the current state, but it will emit some exceptions.

@upsj @yhmtsai I've added now a test, that would fail with the previous implementation,

Can confirm :)


Translate styled text

Need to externalize and style a more complicated string:
Join us to get registered, get educated, get social, and get vocal.

Reference: https://vue-i18n.intlify.dev/guide/advanced/component.html#component-interpolation
@davidnixon want to take this one?
FYI @demilolu @Sabine-Justilien @cstefano1

@upkarlidder I'm interested

Thank you @IfDougelseSa 🙏🏽 . Are you able to get the site running locally? Let us know if you run into issues. And then feel free to submit a PR for review.
You will have to remove the commented out language switcher to see the translations
https://github.com/Call-for-Code-for-Racial-Justice/Five-Fifths-Voter/blob/4c5cd749b1a48046be7bd80be8e73e89babc6572/ui/src/App.vue#L3
There is another issue to move the language switcher to the right avatar menu.

@IfDougelseSa, we had a PR for this by @davidnixon. Please review when you get a chance.
I did not realize you can add a markdown component. Good to know!
@IfDougelseSa, please feel free to look at another issue. I think this was just a timing issue where you both were working on the same thing. We would still very much like your help with the other issues.
FYI @demilolu

@IfDougelseSa this is another UI one if you are interested - https://github.com/Call-for-Code-for-Racial-Justice/Five-Fifths-Voter/issues/213


connect mongodb cluster error, can you provide a case? Thank you.

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

@yuanshanhshan Hi，The official docs for rk-db/mongodb is missing currently，I will add the official docs ASAP。

@yuanshanhshan Hi, MongoDB docs added into https://docs.rkdev.info/docs/rk-boot/getting-started/database/mongodb/ official site.
If there are still errors occur, please paste your errors in this thread, I will look into that. Thanks

Thanks


Setting base color texture cause error

Current Behavior
Setting a texture to Base Color causes error. - not shown
Expected Behavior
to add texture to texture node
Steps To Reproduce
Open model
migrate standard texture (has standard texture already) will not carry over.
set the texture to the required texture get error.
Traceback (most recent call last):
File "C:\Users\Ron\AppData\Roaming\Blender Foundation\Blender\3.1\scripts\addons\io_scene_gltf2_msfs\blender\msfs_material_prop_update.py", line 128, in update_base_color_texture
msfs.setBaseColorTex(self.msfs_base_color_texture)
AttributeError: 'NoneType' object has no attribute 'setBaseColorTex'
File "C:\Users\Ron\AppData\Roaming\Blender Foundation\Blender\3.1\scripts\addons\io_scene_gltf2_msfs\blender\msfs_material_prop_update.py", line 123, in update_base_color_texture
Traceback (most recent call last):
File "C:\Users\Ron\AppData\Roaming\Blender Foundation\Blender\3.1\scripts\addons\io_scene_gltf2_msfs\blender\msfs_material_prop_update.py", line 128, in update_base_color_texture
msfs.setBaseColorTex(self.msfs_base_color_texture)
AttributeError: 'NoneType' object has no attribute 'setBaseColorTex'
File "C:\Users\Ron\AppData\Roaming\Blender Foundation\Blender\3.1\scripts\addons\io_scene_gltf2_msfs\blender\msfs_material_prop_update.py", line 123, in update_base_color_texture
Environment
- OS:win 10
- Blender:3.1
- glTF-Blender-IO-MSFS:1.1.2

Anything else?
No response

Here migrate material works fine
That .001 in the texture name Is suspicious, Is your texture name really that?
Usually a .001 come from copying textured meshes between files, and cause missing textures in the Sim
Just my 2 cents 😊

Delete your actual material and create a new one, it should work correctly. It happen when you copy a mesh from an other .blend file.

@ronh991 Does the above solve your issue?

Guys this is due to the migration of materials.  The 001 was me doing things numerous times. trying to add the material manually after the migration.
This is the error with migrate.

this is the error after trying to add the texture myself
location: :-1
Traceback (most recent call last):
File "C:\Users\Ron\AppData\Roaming\Blender Foundation\Blender\3.1\scripts\addons\io_scene_gltf2_msfs\blender\msfs_material_prop_update.py", line 128, in update_base_color_texture
msfs.setBaseColorTex(self.msfs_base_color_texture)
AttributeError: 'NoneType' object has no attribute 'setBaseColorTex'
File "C:\Users\Ron\AppData\Roaming\Blender Foundation\Blender\3.1\scripts\addons\io_scene_gltf2_msfs\blender\msfs_material_prop_update.py", line 123, in update_base_color_texture
Traceback (most recent call last):
File "C:\Users\Ron\AppData\Roaming\Blender Foundation\Blender\3.1\scripts\addons\io_scene_gltf2_msfs\blender\msfs_material_prop_update.py", line 128, in update_base_color_texture
msfs.setBaseColorTex(self.msfs_base_color_texture)
AttributeError: 'NoneType' object has no attribute 'setBaseColorTex'
File "C:\Users\Ron\AppData\Roaming\Blender Foundation\Blender\3.1\scripts\addons\io_scene_gltf2_msfs\blender\msfs_material_prop_update.py", line 123, in update_base_color_texture

fixed in migration fix branch


repo: Add cleanup.sh for x64 release job

as space appears to be running out for this job
Signed-off-by: Ryan Northey<EMAIL_ADDRESS>Commit Message:
Additional Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]

cc @mattklein123 @keith

Checking the diskspace at the end of this job:
Before:
/dev/sda1        84G   73G   11G  87% /

After
/dev/sda1        84G   66G   18G  79% /


Video Classification lr_schedulers

🐛 Bug
Discussed in https://github.com/PyTorchLightning/lightning-flash/discussions/1223

Originally posted by dudeperf3ct March  8, 2022
What is best way to access length of train_dataloader?
import torch

import flash
from flash.core.data.utils import download_data
from flash.video import VideoClassificationData, VideoClassifier

# 1. Create the DataModule
# Find more datasets at https://pytorchvideo.readthedocs.io/en/latest/data.html
download_data("https://pl-flash-data.s3.amazonaws.com/kinetics.zip", "./data")

datamodule = VideoClassificationData.from_folders(
    train_folder="data/kinetics/train",
    val_folder="data/kinetics/val",
    clip_sampler="uniform",
    clip_duration=1,
    decode_audio=False,
    batch_size=1,
)

# 2. Build the task
model = VideoClassifier(backbone="x3d_xs",
                        labels=datamodule.labels,
                        pretrained=True,
                        lr_scheduler=('onecyclelr', {"max_lr" : 1e-3}))

# 3. Create the trainer and finetune the model
trainer = flash.Trainer(
    max_epochs=1, gpus=torch.cuda.device_count(), strategy="ddp" if torch.cuda.device_count() > 1 else None
)
trainer.finetune(model, datamodule=datamodule, strategy="freeze")

Using OneCycleLR requires to specify either total_steps or both epochs and steps_per_epoch. There's a get_num_training_steps() to obtain total steps but how can we use it in above example?
Bug Report:
Traceback (most recent call last):
  File "test_kinetics.py", line 43, in <module>
    trainer.finetune(model, datamodule=datamodule, strategy="freeze")
  File "/opt/conda/lib/python3.8/site-packages/flash/core/trainer.py", line 161, in finetune
    return super().fit(model, train_dataloader, val_dataloaders, datamodule)
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 740, in fit
    self._call_and_handle_interrupt(
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 685, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 777, in _fit_impl
    self._run(model, ckpt_path=ckpt_path)
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1145, in _run
    self.accelerator.setup(self)
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/accelerators/gpu.py", line 46, in setup
    return super().setup(trainer)
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/accelerators/accelerator.py", line 93, in setup
    self.setup_optimizers(trainer)
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/accelerators/accelerator.py", line 354, in setup_optimizers
    optimizers, lr_schedulers, optimizer_frequencies = self.training_type_plugin.init_optimizers(
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 245, in init_optimizers
    return trainer.init_optimizers(model)
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/optimizers.py", line 35, in init_optimizers
    optim_conf = self.call_hook("configure_optimizers", pl_module=pl_module)
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1501, in call_hook
    output = model_fx(*args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/flash/core/model.py", line 500, in configure_optimizers
    return [optimizer], [self._instantiate_lr_scheduler(optimizer)]
  File "/opt/conda/lib/python3.8/site-packages/flash/core/model.py", line 756, in _instantiate_lr_scheduler
    lr_scheduler: Union[_LRScheduler, Dict[str, Any]] = lr_scheduler_fn(optimizer, **lr_scheduler_kwargs)
  File "/opt/conda/lib/python3.8/site-packages/torch/optim/lr_scheduler.py", line 1174, in __init__
    raise ValueError("You must define either total_steps OR (epochs AND steps_per_epoch)")
ValueError: You must define either total_steps OR (epochs AND steps_per_epoch)

Replacing onecyclelr with hugging face scheduler (installed transformers before running)
lr_scheduler=('cosine_schedule_with_warmup', {"num_warmup_steps" : 0.1})

Bug report:
Traceback (most recent call last):
  File "test_kinetics.py", line 43, in <module>
    trainer.finetune(model, datamodule=datamodule, strategy="freeze")
  File "/opt/conda/lib/python3.8/site-packages/flash/core/trainer.py", line 161, in finetune
    return super().fit(model, train_dataloader, val_dataloaders, datamodule)
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 740, in fit
    self._call_and_handle_interrupt(
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 685, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 777, in _fit_impl
    self._run(model, ckpt_path=ckpt_path)
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1145, in _run
    self.accelerator.setup(self)
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/accelerators/gpu.py", line 46, in setup
    return super().setup(trainer)
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/accelerators/accelerator.py", line 93, in setup
    self.setup_optimizers(trainer)
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/accelerators/accelerator.py", line 354, in setup_optimizers
    optimizers, lr_schedulers, optimizer_frequencies = self.training_type_plugin.init_optimizers(
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 245, in init_optimizers
    return trainer.init_optimizers(model)
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/optimizers.py", line 35, in init_optimizers
    optim_conf = self.call_hook("configure_optimizers", pl_module=pl_module)
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1501, in call_hook
    output = model_fx(*args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/flash/core/model.py", line 500, in configure_optimizers
    return [optimizer], [self._instantiate_lr_scheduler(optimizer)]
  File "/opt/conda/lib/python3.8/site-packages/flash/core/model.py", line 744, in _instantiate_lr_scheduler
    num_training_steps: int = self.get_num_training_steps()
  File "/opt/conda/lib/python3.8/site-packages/flash/core/model.py", line 629, in get_num_training_steps
    dataset_size = len(self.train_dataloader())
  File "/opt/conda/lib/python3.8/site-packages/pytorch_lightning/core/hooks.py", line 477, in train_dataloader
    raise NotImplementedError("`train_dataloader` must be implemented to be used with the Lightning Trainer")
NotImplementedError: `train_dataloader` must be implemented to be used with the Lightning Trainer

Versions
pl/flash/transformers/torch : 1.5.10 / 0.7.1 / 4.17.0 / 1.9.0

Hey @dudeperf3ct thanks for reporting this! I'm not sure it's possible for us to support these types of schedulers for video classification as it uses iterable datasets that don't have a defined length. Let us know if you are aware of any workarounds we could try. Otherwise I will leave this open as a request for a better error message in these cases.
Hope that helps :smiley:

Thank you! Can you help me understand why we cannot use get_num_training_steps() method to get total number of training steps for onecyclelr? Is there a better way to get total number of training steps for e.g. in pytorch we can get it using len(train_dataloader).


iterator: NewRootLookuper() and cleanup

I had changed Iterate to not call dns.Fqdn because Lookup() is defined to use qualified names, but Iterate itself breaks that rule

I had sneaked a commit that deserves a dedicated PR


FAQ Tweaks, Including Root Chain FAQ + Terminology

:rotating_light: Best reviewed on a per-commit basis, each commit is self-contained.
The root chain FAQ was suggested here
I figured it would also be useful to add a page explaining some of the terminology used in TLS, too - especially terms which are frequently misused or can cause confusion.

I've read through the doc again and I think this is a great addition to the website and will be really useful for folks- thanks for this!
It looks like all the comments have been addressed too.
/approve
/lgtm


Installed, now mainsail webgui stick on Initializing

I've followed the installation steps to install the companion onto my RPi hosted Klipper/Mainsail (fully updated versions), however now I am not able to load the webgui - I'm stuck on the webpage saying "Initializing" and ERROR in the top of the browser.
I didn't see any errors during the installation. I have also added the companion update info to moonraker.conf
Fortunately I do still have SSH onto the Pi. When I inspect the moonraker.log I'm now seeing a repeat of this:

2023-12-30 00:06:19,440 [websockets.py:open()] - Websocket Opened: ID:<PHONE_NUMBER>, Proxied: False, User Agent: , Host Name: <IP_ADDRESS>
2023-12-30 00:06:19,446 [common.py:build_error()] - JSON-RPC Request Error - Requested Method: server.connection.identify, Code: 400, Message: Agent 'OctoEverywhere' already registere>
2023-12-30 00:06:19,449 [common.py:build_error()] - JSON-RPC Request Error - Requested Method: server.database.post_item, Code: -31000, Message: Attempt to operate on closed/deleted/d>
2023-12-30 00:06:19,456 [websockets.py:on_close()] - Websocket Closed: ID:<PHONE_NUMBER> Close Code: 1000, Close Reason: None, Pong Time Elapsed: 0.02

octoeverywhere.log is repeating this:

2023-12-30 00:09:53,628 - INFO - Moonraker client websocket shutdown called.
2023-12-30 00:09:53,630 - INFO - Moonraker client websocket shutdown complete.
2023-12-30 00:10:13,647 - INFO - Moonraker websocket connection closed.
2023-12-30 00:10:13,648 - INFO - Moonraker client websocket connection lost. We will try to restart it soon.
2023-12-30 00:10:15,653 - INFO - Connecting to moonraker: ws://<IP_ADDRESS>:7125/websocket
2023-12-30 00:10:15,660 - INFO - Websocket connected
2023-12-30 00:10:15,662 - INFO - Moonraker client websocket opened.
2023-12-30 00:10:15,662 - INFO - Moonraker client waiting for klippy ready...
2023-12-30 00:10:15,663 - INFO - Authenticating with moonraker...
2023-12-30 00:10:15,669 - ERROR - Ensure database entry item post failed. -31000 - Attempt to operate on closed/deleted/dropped object.
2023-12-30 00:10:15,670 - INFO - Moonraker client waiting for klippy ready...
2023-12-30 00:10:15,673 - ERROR - Moonraker client is in an unknown klippy waiting state. state 'disconnected'
2023-12-30 00:10:15,673 - ERROR - Moonraker client exception in klippy waiting logic.; Exception Exception: Unknown klippy waiting state. disconnected; Traceback (most recent call las>
File "/home/tim/octoeverywhere/moonraker_octoeverywhere/moonrakerclient.py", line 588, in _AfterOpenReadyWaiter
raise Exception(f"Unknown klippy waiting state. {state}")
Exception: Unknown klippy waiting state. disconnected

And mobileraker.log is showing this after the last restart:

--------------------MobilerakerCompanion Log Start--------------------
Git Version: v0.4.0-37-gf65b679
2023-12-30 00:10:22,192 root INFO - MobilerakerCompanion version: v0.4.0-37-gf65b679
2023-12-30 00:10:22,193 root INFO - Passed config file is: /home/tim/Mobileraker.conf
2023-12-30 00:10:22,194 root INFO - Checking mobileraker_companion dir
2023-12-30 00:10:22,194 root INFO - Checking klipper_config dir
2023-12-30 00:10:22,195 root INFO - Checking printer_data/config dir
2023-12-30 00:10:22,195 root INFO - Checking user dir
2023-12-30 00:10:22,196 root WARNING - No config file was found, using default fallback values!
2023-12-30 00:10:22,196 root INFO - Read 1 printer config sections
2023-12-30 00:10:22,261 root INFO - Main section read, language:"en", timezone:"Australia/Brisbane", eta_format:"%d.%m.%Y, %H:%M:%S", include_snapshot:"True"
2023-12-30 00:10:22,263 mobileraker.jrpc INFO - Trying to connect to: ws://<IP_ADDRESS>:7125/websocket api key
2023-12-30 00:10:22,274 mobileraker.jrpc INFO - WebSocket connected
2023-12-30 00:10:22,275 mobileraker.jrpc INFO - Notifying listeners about connection state True
2023-12-30 00:10:22,276 mobileraker.sync INFO - Doing a (Re)Sync with moonraker
2023-12-30 00:10:22,277 mobileraker.sync INFO - Syncing klippy Objects
2023-12-30 00:10:22,295 mobileraker.sync INFO - Received server info: {'jsonrpc': '2.0', 'result': {'klippy_connected': False, 'klippy_state': 'disconnected', 'components': ['klippy_c>
2023-12-30 00:10:22,297 mobileraker.sync WARNING - Klippy was not ready. Trying resync again in 2 seconds...
2023-12-30 00:10:22,298 mobileraker.jrpc WARNING - Error message received from WebSocket-Server Attempt to operate on closed/deleted/dropped object.
2023-12-30 00:10:22,300 mobileraker._Default WARNING - Could not write companion meta into moonraker database, moonraker returned error Attempt to operate on closed/deleted/dropped ob>
2023-12-30 00:10:24,298 mobileraker.sync INFO - Syncing klippy Objects
2023-12-30 00:10:24,302 mobileraker.sync INFO - Received server info: {'jsonrpc': '2.0', 'result': {'klippy_connected': False, 'klippy_state': 'disconnected', 'components': ['klippy_c>
2023-12-30 00:10:24,304 mobileraker.sync WARNING - Klippy was not ready. Trying resync again in 4 seconds...
2023-12-30 00:10:28,305 mobileraker.sync INFO - Syncing klippy Objects
2023-12-30 00:10:28,314 mobileraker.sync INFO - Received server info: {'jsonrpc': '2.0', 'result': {'klippy_connected': False, 'klippy_state': 'disconnected', 'components': ['klippy_c>
2023-12-30 00:10:28,315 mobileraker.sync WARNING - Klippy was not ready. Trying resync again in 8 seconds...
2023-12-30 00:10:36,316 mobileraker.sync INFO - Syncing klippy Objects
2023-12-30 00:10:36,320 mobileraker.sync INFO - Received server info: {'jsonrpc': '2.0', 'result': {'klippy_connected': False, 'klippy_state': 'disconnected', 'components': ['klippy_c>
2023-12-30 00:10:36,320 mobileraker.sync WARNING - Klippy was not ready. Trying resync again in 16 seconds...
2023-12-30 00:10:52,322 mobileraker.sync INFO - Syncing klippy Objects
2023-12-30 00:10:52,327 mobileraker.sync INFO - Received server info: {'jsonrpc': '2.0', 'result': {'klippy_connected': False, 'klippy_state': 'disconnected', 'components': ['klippy_c>
2023-12-30 00:10:52,328 mobileraker.sync WARNING - Klippy was not ready. Trying resync again in 32 seconds...
2023-12-30 00:11:24,331 mobileraker.sync INFO - Syncing klippy Objects
2023-12-30 00:11:24,334 mobileraker.sync INFO - Received server info: {'jsonrpc': '2.0', 'result': {'klippy_connected': False, 'klippy_state': 'disconnected', 'components': ['klippy_c>
2023-12-30 00:11:24,335 mobileraker.sync WARNING - Klippy was not ready. Trying resync again in 64 seconds...
2023-12-30 00:12:28,338 mobileraker.sync INFO - Syncing klippy Objects
2023-12-30 00:12:28,345 mobileraker.sync INFO - Received server info: {'jsonrpc': '2.0', 'result': {'klippy_connected': False, 'klippy_state': 'disconnected', 'components': ['klippy_c>
2023-12-30 00:12:28,346 mobileraker.sync WARNING - Klippy was not ready. Trying resync again in 128 seconds...
2023-12-30 00:14:36,349 mobileraker.sync INFO - Syncing klippy Objects
2023-12-30 00:14:36,353 mobileraker.sync INFO - Received server info: {'jsonrpc': '2.0', 'result': {'klippy_connected': False, 'klippy_state': 'disconnected', 'components': ['klippy_c>
2023-12-30 00:14:36,361 mobileraker.sync WARNING - Klippy was not ready. Trying resync again in 256 seconds...
2023-12-30 00:17:09,403 mobileraker.sync INFO - Moonraker's connection to Klippy has terminated/disconnected
2023-12-30 00:17:09,405 mobileraker._Default INFO - No last snapshot available. Evaluating!
2023-12-30 00:17:09,405 mobileraker._Default INFO - Snapshot passed threshold. LastSnap: None, NewSnap: PrinterSnapshot(klippy_ready=False, print_state=error, m117=None, m117_hash=, v>
2023-12-30 00:17:09,410 mobileraker.jrpc WARNING - Error message received from WebSocket-Server Key 'fcm' in namespace 'mobileraker' not found
2023-12-30 00:17:09,411 mobileraker._Default WARNING - Could not fetch app cfgs from moonraker, moonraker returned error Key 'fcm' in namespace 'mobileraker' not found
2023-12-30 00:17:09,412 mobileraker._Default INFO - ---- Completed Evaluations Task! ----
2023-12-30 00:17:13,010 mobileraker.sync INFO - Klippy has reported a ready state
2023-12-30 00:17:13,011 mobileraker.sync INFO - Doing a (Re)Sync with moonraker
2023-12-30 00:17:13,012 mobileraker.sync INFO - Syncing klippy Objects
2023-12-30 00:17:13,015 mobileraker.sync INFO - Received server info: {'jsonrpc': '2.0', 'result': {'klippy_connected': False, 'klippy_state': 'disconnected', 'components': ['klippy_c>
2023-12-30 00:17:13,016 mobileraker.sync WARNING - Klippy was not ready. Trying resync again in 2 seconds...
2023-12-30 00:17:15,018 mobileraker.sync INFO - Syncing klippy Objects
2023-12-30 00:17:15,024 mobileraker.sync INFO - Received server info: {'jsonrpc': '2.0', 'result': {'klippy_connected': False, 'klippy_state': 'disconnected', 'components': ['klippy_c>
2023-12-30 00:17:15,025 mobileraker.sync WARNING - Klippy was not ready. Trying resync again in 4 seconds...
2023-12-30 00:17:19,027 mobileraker.sync INFO - Syncing klippy Objects
2023-12-30 00:17:19,034 mobileraker.sync INFO - Received server info: {'jsonrpc': '2.0', 'result': {'klippy_connected': False, 'klippy_state': 'disconnected', 'components': ['klippy_c>
2023-12-30 00:17:19,035 mobileraker.sync WARNING - Klippy was not ready. Trying resync again in 8 seconds...
2023-12-30 00:17:27,037 mobileraker.sync INFO - Syncing klippy Objects
2023-12-30 00:17:27,043 mobileraker.sync INFO - Received server info: {'jsonrpc': '2.0', 'result': {'klippy_connected': False, 'klippy_state': 'disconnected', 'components': ['klippy_c>
2023-12-30 00:17:27,044 mobileraker.sync WARNING - Klippy was not ready. Trying resync again in 16 seconds...
2023-12-30 00:17:43,045 mobileraker.sync INFO - Syncing klippy Objects
2023-12-30 00:17:43,051 mobileraker.sync INFO - Received server info: {'jsonrpc': '2.0', 'result': {'klippy_connected': False, 'klippy_state': 'disconnected', 'components': ['klippy_c>
2023-12-30 00:17:43,052 mobileraker.sync WARNING - Klippy was not ready. Trying resync again in 32 seconds...
2023-12-30 00:18:15,056 mobileraker.sync INFO - Syncing klippy Objects
2023-12-30 00:18:15,062 mobileraker.sync INFO - Received server info: {'jsonrpc': '2.0', 'result': {'klippy_connected': False, 'klippy_state': 'disconnected', 'components': ['klippy_c>
2023-12-30 00:18:15,063 mobileraker.sync WARNING - Klippy was not ready. Trying resync again in 64 seconds...
2023-12-30 00:18:52,363 mobileraker.sync INFO - Syncing klippy Objects
2023-12-30 00:18:52,367 mobileraker.sync INFO - Received server info: {'jsonrpc': '2.0', 'result': {'klippy_connected': False, 'klippy_state': 'disconnected', 'components': ['klippy_c>
2023-12-30 00:18:52,368 mobileraker.sync WARNING - Klippy was not ready. Trying resync again in 300 seconds...
2023-12-30 00:19:19,065 mobileraker.sync INFO - Syncing klippy Objects
2023-12-30 00:19:19,071 mobileraker.sync INFO - Received server info: {'jsonrpc': '2.0', 'result': {'klippy_connected': False, 'klippy_state': 'disconnected', 'components': ['klippy_c>
2023-12-30 00:19:19,071 mobileraker.sync WARNING - Klippy was not ready. Trying resync again in 128 seconds...
2023-12-30 00:21:27,073 mobileraker.sync INFO - Syncing klippy Objects
2023-12-30 00:21:27,078 mobileraker.sync INFO - Received server info: {'jsonrpc': '2.0', 'result': {'klippy_connected': False, 'klippy_state': 'disconnected', 'components': ['klippy_c>
2023-12-30 00:21:27,080 mobileraker.sync WARNING - Klippy was not ready. Trying resync again in 256 seconds...

Restarting moonraker and octoeverywhere services hasn't changed the experience unfortunately.
What else can I try?
Thank you.

Hey,
Sorry to hear about that experience. However, I do not think that this is related to the companion. To me, it seems like Klipper has a problem connecting since it is in the disconnected state and both, octoeverywhere and companion report connection problems to moonraker.
Therefore, could you also upload the klipper.log file?

Oh really, that's interesting. I was successfully printing just before making the installation so that's what made me look into this first. Granted I'm not very technical on this side of printing, I can set things up but my troubleshooting game is weak.
Here's the klippy.log, figured it was easier to just put the whole thing on here.
klippy.log

Thanks. Sadly the logs do not indicate any issue with klipper and therefore my assumption is wrong.
Thus, could you upload all log files (moonraker.log, mobileraker.log, ...). Can you also post the version of Moonraker you are using?
The companion it self is a passive listener except for the startup where it writes some meta data into the printer's database like its version. Besides that, all it does is listening to the same source as the GUIs do.

Sure thing - it's the latest version I believe, v0.8.0-239-gc4d15e4.
I have also just noticed that I still have Obico running on this thing too, I don't use it anymore - haven't for several months - but I think it seems to still be reaching out. Added the most recent log for that here too, though I don't believe it's conflicting with the operation.
mobileraker.log
moonraker.log
[octoevery
moonraker-obico.log
where.log](https://github.com/Clon1998/mobileraker_companion/files/13798043/octoeverywhere.log)

Thanks,
it seems like octoeverywhere is sending a database request close to every second which fails. Therefore, Id suggest to stop octoeverywhere and the mobileraker_companion services and restart moonraker just to see if we get the UIs back working.
Ensure you are connected via SSH to the printer and run:
sudo systemctl stop mobileraker.service
sudo systemctl stop octoeverywhere.service
sudo systemctl restart moonraker.service

Fantastic, that has restored the function again.
OK so I clearly need to focus more on general troubleshooting, thank you so much for your assistance, especially at this time of the year.

Fantastic, that has restored the function again.
OK so I clearly need to focus more on general troubleshooting, thank you so much for your assistance, especially at this time of the year.

Fantastic, that has restored the function again.
OK so I clearly need to focus more on general troubleshooting, thank you so much for your assistance, especially at this time of the year.

Awesome, can you do me a favor and start the companion once again (Either via mainsail/Fluidd or by replacing the stop with start in the bash command above)

Yes, I have restarted the companion again, and testing with a small print job I am now receiving updates on my phone.


Add links and beautify relation in bread views

This simple PR just improve the visibility of the relations. I used "badge" system for the browse view, and I add links to the related entity for each relation.

The link part is very similar to https://github.com/the-control-group/voyager/pull/3695 where I already commented that I'm not a friend of.

Yes I just saw, I will change the way to get the slug and update it. My PR is more complete by the way on the view part.

If you find a way to do this in the controller, then please do it.
Also your PR reverts the changes from https://github.com/the-control-group/voyager/pull/3444. Please don't do that.

I purpose to store the slug in the details of the relation. It's possible to update the slug when you update the bread. I think the simplest solution.

I second this feature.
It is mind blowing that it currently is not possible.
@emptynick any update on the status of this?


add: Tanbir Ali

Description
I have added my profile in the profile.json
Checklist

[x] I have read and followed the Contribution Guidelines.

@tanbirali Thank you so much for your contribution! Your efforts are greatly appreciated and will go a long way in improving our project. Please feel free to share it with others and Star the repo to help us grow even further!


Adding support for Ubuntu dev environment

Coverage remained the same at 62.706% when pulling 7416d648fed38696efd02fef4970800126e2bfce on vinodtalati:develop into f283f2ee1c0fa797fb752d8712854aebf35cc298 on intuit:develop.

Coverage remained the same at 62.706% when pulling bc48c79831a310c7d8665a9d720146329f6cac97 on vinodtalati:develop into f283f2ee1c0fa797fb752d8712854aebf35cc298 on intuit:develop.

Coverage remained the same at 62.706% when pulling fb8bff1e18044de8d1ee82da7fbf1e523248987a on vinodtalati:develop into f283f2ee1c0fa797fb752d8712854aebf35cc298 on intuit:develop.

Coverage remained the same at 62.706% when pulling fb8bff1e18044de8d1ee82da7fbf1e523248987a on vinodtalati:develop into f283f2ee1c0fa797fb752d8712854aebf35cc298 on intuit:develop.

Coverage remained the same at 62.706% when pulling fb8bff1e18044de8d1ee82da7fbf1e523248987a on vinodtalati:develop into f283f2ee1c0fa797fb752d8712854aebf35cc298 on intuit:develop.

@vinodtalati the included git-diff is a delta between your PR, wasabi:develop and some subtle changes of my own. it looks to work for me, namely:
% ./bin/wasabi.sh package # gens 2-rpms and 2-debs in the [ws]/target dir
i should've done the changes as a PR against your PR ... but i opted to cut to the chase, see how much work is required and just ran with it creating a diff i hope you and i can work against to get your PR mergable.
let's do this :)

james

jwtodd-diff.txt

We should reflect in the README.md that Ubuntu is supported - currently it states only MacOSX is supported. For my windows branch I basically changed it like this (WIP): https://github.com/intuit/wasabi/blob/feature/windows/README.md#bootstrap-your-environment

+1 @shoeffner ... good call

Coverage remained the same at 63.687% when pulling bdf507cdce5b51b6ffdfdfbf82af3f0c01af48f6 on vinodtalati:develop into 9b907f00594a52ace34c68615e326ac4f4b55b1e on intuit:develop.

@jwtodd Changes merged based on your diff. Can you review?

Coverage remained the same at 63.687% when pulling bdfde2845bf46134f2e475da4ef899f92ee8ac23 on vinodtalati:develop into 9b907f00594a52ace34c68615e326ac4f4b55b1e on intuit:develop.

Coverage remained the same at 63.687% when pulling bdfde2845bf46134f2e475da4ef899f92ee8ac23 on vinodtalati:develop into 9b907f00594a52ace34c68615e326ac4f4b55b1e on intuit:develop.

Coverage remained the same at 63.687% when pulling 7aba14295997fce063daade62620cc5bf956fc49 on vinodtalati:develop into 9b907f00594a52ace34c68615e326ac4f4b55b1e on intuit:develop.

Tested on Ubuntu and MAC. Everything works now. Here's the output on Ubuntu (last few lines only):
Package version '1.0.20160902232623-SNAPSHOT' includes dashes, converting to underscores {:level=>:warn, :file=>"fpm/package/rpm.rb", :line=>"548", :method=>"version"}
Reading template {:path=>"/usr/local/share/gems/gems/fpm-1.6.2/templates/rpm.erb", :level=>:info, :file=>"fpm/package.rb", :line=>"328", :method=>"template"}
Running rpmbuild {:args=>["rpmbuild", "-bb", "--target", "noarch-unknown-linux", "--define", "buildroot /tmp/package-rpm-build20160906-1-kxdxpz/BUILD", "--define", "_topdir /tmp/package-rpm-build20160906-1-kxdxpz", "--define", "_sourcedir /tmp/package-rpm-build20160906-1-kxdxpz", "--define", "_rpmdir /tmp/package-rpm-build20160906-1-kxdxpz/RPMS", "--define", "_tmppath /tmp", "/tmp/package-rpm-build20160906-1-kxdxpz/SPECS/wasabi-ui-build.spec"], :level=>:info, :file=>"fpm/package/rpm.rb", :line=>"516", :method=>"output"}
Running command {:args=>["rpmbuild", "-bb", "--target", "noarch-unknown-linux", "--define", "buildroot /tmp/package-rpm-build20160906-1-kxdxpz/BUILD", "--define", "_topdir /tmp/package-rpm-build20160906-1-kxdxpz", "--define", "_sourcedir /tmp/package-rpm-build20160906-1-kxdxpz", "--define", "_rpmdir /tmp/package-rpm-build20160906-1-kxdxpz/RPMS", "--define", "_tmppath /tmp", "/tmp/package-rpm-build20160906-1-kxdxpz/SPECS/wasabi-ui-build.spec"], :level=>:debug, :file=>"fpm/util.rb", :line=>"144", :method=>"execmd"}
Process is running {:pid=>11, :level=>:debug, :file=>"fpm/util.rb", :line=>"162", :method=>"execmd"}
Building target platforms: noarch-unknown-linux {:level=>:info, :file=>"cabin/mixins/pipe.rb", :line=>"47", :method=>"block in pipe"}
Building for target noarch-unknown-linux {:level=>:info, :file=>"cabin/mixins/pipe.rb", :line=>"47", :method=>"block in pipe"}
Executing(%prep): /bin/sh -e /tmp/rpm-tmp.HlcbT2 {:level=>:info, :file=>"cabin/mixins/pipe.rb", :line=>"47", :method=>"block in pipe"}
Executing(%build): /bin/sh -e /tmp/rpm-tmp.P08kbO {:level=>:info, :file=>"cabin/mixins/pipe.rb", :line=>"47", :method=>"block in pipe"}
Executing(%install): /bin/sh -e /tmp/rpm-tmp.pW9Jtz {:level=>:info, :file=>"cabin/mixins/pipe.rb", :line=>"47", :method=>"block in pipe"}
Processing files: wasabi-ui-build-1.0.20160902232623_SNAPSHOT-20160906222030.noarch {:level=>:info, :file=>"cabin/mixins/pipe.rb", :line=>"47", :method=>"block in pipe"}
Provides: wasabi-ui-build wasabi-ui-build = 1.0.20160902232623_SNAPSHOT-20160906222030 {:level=>:info, :file=>"cabin/mixins/pipe.rb", :line=>"47", :method=>"block in pipe"}
Requires(interp): /bin/sh /bin/sh /bin/sh /bin/sh {:level=>:info, :file=>"cabin/mixins/pipe.rb", :line=>"47", :method=>"block in pipe"}
Requires(rpmlib): rpmlib(PayloadFilesHavePrefix) <= 4.0-1 rpmlib(CompressedFileNames) <= 3.0.4-1 {:level=>:info, :file=>"cabin/mixins/pipe.rb", :line=>"47", :method=>"block in pipe"}
Requires(pre): /bin/sh {:level=>:info, :file=>"cabin/mixins/pipe.rb", :line=>"47", :method=>"block in pipe"}
Requires(post): /bin/sh {:level=>:info, :file=>"cabin/mixins/pipe.rb", :line=>"47", :method=>"block in pipe"}
Requires(preun): /bin/sh {:level=>:info, :file=>"cabin/mixins/pipe.rb", :line=>"47", :method=>"block in pipe"}
Requires(postun): /bin/sh {:level=>:info, :file=>"cabin/mixins/pipe.rb", :line=>"47", :method=>"block in pipe"}
Wrote: /tmp/package-rpm-build20160906-1-kxdxpz/RPMS/noarch/wasabi-ui-build-1.0.20160902232623_SNAPSHOT-20160906222030.noarch.rpm {:level=>:info, :file=>"cabin/mixins/pipe.rb", :line=>"47", :method=>"block in pipe"}
Executing(%clean): /bin/sh -e /tmp/rpm-tmp.H5xvo6 {:level=>:info, :file=>"cabin/mixins/pipe.rb", :line=>"47", :method=>"block in pipe"}
Created package {:path=>"wasabi-ui-build-1.0.20160902232623_SNAPSHOT-20160906222030.noarch.rpm", :file=>"clamp/command.rb", :line=>"68", :method=>"run"}
Cleaning up staging path {:path=>"/tmp/package-dir-staging20160906-1-1ryut8y", :level=>:debug, :file=>"fpm/package.rb", :line=>"279", :method=>"cleanup_staging"}
Cleaning up build path {:path=>"/tmp/package-dir-build20160906-1-1il20i7", :level=>:debug, :file=>"fpm/package.rb", :line=>"286", :method=>"cleanup_build"}
Cleaning up build path {:path=>"/tmp/package-rpm-build20160906-1-kxdxpz", :level=>:debug, :file=>"fpm/package.rb", :line=>"286", :method=>"cleanup_build"}
deployable build packages:
./target/wasabi-main-build-1.0.20160902232623_SNAPSHOT-20160906221713.noarch.rpm
./target/wasabi-main-build_1.0.20160902232623-SNAPSHOT-20160906221713_all.deb
./target/wasabi-ui-build-1.0.20160902232623_SNAPSHOT-20160906222030.noarch.rpm
./target/wasabi-ui-build_1.0.20160902232623-SNAPSHOT-20160906222030_all.deb

Coverage remained the same at 63.687% when pulling 68f49f3f0909cd5ac051928ff9ee0a651197fff7 on vinodtalati:develop into 9b907f00594a52ace34c68615e326ac4f4b55b1e on intuit:develop.

:ship it:

Merged :) ... Let's take on the next issue :)


fix: image digest supports several algos, not just sha256

Signed-off-by: Wei Fu<EMAIL_ADDRESS>1.Describe what this PR did
Fix bug: The image should support sha256, sha512 or other no-registered algorithms.
https://github.com/opencontainers/image-spec/blob/master/descriptor.md#digests
2.Does this pull request fix one issue?
None
3.Describe how you did it
Follow the image-spec.
4.Describe how to verify it
None
5.Special notes for reviews
None

The change is ready. PTAL @skoo87  thanks

LGTM


Switch to gulp-typescript instead of gulp-tsc

Any particular reason to make the switch?

Incremental compilation (so faster builds)
Error reporting
Different output streams for .js, .d.ts files.
Support for sourcemaps using gulp-sourcemaps
Compile once, and filter different targets
gulp-typescript supports the tsconfig.json file. gulp-tsc doesn't.


how to share default service

Which Component
Nacos Discovery
I want to register a default group service  and share with other group service.  If no this feature, each of us need to start up too many IDE and service, but I just need to care for my code.

Can you give me an example? I'm not quite sure what your question is.

 As you know, every programmer just develop one or some of micro service. when I try to debug my code,
 I use myself discovery group. how can i just start up my service and use others public service?
 we are not in one discovery group and I don't want start up all of the micro service.
------------------ 原始邮件 ------------------
发件人:                                                                                "alibaba/spring-cloud-alibaba"                                                                                @.>;
发送时间: 2021年4月1日(星期四) 下午3:53
@.>;
<EMAIL_ADDRESS>主题: Re: [alibaba/spring-cloud-alibaba] how to share default service (#2018)
Can you give me an example? I'm not quite sure what your question is.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Different grouping calls are not supported. If you want local debugging, you can try registering the local service with the same namespace and dropping the old service offline.


More user-friendly editor manipulation of export var such as Vector2 and Path2D

It would be helpful if the editor supported more export var types.
What I mean by this is, when exporting a Vector2, it would be better to have a draggable point visible in the scene, instead of changing the values manually.
Even more importantly, some data types such as Path2D aren't usable as export vars, as the buttons in the editor to modify points do not appear. (This could possibly be two seperate issues.)
My use-case: 'dynamic' nodes which draw themselves

What I mean by this is, when exporting a Vector2, it would be better to have a draggable point visible in the scene, instead of changing the values manually.

That would make sense if your Vector2 is a position, but there are many uses of Vector2 which are not positions and for which a placement on the scene like Postion2D does not make sense (e.g. a direction vector, or an initial velocity vector). It does sound like the Position2D node would be a good match for your use case instead of export vars.

Another thought: if user-configurable annotations are added to GDScript via plugins, then one could conceivably write a plugin that recognizes a certain type of export flag on Vector2 values and then implements its own EditorInspectorPlugin and canvas viewport override to render a Position2D node exclusively within the editor, with the sole purpose of assigning a value to the Vector2.
That would be a solution that doesn't involve modifying PropertyInfo hint values, adding a new EditorProperty type, or changing anything else about the engine/editor tailored for this solution. It would just be low-level stuff (add annotations, allow plugins to create new annotations) that empowers plugin developers to create better editor tools to match their needs.
This would be another use-case for having annotations that impart some level of functionality rather than just metadata (#20318).

I have a proposal about Vector2/3 visual editing at godotengine/godot-proposals#397, and in fact created an editor plugin for it. It does not allow to export Vector2/3 with a hint, but instead allows to edit a resource containing the vectors themselves, which is supported currently. This certainly has limitations but as an initial implementation this may be a good enough workaround.
Of course it would be much more convenient to embed an EditorInspectorPlugin without resorting to implementing an entire Resource just to be able to edit these core data types, via annotations or similar. But well, if you also want to take advantage of the Resource being shared between instances, that's also a good reason to use the plugin I've linked.

Superseded by https://github.com/godotengine/godot-proposals/issues/397.


Prevent the compiler finding the wrong CGFloat

As open on this Issue, this small PR allows to use the library on iOS, WatchOS and tvOS.
Checklist

[x] The provided tests still run.
[ ] I've created new tests where needed.
[ ] I've updated the documentation if necessary.

Motivation and Context
Then we can use the library on iOS, the issue is this one
Description
Adding a compiler flag, to import CoreGraphics will fix the compiler getting crazy of which CGFloat to use.

Now available in https://github.com/MrLotU/SwiftPrometheus/releases/tag/1.0.0-alpha.7


Discord intergration with screenshot plugin

I have been using runelite for a couple of years already, but found out it could also make screenshots. these are getting placed in the user file of the machine. i found out these where not saved in the cloud or any thing like that, so i am missing some cool screenshots :(
that made me look into the plugin hub of runelite, here i found some plugins that use webhooks to upload the screenshots to any discord channel. these plugins work fine, but as far as i can see the standard screenshot plugin has more options. it would be realy nice if this plugin could use webhooks aswell.
i see an option to upload the screenshots to imgur, but i cant find any documentation to link it with my imgur account.

You can always set up the folder to be synchronised with your preferred web service. Duplicate of #3147
Integrating into Discord webhooks is, like you said, already something that hub plugins do. It would likely be better if those plugins expanded their options.
The screenshot plugin uploads to Imgur through the RL API, and is heavily rate limited. Duplicate of #2120

ahh okay thanks for the clear awnser :)

As these requests are covered by other issues, I am closing this issue.


Remove version: Sogou.SogouInput version <IP_ADDRESS>60

[Automated] It returns code over 400 in all urls
Microsoft Reviewers: Open in CodeFlow

URL: https://ime.sogoucdn.com/sogou_pinyin_<IP_ADDRESS>60.exe
Response Error: The remote server returned an error: (403) Forbidden.
(Automated message - build 910)


The requested certificate template is not supported by this CA.

This error occurred when I was using ESC4，By using the command - vulnerable, it can be determined that the template has an ESC4 vulnerability
certipy req -username<EMAIL_ADDRESS>-password "password@123" -ca DC1-CA -target DC1.test.com -template HorizonV8 -upn<EMAIL_ADDRESS>-dns-tcp -ns <IP_ADDRESS>
Certipy v4.8.2 - by Oliver Lyak (ly4k)

[] Requesting certificate via RPC
[-] Got error while trying to request certificate: code: 0x80094800 - CERTSRV_E_UNSUPPORTED_CERT_TYPE - The requested certificate template is not supported by this CA.
[] Request ID is 129
Would you like to save the private key? (y/N) n
[-] Failed to request certificate

This is mostly because the certificate template you are using is not enabled. Run the certipy find command again with -enabled to get only enabled certificates.

This is mostly because the certificate template you are using is not enabled. Run the certipy find command again with -enabled to get only enabled certificates.

But I used this template to successfully apply for the pfx certificate using esc1.

Looking forward to your reply again

See https://github.com/ly4k/Certipy/issues/189#issuecomment-1908163430

参见#189（评论）

Do you have detailed usage instructions for reference? Thank you for your reply


Error When Inferencing

When I run !python chat.py, I got AssertionError: libcuda.so cannot found!.
I'm using colab.

Solution could be found on provided colab notebook -> (https://colab.research.google.com/drive/1dUlEYnRbgJYg4_kofNpsCddLCh6vltNK?usp=sharing#scrollTo=ml0er1AqpO0t)


replaceState per parameter?

Thank you so much for this library. It's saved me so much hassle, and I think it's the correct way to think about state and url as a window into the state.
Currently we have a global option replaceState to say whether each change to the state should update the url by replacing it (not creating a back/forward history entry) or pushing to it (creating a back/forward history entry). Is it possible to have this option on a per-parameter basis, such that changes to some parameters don't create the entry, but some do?
Going even further, could we get the new and previous values and decide based on that whether to push or replace? So it would look something like this:
ReduxQuerySync({
    store,
    params: {
        p: {
            selector: state => state.pageNumber,
            action: value => ({type: 'setPageNumber', payload: value}),
            replaceState: (value, prevValue) => someLogic ? true : false
        },
    }
})

This might be overcomplicating things though. I can explain my use case if need be.
If this is not on the roadmap, do you have a recommended way to implement this ourselves?

Glad you like it!
This module could be modified to allow passing a replaceState option for
individual params that would override the ‘global’ replaceState option.
It would need some changes in the handleStateUpdate function; I suppose
it could check which parameters are being modified, and use history.push
if the replaceState option is false for at least one of the modified
parameters. Alternatively, it could run two rounds: change all ‘replace’
parameters, then change all ‘push’ parameters (or vice versa; not sure
what behaviour we’d want the browser’s back button to have).
Actually, the latter two-step behaviour may be possible without any
change to this module. If you’d simply use this model twice, once with
the ‘push’ params, once with the ‘replace’ params, then if all is
correct these will independently modify the window location. Or if not
all is correct, it might mess up or end up in an infinite loop where the
two invocations undoing each other’s modifications (which I’d consider a
bug; the module should not interfere with location changes unrelated to
the parameter names passed to it). I am curious which of these two
possible worlds we live in.

Actually I should mention that I was able to solve my issue simply with the use of the defaultValue option which I didn't see before. Though I could still see myself using this in the future. The case would be: where I want a parameter to be in the url (say for sharing purposes), but that parameter isn't important enough to warrant its own undo/redo entry (via the back/forward history).
So perhaps since it would be some work, and possibly some breaking changes, you should just wait to see if there's anyone else interested in this feature. See if this issue gets some 👍 's.

I need this above behaviour for my project. I need to replace in some cases and push in other case. It would be great if we implement this behaviour.
@vincerubinetti can you guide me how exactly did you solve this issue with defaultValue

@deepakgupta25 I don't think I can help you there. What I think I meant by using defaultValue is that I needed, well, a default value... and I was looking to use replace the first time, then push all subsequent times. At least I think that's what I was looking for; this was 2 years ago and I don't even remember the project this was for.
If you really need arbitrary switching back and forth between replace and push, you'd need the library to implement it as requested.


Tolerate peer disconnects

Context
Messenger manages a per-peer queue of messages. Furthermore, those queues are not cleaned up right now, which is technically a bug, but the plan to fix this is directly related to peer disconnects handling. The goal is to allow queues to leave for some configurable time if related peer disconnects, so once it reconnects the same queue comes into place with sending msgs. This is possible due to the non-blocking API.
This also requires migration to a non-native channel implementation that would support pushing items in front to remain to keep msg ordering. More explanations here.

The PR was merged but it doesnt really fix the issue, as it much harder and requires deeper changes in the way muxer works. Basically, stream should buffer Writes, so that user know was the message actual send or not.

The solution to the problem, could be an additional error channel attached to each stream in the implementation of most of libp2p muxers. The Write implementation, after writing to a buffer or a write channel, would block on the new error channel and wait until the session/connection of the muxer notified the stream back with nil or error. This way libp2p streams would be able to provide a guarantee that any msg written with success was also written to the underlying session/connection. However, this makes all the writes to the libp2p streams blocking, and there might be reasons to keep them so.


Skipping vcf2db.py?

I'm running the pipeline on a practice AshkenaziFam dataset before trying it on my own data.  I'm having trouble getting the AshkenaziFam-ensemble.db file at the end of my run.  Previously, I was running an older version of the pipeline where VEP wasn't working properly (version 1.1.6).  When I was running this version the AshkenaziFam-ensemble.db file was being created no problem.  When I updated to the latest version so that VEP would run, the AshkenaziFam-ensemble.db file was suddenly no longer being created.  The other files are still there (AshkenaziFam-freebayes.db, etc), just the ensemble one is missing.  Everytime I run the pipeline I run it as a brand new instance in a new folder, so it doesn't have anything to do with the program thinking files are already there.
I tried to go through my new log and command files to compare it to the old ones to see what the differences were.  My old command file seems to run the following lines:
[2019-11-14T04:49Z]<EMAIL_ADDRESS>-p 7 -lua<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>| sed -e 's/Number=A/Number=1/g' | bgzip -c ><EMAIL_ADDRESS>[2019-11-14T05:17Z]<EMAIL_ADDRESS>-f -p vcf<EMAIL_ADDRESS>[2019-11-14T05:17Z]<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>While the newer log file only has:
[2020-02-01T07:08Z]<EMAIL_ADDRESS>-p 7 -lua<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>|  bgzip -c ><EMAIL_ADDRESS>[2020-02-01T07:37Z]<EMAIL_ADDRESS>-f -p vcf<EMAIL_ADDRESS>It seems to skip over the vcf2db.py command.  (I'm assuming the slight difference in the naming is because VEP is now running.)
I tried to run vcf2db on my own, but the .ped file is missing and I can't figure out where this file is created.
My log file seems to indicate that gemini isn't running, but I also can't find the command where gemini is supposed to run and I have no idea why it's only skipping over gemini the once and not for the other instances as well.
[2020-02-01T07:08Z] multiprocessing: prep_gemini_db
[2020-02-01T07:08Z] Not running gemini, no samples with variants found: 151002_7001448_0359_AC7F6GANXX_Sample_HG002-EEogPU_v02-KIT-Av5_AGATGTAC_L008_posiSrt_markDup, 151002_7001448_0359_AC7F6GANXX_Sample_HG003-EEogPU_v02-KIT-Av5_TCTTCACA_L008_posiSrt_markDup, 151002_7001448_0359_AC7F6GANXX_Sample_HG004-EEogPU_v02-KIT-Av5_CCGAAGTA_L008_posiSrt_markDup
I also tried updating Gemini using "bcbio_nextgen.py upgrade --datatarget gemini" but that didn't seem to help.
Any ideas?  I've uploaded the latest log and command files.
bcbio-nextgen.log
bcbio-nextgen-commands.log

Thank you, could you upload the ../config/whateveryouarerunningwith.yaml configuration file as well, so we can take a look? Sorry for the slow response, we've been busy/away the past few days.

Hi @tathey !
Thanks for reporting!
So the issue is gemini.db is not created for ensemble calling.
There is no errors in the bcbio log itself.
Could you add your yaml and confirm you bcbio and vep versions, and also your annotation configuration (you are using vcfanno conf from cre, that could have been updated since 1.1.6)?
bcbio_nextgen.py --version
vep

Sergey

Sorry for taking so long to respond.  I was away last week.
Here is the yaml file AshkenaziFam.yaml.txt (I had to add .txt to the name in order for github to let me upload it).  I'm running bcbio version 1.2.0a, ensembl 99.d3e7d31, ensembl-funcgen 99.0832337, ensembl-io 99.441b05b, ensembl-variation 99.642e1cd, ensembl-vep 99.0.
There don't seem to be any errors in the bcbio log.  It just says that it's skipping the gemini calling step because it couldn't find samples with variants.  The Ashkenazi-ensemble.vcf.gz does have variants in it though.
I was talking to Dennis at CCM.  He said that they use bcbio version 1.1.2 for their scripts, so I'm thinking the vcfanno conf would be for that.  He also pointed out that my file names are 151002_7001448_0359_AC7F6GANXX_Sample_HG002-EEogPU_v02-KIT-Av5_AGATGTAC_L008.posiSrt.markDup but the error is calling them 151002_7001448_0359_AC7F6GANXX_Sample_HG002-EEogPU_v02-KIT-Av5_AGATGTAC_L008_posiSrt_markDup.  He was wondering if the change from "." to "_" is what's causing the error.  I had also run a different file through the pipeline that had a simple name that ended in .fastq.gz.  I got the same error and it also changed the name to _fastq_gz
Thanks again for looking at this.  I should be quicker at responding now that I'm back from vacation.
Taryn

Hi Taryn @tathey !
Thanks for the follow up!
I'd also suggest not to use complex file names and sample names, as these strings go into bam files (readgroup) and into the gemini database. Avoiding dots in the sample name and @ (and other special characters) in the filenames might help as well.
currently I'm running it with:
  analysis: variant2
  description: HG004_NA24143_mother
  files:
  - /path/to/ashkenazim_trio/input/HG004_NA24143_mother.bam.chr22.bam

Hopefully, it would help to resolve, let us know otherwise.  I am also running your config with chr22 - will see soon what I get.
Some more thoughts:

Ashkehazim trio is a very important use case for bcbio, since it is a user entry point for Clinical and Clinical Research variant calling users. If you'd like to contribute the user story, that would be fantastic. We would include it into the docs (just write up what you are doing as an end user: NIST article, where did you get input files, bcbio config, runnig bcbio, results, validation results).
Would you consider running analyses with hg38? Currently we support gnomad v3 which are native calls of gnomad genomes in hg38. Exomes are still lifted over from grch37 to hg38, but having native hg38 genome calls is a game changer.
What is your opinion on using 4 variant callers and ensembl calling?  Several years ago when calling algorithms were more or less on par it made sense, but now you may just use gatk4, and consider all variants, not only PASS filters if you would like to pull 'junk' variants.
I'd like to review https://github.com/naumenko-sa/cre and move vcfanno.conf and some scripts to bcbio to have an excel of prioritized variants among bcbio output files. Any suggestions how it should be done for the benefit of a wider user base is appreciated.

Sergey

Hi @tathey !
Finally, I was able to reproduce the issue, and found that since Oct 3, 2019 gemini.db is not created for ensemble:
https://github.com/bcbio/bcbio-nextgen/commit/8d60e8243ed112f0ac1f558a446cb08c786c0c1c#diff-9d644e8df7caebccd9435edfa4ad2484
@chapmanb could you please clarify this change?
Was it because of issues with ensemble.db in somatic calling (vardict+mutect2+strelka2)?
In germline ensemble gemini db worked well with ensemble: gatk/samtools/freebayes/platypus.
Isn't it a redundant switch?

default is no gemini db;
user switches tools_on: gemini
we are switching gemini off in the code for ensemble or mutect;

Would it be an option for the original issue not to switch on gemini, or not to use ensemble?
@tathey: on the practical side, if you are establishing a pipeline, would you consider for germline calling  just to use gatk and have gemini.db for it? Include all calls, if you want to investigate 'trash' ones. Combining calls from multiple callers is tricky, and the gain is questionable, so we won't be able to prioritize patching ensemble/ensemble+gemini calling if you hit some issues in the future.
Sergey

Sergey;
Sorry about the confusion here. The issue with ensemble in general is that it pulls together calls from multiple callers. This was creating issues where GEMINI was unhappy due to inconsistency of tags between different caller INFO fields and the headers.
I don't remember the exact motivator and callers that triggered it, but it's a general issue since we don't try to harmonize ensemble calls. So what we tried to do is allow users to turn on gemini so they could get it for individual callers, but avoid the intermittent errors from vcf2db that were due to ensemblization.
I agree with your advice that sticking to a single caller for looking into gemini databases is the best approach. Hope this helps.

Thank you both for all of your help.  This explains a lot, since I couldn't figure out why my previous download was creating the ensemble.db file and next download was not.
I was able to run vcf2db.py by creating my own ensemble.ped file (with the help of CCM), but the cre step failed to run because it could not find CADD scores.  Previously, I was told to stop running the dbNSFP part of the pipeline because the pipeline kept stalling at this step.  I'm pretty sure this is where CADD scores get added, so I'm going to work towards getting this part working.
I've never looked into the stats of what variants get called by each caller, but I'm hesitant to hold up one single variant caller as a golden standard to use above all others.  Sergey, you mentioned that these callers are no longer on par, so it sounds like your opinion is that GATK is giving us the most accurate calls out of all of the callers.  I know that in the past (when I used to work on bacteria) I was having trouble automatically combining calls between Freebayes and GATK, so I can see how only using one caller would eliminate a huge complication.

Thanks, Brad!
Once @hackdna unfreezes documentation, we will clarify there how ensemble&gemini interact.
@tathey:

dbNFSP should work. If not, please raise an issue. I have recently installed it for hg38 - it was fine;
we do have some recent and not so recent validations:
https://github.com/bcbio/bcbio-nextgen/issues/2623
https://github.com/bcbio/bcbio-nextgen/issues/2489

FreeBayes had inferior results in indels in WES (high FDR, FNR) and running FreeBayes for WGS was problematic (gatk is much faster).
~2 years ago I validated gatk4 vs ensemble2of4 and other callers and the conclusion was that ensemble did gave some +39 variants of 38K target, but also was much worse in indels (that is the difference of gatk4 vs other callers - they are worse in indel calling and that makes ensemble worse in indel calling).
If you do clinical or clinical research variant calling, you should validate NA12878 and/or Askenazi trio yourself. If you do so with bcbio, we'd be happy to publish validation results in the docs: https://github.com/bcbio/bcbio-nextgen/issues/3038
Sergey

Thanks for your in-depth responses.  I really appreciate them.
I tried adding dbNSFP back into the configuration and sure enough it failed again.
I think that this current Gemini issue is resolved, so I will close it and open a new issue about dbNSFP.
Taryn


adding new section for sf replicator system health reports with new health reports added

this PR is to add specific replicator system health reports in SF system health report documentation. It also adds description for the newly added health codes too.

@imangoodarzi : Thanks for your contribution to the Azure documentation! The author, @oanapl, has been notified to review your proposed change.

@sumukhs to review

@imangoodarzi : Thanks for your contribution to the Azure documentation! The author, @oanapl, has been notified to review your proposed change.

@imangoodarzi : Thanks for your contribution to the Azure documentation! The author, @oanapl, has been notified to review your proposed change.


Add missing translation strings

WIP #20

@tlahmann is this ready?

@tlahmann is this ready?

This is ready as far as 'translation files are synchronized'


Question: No external access / local VPN setup

Hi,
I've setup pivpn and am trying to connect my machine via tunnelblink but
My client config shows the following but I haven't setup any port forwarding from <IP_ADDRESS> so I can't get tunnelblink to connect.
remote <IP_ADDRESS> 1194

I understand the above IP to be my externally facing IP address. However, I'm not really interested in opening a port to the outside world, I'm mostly interested in routing traffic from machines on my local network through openVPN to get encrypted/anonymous traffic once out on the web.
Is this a common use case or am I swimming against the tide here?
In summary, should I be using pivpn/openVPN if I don't need to connect to my home network from the internet but instead just want anonymous surfing from within? Can I set things up without port forwarding and just route locally?
Thanks in advance

hello, @tobyweston there is no point in using a VPN withing your home network.
The purpose of a VPN is to safely connect to any given network from an unsafe one.
For example, you are at a hotel, and you want to connect to your home, that way, you hide your traffic from prying eyes, you can access to devices on your home network, and your externally facing IP address changes, instead of having the hotel IP address you start having your Home IP address, which allows you to bypass country restrictions if, for example, you are in a country with heavy internet restrictions.
it is pointless to route all your home network through a home VPN.
if you want some privacy from your home through the internet, you should have a VPN somewhere outside your home, there are some free VPNs (most of them are not reliable), some that are paid and have a great reputation, or you can just buy a VPS and setup OpenVPN on that VPS ( which require a bit more of knowledge but is not that hard).
Hope I've been clear to you, however, and since this is just a superficial explanation, I still recommend you to research a little more about VPN and HTTPS, so you can have a better idea of how they work.

Thanks for the reply. That's cleared it up for me a little. I was thinking I could use the VPN as a gateway to the internet like an anonymizing proxy I guess.


Bugfix on Conman

@yogthos
A few months ago you kindly accepted my PR to allow multiple hugsql scripts in system.edn.  The previous code inspects file edit time and if it has updated reloads conman.  Now that we have multiple files we have to map over all the file edit times but of course map is lazy.  As a result the previous edit times would get evaluated at the same time as the current one and never show any difference (!).  Very easy fix!
Matt

thanks, by the way you can also use mapv which is eager. :)

This was probably my bad. I do remember testing it but looking at the code I cannot understand how... Anyways, thanks for fixing it!

teamwork :)

and added fix for last-modified to handle jar gracefully https://github.com/kit-clj/kit/blob/2cc5d77adeb601d268ab9cf7f67a6cae565d878c/libs/kit-core/src/kit/ig_utils.clj

Great work @yogthos

Thanks @yogthos!


Updated game text style

Description:

Updated game text color to green

Accepted: Updated game text color to green


Consider renaming this repo

Having both tccm and tccm-model is confusing
maybe call the latter tccm-lite?
or maybe the two have diverged enough that they should no longer have the same base name

I agree it is confusing.
I think that if we keep tccm named as tccm, tccm-model --> tccm-lite is not a bad suggestion.
I'm often not a fan of <org>/<repo> where <org> and <repo> have the same name. I'm generally fine with this if the repo of the same name has no content in it, and is just used for organizing issues, etc. Or, if it simply has documentation (although in that case adding -documentation to the name might be better than leaving the repo name the same as the org.
@cmungall This is actually an issue for me with LinkML too. If you go to linkml/linkml, there is actually a problem to tccm/tccm. Actually, I'm still not sure what this repository is for. I understand that the issues there are broadly pertaining to LinkML, but I don't understand what the contents of the repo are for; what purpose the repository serves, and the README doesn't help there either (although it does link to docs, which is good).

Hmm, it’s not good if you don’t know what the linkml repo is for, can you
make an issue (in linkml!) for that?
Should we flesh this out
https://linkml.io/linkml/developers/organization.html
On Fri, May 6, 2022 at 11:40 AM Joe Flack @.***> wrote:

I agree it is confusing.
I think that if we keep tccm named as tccm, tccm-model --> tccm-lite is
not a bad suggestion.
I'm often not a fan of / where  and  have the same
name. I'm generally fine with this if the repo of the same name has no
content in it, and is just used for organizing issues, etc. Or, if it
simply has documentation (although in that case adding -documentation to
the name might be better than leaving the repo name the same as the org.
@cmungall https://github.com/cmungall This is actually an issue for me
with LinkML too. If you go to linkml/linkml
https://github.com/linkml/linkml, there is actually a problem to
tccm/tccm. Actually, I'm still not sure what this repository is for. I
understand that the issues there are broadly pertaining to LinkML, but I
don't understand what the contents of the repo are for; what purpose the
repository serves, and the README doesn't help there either (although it
does link to docs, which is good).
—
Reply to this email directly, view it on GitHub
https://github.com/HOT-Ecosystem/tccm/issues/7#issuecomment-1119912227,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAMMOKGWARF7OV6DNWY5GTVIVRRZANCNFSM5UTZGBCQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

@cmungall Yes, my apologies for my ignorance, but maybe it's a good thing in this case. I'll make an issue.
I'm not sure fleshing out that page is necessary. I think it's better to consider what LinkML looks like outside of our small developers sphere. I'm thinking in the context of promoting adoption. I'm not sure what kinds of search queries might lead people to find LinkML, but I think that since we do want to encourage its eventual more widespread use, it would be good for the purpose of each repo to be clear, even in the case where someone first arrives at the repo with no prior context.
I created the issue here: https://github.com/linkml/linkml/issues/762


Cannot transmute between types of different sizes.

Running into this issue with rustc 1.59.0
= note: source type: InnerNode (384 bits)
= note: target type: critbit::_::{closure#0}::TypeWithoutPadding (256 bits)

Hi,
Can you please post your solana --version?
Are you importing the AOB with some specific features?
Thanks

@Lcchy solana-cli 1.9.13 (src:3ac7e043; feat:1070292356), no just doing a cargo build

Bump :)

All right, can you tell me which OS you are using? I don't see where this could be coming from right now, but I suspect some versioning/dependency issue. I am trying to reproduce it.

I confirm I cannot reproduce either.
Could you upload a minimal example on Github so we can help?

sorry for delay in responding, a macbook pro Apple M1 Pro chip

We tested it on an M1 cpu and we're indeed getting the same error. We will look into it but we don't expect to find something quickly as the error is linked to how Rust interacts with the hardware, sorry.
For the record, adding #[repr(C, align(8))] to the struct to force byte alignment does not help

HI @callensm, so we don't have an M1 Mac to properly test this, but my best recommendation would be to run try and compile + run through Rosetta. This repository might be of some help https://github.com/FdezRomero/rosetta-cli.
In general we're not really too worried about this : we build programs for the Solana VM which has a fixed architecture (BPF), which means we don't really need to worry about portability of the program itself, which is why I think that it's actually necessary for any Solana developer to have a development environment which is compatible with the Solana VM's architecture. I hope this can be achieved through Rosetta.
In any case, we might implement some feature flags to enable limited compatibility with different architectures.
Another element is that reading through the M1 arch docs I don't see the difference with BPF and x86_64, which makes me wonder if this might be a compilation issue with rustc (maybe there is a way to configure the compilator to fix this).

requiring rosetta doesn't seem like a reasonable solution. there are fixes to these alignment issues and the remainder of the rust and solana ecosystem support the aarch64 now.

We chose to use bytemuck in our programs, which makes extensive use of direct type casting. This means that the way we write define our structs and data structures is highly specific to the BPF and x86_64 architectures. Maybe there is a solution I'm not seeing here, and I'd be open to suggestions, but to me it seems like it's out of our hands, at least as long as we want to keep using the most efficient data structures for our particular problems.

I looked a bit more into it and found this issue : https://github.com/solana-labs/solana/issues/21269. It seems that the issue stems from different u128 alignment requirements. I've made an attempt to solve this issue in PR #64. If anyone with an M1 Mac can test this out, it would be of great help!

I looked a bit more into it and found this issue : solana-labs/solana#21269. It seems that the issue stems from different u128 alignment requirements. I've made an attempt to solve this issue in PR #64. If anyone with an M1 Mac can test this out, it would be of great help!

Just checked and this does resolve the aarch64 build target error.
Thank you!

I think this issue can be closed, since the merge of #64

seems like aarch64 branch changes were lost, just tried to recompile on an m1 and it's not compiling again.

Hi @uwecerron! Looking at the code right now I'm not sure I understand where this might me coming from. Would you mind posting the error you're getting?
Reopening in the meantime

We have a working solution for this in the pipeline at #82. Once we're done with review we should be able to mainline this. The only issue I am seeing is that developers down the line will have to specifically support the aarch64 architecture since the struct definitions change.
u128 is a really problematic type in rustc right now, and this seems to be due to a long standing LLVM bug [1]. In theory aarch64 and x86_64 should have the exact same alignment requirements for u128 but this isn't the case. This is actually quite fortunate for us since this means that x86_64 has the same constraints as the eBPF architecture, which makes our job easier when on x86_64.
Once this bug is resolved we might have to rewrite the AOB to natively use [u64;2] in its logic, but since the issue won't be resolved anytime soon I am not sure it's worth it right now.
[1] https://github.com/rust-lang/rust/issues/54341


remove package-lock.json

It does not make sense for a library to use a lockfile.

@davidchambers Would you like to merge this PR?

Due to recently colors npm event: https://github.com/Marak/colors.js/issues/285, I think it's better not remove package lock file. The devDependencies maybe be injected with malware.
We can trust package lock file, but not package.json.

The package lock file isn't published. It should be consumed by CI to always install an exact working copy.

Currently this repo have a one year old (Jan 26, 2021) lock file with 50 vulnerabilities (23 moderate, 18 high, 9 critical)^1 during npm ci, I would choose to have no lock file in exchange with up to dated mitigation.

And no, the Dependabot automation have no difference with no lock file, it only produces spamming commits and bulky objects.

Currently this repo have a one year old (Jan 26, 2021) lock file with 50 vulnerabilities (23 moderate, 18 high, 9 critical)1 during npm ci, I would choose to have no lock file in exchange with up to dated mitigation.
Footnotes

https://github.com/ramda/ramda/runs/4748752673?check_suite_focus=true#step:3:91 ↩

The solution should be to remake the lockfile not delete it entirely.

With thousands of indirect dependencies from node_modules, what's the best strategy off to remake & commit lockfile effectively?


feat: add tx_count sorting to tokens endpoint

adds tx_count order by to brc20 tokens endpoint
the normalize/rollback is just a complete re-count, but it's only done if a rollback was seen. is this a sensible approach (assuming rollbacks don't happen often and it's okay to take a performance hit?)

Closing in favor of #215


OpenTelemetry Traces & Logging

Feature Description
OTEL Plugin / Hook-based system to track a packet's lifecycle through the broker.
Would like observability into how my hooks are dealing with connecting clients and the lifecycle of the client / packet being processed.
Current Thinking

Each packet could be it's own Trace, and every Hook interacting with said packet / transaction would have it's own span within that trace.
Potentially use the packetId as the Trace identifier

Seeing this mentioned elsewhere as well, have had some time to think about this - in my current environment, it is important for me to trace the packet outside of the "context" of the broker as well ( client -> broker -> queue -> service ) till the packet is processed by a consumer / subscriber.
That being said, what we're doing internally is inserting a trace_id to the packet using the existing OnPacketRead hook.
This trace_id is generated using the client_id and the current epoch time (ms).
Each hook then adds a span ( or span events / logs ) to this trace. This trace (for now) is closed by the service, basically the last entity that needs to deal with the packet payload.
Thoughts / suggestions @mochi-co ?

Update here :
I have quite successfully been using OnPacketRead and OnPacketProcessed Hooks to add / remove TraceIds from packet payloads
I have one request, could we also expose the entire packet to the OnACLCheck Hook ?
Examples of how I'm doing things below :
Helper Functions
package utils

import (
	"bytes"

	"go.opentelemetry.io/otel/trace"
)

var TraceSeparator = []byte(":TID:")

// Attaches a trace id to a payload
func AttachTraceId(payload []byte, traceId []byte) []byte {
	tracePayload := append(TraceSeparator, traceId...)
	return append(payload, tracePayload...)
}

// Parse a trace id from a raw byte array
func ParseTraceIdFromBytes(traceId []byte) trace.TraceID {
	tId, err := trace.TraceIDFromHex(string(traceId))
	if err != nil {
		return trace.TraceID{}
	}
	return tId
}

// Removes the trace id from a payload (if it exists)
func RemoveTraceIdFromPayload(payload []byte) []byte {
	index := bytes.Index(payload, TraceSeparator)
	if index != -1 {
		return payload[:index]
	}
	return payload
}

func TraceIdFromPacket(payload []byte) trace.TraceID {
	index := bytes.Index(payload, TraceSeparator)
	if index != -1 {
		traceIdBytes := payload[index+len(TraceSeparator):]
		return ParseTraceIdFromBytes(traceIdBytes)
	}
	return trace.TraceID{}
}

Span Functions
// returns a new span context from the given traceID
// if traceID is nil, then a new traceID is generated
func SpanFromPacketOrCtx(tracer trace.Tracer, pk *packets.Packet, operation string, ctx context.Context) (context.Context, trace.Span) {
	if pk == nil {
		return tracer.Start(ctx, operation)
	}
	// try to get the traceID from the packet
	traceID := TraceIdFromPacket(pk.Payload)
	// if valid then new span from this
	if traceID.IsValid() {
		ctx = trace.ContextWithSpanContext(ctx, trace.SpanContextFromContext(ctx).WithTraceID(traceID))
	}
	// Start will either create a new span or use the existing one
	// if the ctx already has a traceID then it will use that, otherwise it will generate a new one
	tCtx, span := tracer.Start(ctx, operation)
	// if not valid then add the traceID to the packet
	if !traceID.IsValid() {
		// add the traceID to the packet
		traceIDBytes, _ := span.SpanContext().TraceID().MarshalJSON()
		pk.Payload = AttachTraceId(pk.Payload, traceIDBytes[1:len(traceIDBytes)-1])
	}
	return tCtx, span
}

Trace Hook
func (h *TraceHook) Init(_config any) error {
	if val, ok := _config.(*Data); !ok || val == nil {
		return mqtt.ErrInvalidConfigType
	}
	config := _config.(*Data)
	if config.Tracer == nil {
		return mqtt.ErrInvalidConfigType
	}
	h.tracer = config.Tracer
	return nil
}

func (h *TraceHook) ID() string {
	return "trace-hook"
}

func (h *TraceHook) Provides(b byte) bool {
	return bytes.Contains([]byte{
		mqtt.OnPacketRead,
		mqtt.OnPacketEncode, // before sending to client
	}, []byte{b})
}

// insert the traceID into the packet payload
func (h *TraceHook) OnPacketRead(cl *mqtt.Client, pk packets.Packet) (packets.Packet, error) {
	_, span := U.SpanFromPacketOrCtx(h.tracer, &pk, h.ID(), context.TODO())
	SetClientAttributes(cl, &span)
	span.AddEvent("packet-read", trace.WithAttributes(attribute.String("topic", pk.TopicName)))
	return pk, nil
}

// remove the traceID from the packet payload if we are transmitting to a device
func (h *TraceHook) OnPacketEncode(cl *mqtt.Client, pk packets.Packet) packets.Packet {
	_, span := U.SpanFromPacketOrCtx(h.tracer, &pk, h.ID(), context.TODO())
	span.AddEvent("packet-encode", trace.WithAttributes(attribute.String("topic", pk.TopicName)))
	if U.IsClientDevice(cl) {
		span.AddEvent("transmit to device", trace.WithAttributes(attribute.String("device", string(cl.Properties.Username))))
		// remove the traceID from the packet before sending it to a device
		pk.Payload = U.RemoveTraceIdFromPayload(pk.Payload)
	} else {
		span.AddEvent("transmit to client", trace.WithAttributes(attribute.String("client", string(cl.Properties.Username))))
	}
	span.End()
	return pk
}

Having access to the packet for every Hook would complete this system
We could also allow for the traceId to be a part of the Packet struct
Thoughts @mochi-co @dgduncan ?

This discussion is now located at https://github.com/orgs/mochi-mqtt/discussions/280#discussioncomment-6755593


Sort imports following PEP 8.

Sort imports following PEP 8 - https://peps.python.org/pep-0008/#imports.

Oh also, I didn't change all imports to absolute imports like PEP 8 says. If I'm going to do that, I'll do it later in a bigger PR.


Multiline support: m tests/test.rb:9:19

My use case for m is that I'd like to run multiple test lines at the same time: m tests/test.rb:9:19 to run both tests on lines 9 and 19.
I'm submitting the PR without test coverage just to collect the feedback on my feature.
If contributors find it useful, I'll be happy to discuss the details of implementation and add test coverage.
review @zamith @qrush

I have two use cases:

I just wrote 2 new tests in shop_test.rb and I'd like to run both of them.
I want to run tests that are affected by changed lines. To do so, I have a git wrapper that checks diff and calculates what lines in shop_test.rb have been changed and then runs them.

It's worth mentioning that RSpec supports rspec path/to/a_spec.rb:37:42.

Ok, I don't have any strong reason not to add this, so I'd say it's up to implementation now. If it is not a big hit performance wise on running m without it, I should be fine with it.
Do you want to give it a go?

@kirs Are you still interested in this?

I like this idea you should totally add it.

What is holding this up. The syntax here would never affect anyone who is not trying to do this, I see no reason not to merge this.


[SPARK-48012][SQL] SPJ: Support Transfrom Expressions for One Side Shuffle

Why are the changes needed?
Support SPJ one-side shuffle if other side has partition transform expression
How was this patch tested?
New unit test in KeyGroupedPartitioningSuite
Was this patch authored or co-authored using generative AI tooling?
No.

Some implementation notes.  SPARK-41471 works by making the ShuffleExchangeExec side of the join have a KeyGroupedPartitioning, which is created by the other side's KeyGroupedShuffleSpec and is a clone of it (with the other side's partition expression and values).  That way both sides of the join have KeyGroupedPartioning and SPJ can work.
Code changes:

Remove check in KeyGroupedShuffleSpec::canCreatePartitioning that allows only AttributeReference, and add support for TransformExpression
Implement TransformExpression.eval(), by re-using the code from V2ExpressionUtils.  This allows the ShuffleExchangeExec to evaluate the partition key with transform expressions from each row.

Some fixes:

normalize the valueMap key type in KeyGroupedPartitioner to use specific Seq implementation class.  Previously the partitioner's map are initialized with keys as Vector , but then compared with keys as ArraySeq, and these seem to have different hashcodes, so will always create new entries with new partition ids.
add support in V2ExpressionUtil for Scala 'static' invoke() methods for ScalarFunctions (currently only Java static invoke() method is supported).  This was needed, for example, in our test scala YearsTransform.
Change the test YearsTransform to have the same logic as the InMemoryBaseTable.  This was pointed out in SPARK-41471 pr.

Limitations:

This feature is disabled if partiallyClustered is enabled.  Partiallly clustered implies the partitioned side of the join has multiple partitions with the same value, and does not group them.  Not sure at the moment, how the shuffle side partitioner on the shuffle side can handle that.
This feature is disabled if allowJoinKeysLessThanPartitionKeys is enabled and partitions are transform expressions.  allowJoinKeysLessThanPartitionKeys feature works by 'grouping' the BatchScanExec's partitions again by join keys.  If enabled along with this feature, there is a failure happens when checking that both sides of the join (ShuffleExchangeExec and the partitioned BatchScanExec side) have the same number of partitions.  This actually works in the first optimizer pass, as ShuffleExchangeExec's KeyGroupedPartioning is created as a clone of the other side (including partition values).  But after that there is a 'grouping' phase triggered here:

        // Now we need to push-down the common partition information to the scan in each child
        newLeft = populateCommonPartitionInfo(left, mergedPartValues, leftSpec.joinKeyPositions,
          leftReducers, applyPartialClustering, replicateLeftSide)
        newRight = populateCommonPartitionInfo(right, mergedPartValues, rightSpec.joinKeyPositions,
          rightReducers, applyPartialClustering, replicateRightSide)

This updates the number of partitions on the BatchScanExec after the grouping by join key.  But it does not update the ShuffleExchangeExec number of partitons.  Hence the error in subsequent optimizer pass:
requirement failed: PartitioningCollection requires all of its partitionings have the same numPartitions.
java.lang.IllegalArgumentException: requirement failed: PartitioningCollection requires all of its partitionings have the same numPartitions.
	at scala.Predef$.require(Predef.scala:337)
	at org.apache.spark.sql.catalyst.plans.physical.PartitioningCollection.<init>(partitioning.scala:550)
	at org.apache.spark.sql.execution.joins.ShuffledJoin.outputPartitioning(ShuffledJoin.scala:49)
	at org.apache.spark.sql.execution.joins.ShuffledJoin.outputPartitioning$(ShuffledJoin.scala:47)
	at org.apache.spark.sql.execution.joins.SortMergeJoinExec.outputPartitioning(SortMergeJoinExec.scala:39)
	at org.apache.spark.sql.execution.exchange.EnsureRequirements.$anonfun$ensureDistributionAndOrdering$1(EnsureRequirements.scala:66)
	at scala.collection.immutable.Vector1.map(Vector.scala:2140)
	at scala.collection.immutable.Vector1.map(Vector.scala:385)
	at org.apache.spark.sql.execution.exchange.EnsureRequirements.org$apache$spark$sql$execution$exchange$EnsureRequirements$$ensureDistributionAndOrdering(EnsureRequirements.scala:65)
	at org.apache.spark.sql.execution.exchange.EnsureRequirements$$anonfun$1.applyOrElse(EnsureRequirements.scala:657)
	at org.apache.spark.sql.execution.exchange.EnsureRequirements$$anonfun$1.applyOrElse(EnsureRequirements.scala:632)

This can be reproduced by removing this check and running the relevant unit test added in this pr.  It needs more investigation to be enabled in follow up pr.

Tests seem to pass, not sure why there is build error  on the last step.
************************
* Building error docs. *
************************
sh: 1: python: not found

Anyway, @sunchao , @Hisoka-X can you guys take a look?

Thanks! Merged to master/4.0


[Codegen] Add missing barrier to GPUVectorAlloc

We need another barrier before the write to share memory to wait for reads of the same memory coming from the previous iteration of a loop.

Great! Thanks for fixing it!