TypeError when using MAPPO with custom environment

Hi there,
I'm attempting to integrate Mava into my project, and I'm running into some problems.
I've created a custom environment in Unity using their MLAgents package, and I wrote a wrapper for it based on Mava's PettingZooParallelEnvWrapper. When using your mappo example with my custom environment, everything starts up fine, the agents start taking actions but after a few seconds I receive the error below.
The environment returns observations as float32 arrays, and in the wrapper's observation_spec() it's also defined as float32.
Any ideas what might be causing this? Thank you in advance.
[evaluator/0] Traceback (most recent call last):
[evaluator/0]   File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/launchpad/nodes/python/process_entry.py", line 110, in <module>
[evaluator/0]     app.run(main)
[evaluator/0]   File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/absl/app.py", line 312, in run
[evaluator/0]     _run_main(main, args)
[evaluator/0]   File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/absl/app.py", line 258, in _run_main
[evaluator/0]     sys.exit(main(argv))
[evaluator/0]   File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/launchpad/nodes/python/process_entry.py", line 105, in main
[evaluator/0]     functions[task_id]()
[evaluator/0]   File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/launchpad/nodes/python/node.py", line 72, in _construct_function
[evaluator/0]     return functools.partial(self._function, *args, **kwargs)()
[evaluator/0]   File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/launchpad/nodes/courier/node.py", line 124, in run
[evaluator/0]     instance.run()
[evaluator/0]   File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/mava/environment_loop.py", line 582, in run
[evaluator/0]     result = self.run_episode()
[evaluator/0]   File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/mava/environment_loop.py", line 447, in run_episode
[evaluator/0]     actions = self._get_actions(timestep)
[evaluator/0]   File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/mava/environment_loop.py", line 362, in _get_actions
[evaluator/0]     return self._executor.select_actions(timestep.observation)
[evaluator/0]   File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/mava/systems/tf/mappo/execution.py", line 176, in select_actions
[evaluator/0]     actions, log_probs = self._select_actions(observations)
[evaluator/0]   File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
[evaluator/0]     raise e.with_traceback(filtered_tb) from None
[evaluator/0]   File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py", line 1129, in autograph_handler
[evaluator/0]     raise e.ag_error_metadata.to_exception(e)
[evaluator/0] TypeError: in user code:
[evaluator/0]
[evaluator/0]     File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/mava/systems/tf/mappo/execution.py", line 159, in _select_actions  *
[evaluator/0]         action, log_prob = self._policy(agent, observation)
[evaluator/0]     File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/mava/systems/tf/mappo/execution.py", line 123, in _policy  *
[evaluator/0]         policy = self._policy_networks[agent_key](batched_observation)
[evaluator/0]     File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/sonnet/src/utils.py", line 70, in _decorate_unbound_method  *
[evaluator/0]         return decorator_fn(bound_method, self, args, kwargs)
[evaluator/0]     File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/sonnet/src/base.py", line 272, in wrap_with_name_scope  *
[evaluator/0]         return method(*args, **kwargs)
[evaluator/0]     File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/sonnet/src/sequential.py", line 72, in __call__  *
[evaluator/0]         outputs = mod(outputs, *args, **kwargs)
[evaluator/0]     File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/sonnet/src/utils.py", line 70, in _decorate_unbound_method  *
[evaluator/0]         return decorator_fn(bound_method, self, args, kwargs)
[evaluator/0]     File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/sonnet/src/base.py", line 272, in wrap_with_name_scope  *
[evaluator/0]         return method(*args, **kwargs)
[evaluator/0]     File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/sonnet/src/sequential.py", line 72, in __call__  *
[evaluator/0]         outputs = mod(outputs, *args, **kwargs)
[evaluator/0]     File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/sonnet/src/utils.py", line 70, in _decorate_unbound_method  *
[evaluator/0]         return decorator_fn(bound_method, self, args, kwargs)
[evaluator/0]     File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/sonnet/src/base.py", line 272, in wrap_with_name_scope  *
[evaluator/0]         return method(*args, **kwargs)
[evaluator/0]     File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/mava/components/tf/networks/continuous.py", line 117, in __call__  *
[evaluator/0]         return self._network(tf2_utils.batch_concat(observations))
[evaluator/0]     File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/sonnet/src/utils.py", line 70, in _decorate_unbound_method  *
[evaluator/0]         return decorator_fn(bound_method, self, args, kwargs)
[evaluator/0]     File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/sonnet/src/base.py", line 272, in wrap_with_name_scope  *
[evaluator/0]         return method(*args, **kwargs)
[evaluator/0]     File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/sonnet/src/sequential.py", line 72, in __call__  *
[evaluator/0]         outputs = mod(outputs, *args, **kwargs)
[evaluator/0]     File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/sonnet/src/utils.py", line 70, in _decorate_unbound_method  *
[evaluator/0]         return decorator_fn(bound_method, self, args, kwargs)
[evaluator/0]     File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/sonnet/src/base.py", line 272, in wrap_with_name_scope  *
[evaluator/0]         return method(*args, **kwargs)
[evaluator/0]     File "/home/hebrect/Documents/DeWet/Unity Projects/portals-urp-main/Python/MLAgents_Portal/lib/python3.8/site-packages/sonnet/src/linear.py", line 91, in __call__  *
[evaluator/0]         outputs = tf.matmul(inputs, self.w)
[evaluator/0]
[evaluator/0]     TypeError: Input 'b' of 'MatMul' Op has type float32 that does not match type float64 of argument 'a'.
[evaluator/0]

Are you using this env_wrapper?

@KaleabTessera have you seen this before?

Hey. Are you using this env_wrapper?

No. I had to make a custom make_environment method and didn't include the env_preprocess_wrappers. To be honest, during wrapper implementation I couldn't 100% wrap my mind around what exactly that does. Added to that, Unity' MLAgents package and its PythonAPI aren't very compatible with PettingZoo, the wrapper I created was essentially made from scratch.

I've found the issue.
Unity returns a decision_step object that contains all the agents that require an action at that step. This is how I access each agent's observations. In my environment, there are two agents. Sometimes, only one agent will require an action, which means that only one agent has an observation available. Mava requires obervations for both agents, so in the cases where there aren't any observations available, I generate an empty observation. I did not assign a dtype for this observation, so it defaulted to float64. When I converted it to float32, it solved the error.

Glad you figure it out :+1:


Inconsistency between the paper and code regarding time step sampling during training

Hi, thank you for the great work!
During reproducing, I have encountered a issue such that, in the paper, it is mentioned that:
"During training, we randomly sample a time step t and a diffusion step i and add noise ϵ = (ϵ_loc, ϵ_rot) to a ground-truth trajectory τ_t^0."
However, in the DiffuserActor class, particularly in the forward method, the implementation appears to only randomly sample a diffusion step i for the entire ground-truth trajectory, without randomly sampling a specific time step t. The noise is then added to the whole trajectory rather than a specific time step t, which seems inconsistent with the paper's description.
Code reference:
In forward:
# Sample a random timestep (i.e., diffusion step)
timesteps = torch.randint(
    0,
    self.position_noise_scheduler.config.num_train_timesteps,
    (len(noise),), device=noise.device
).long()

# Add noise to the entire clean trajectory
pos = self.position_noise_scheduler.add_noise(
    gt_trajectory[..., :3], noise[..., :3],
    timesteps
)
rot = self.rotation_noise_scheduler.add_noise(
    gt_trajectory[..., 3:9], noise[..., 3:9],
    timesteps
)

This code suggests that the noise is applied across all time steps of the trajectory based on a sampled diffusion step i, rather than sampling a specific time step t to apply the noise as described in the paper.
Could you clarify whether this is an intentional simplification in the implementation, or if the code needs to be adjusted to match the methodology described in the paper?
Thank you!

Hi,
I see the confusion, I think the paper writing is a bit unclear at this point. The code is correct.
τ_t^0 refers to a trajectory starting at timestep t. A batch of such trajectories is fetched by the data loader (randomly sampled from the training trajectories). Then, diffusion timesteps are randomly sampled, as in the snippet you pasted.
So the sentence in the paper should instead state that we randomly sample training trajectories and also randomly sample diffusion timesteps. We'll revise the writing in the next version to resolve this confusion.

Thank you for the clarification, I am closing this issue now.


updated dg

Coverage remained the same at 75.873% when pulling c451afdae0eec4604b3ed8858bd742499ec8eedd on ALim95:up_dg into 89cdd54e1b2fc00121265799283a928e3f8a23ab on CS2103AUG2017-T15-B4:master.


Textfiled outlined not work

I want to use outlined textfiled but not work
Here my code
<TextField id="excerpt"
                                       label="Deskripsi"
                                       variant="outlined"
                                       multiline
                                       fullWidth
                                       className={classes.input}
                             />

@ardyfeb Your issue has been closed because it does not conform to our issue requirements.
Please provide a full reproduction test case. This would help a lot 👷 .
A live example would be perfect. This codesandbox.io template may be a good starting point. Thank you!

Solved !, I forget to run npm update 🤭


Arom 1.18.0 x64 not showing in treeview right-click menu.

Hi there,
Having trouble with this plugin since 1.18.0 x84 upgrade on MacOSX, as I can't see the sync to/from options in the menus when right clicking on the treeview files/folders.
IT seems to still be able to "sync on save", but I need the sync folder option up and down

css has changed, the menu needs new css selector to show the "sync" entry in menu. its broken since 1.17

@ayurmedia Thank you, will test :)

same here on win 10

@plizt on windows 10 it does not work out of the box, because the rsync-command
(if available at all) has problems with path like "c:/project/" and interprets c: as user etc.
if you want to use it on windows you can use cygwin, follow instructions here:
https://atom.io/packages/atom-sync-cygwin
atom-sync only works on mac (maybe linux) which has ssh + rsync native.
atom-sync-cygwin also works on windows when cygwin is configured correctly.
c:/project is translated to /cygdrive/c/project which then works also on windows. because its a unix-style folder.

Thanks @ayurmedia ! Works for me on MacOSX

@ayurmedia , for me , edit config option doesn't work.

@rubycut , this also happens with me on mac, on windows it seems to work.
i think in atom 1.18 something changed, it was working in 1.16.
workaround is to put a .sync-config.cson in the root folder of your project and copy the sample config.
remote:
      host: "HOSTNAME",
      user: "USERNAME",
      path: "REMOTE_DIR"
    behaviour:
      uploadOnSave: true
      syncDownOnOpen: true
      forgetConsole: false
      autoHideConsole: true
      alwaysSyncAll: false
    option:
      deleteFiles: false
      exclude: [
        '.sync-config.cson'
        '.git'
        'node_modules'
        'tmp'
        'vendor'
      ]

has somebody a tipp why it might not work ?
{
        'label': 'Edit Sync Config'
        'command': 'atom-sync-cygwin:configure'
   }

in menus/atom-sync.cson is there and gets called but its not executed in lib/helper/config-helper.coffee .
configure maybe calls 'initialize' which should create and open the file in atom editor pane:
 initialise: (f) ->
    config = @getConfigPath f
    if not fs.isFileSync config
      csonSample = cson.stringify @sample
      fs.writeFileSync config, csonSample
    atom.workspace.open config

there is the sample, defined.
configFileName seems to be correct.
and @getConfigPath should return the project-folder.
There is no error in Developer / console.

its fixed in "atom-sync-cygwin" , (update to version 0.9.10 or higher)
@subscriptions.add atom.commands.add '.tree-view .full-menu .header.list-item', 'atom-sync-cygwin:configure': (e) =>
      @controller.onCreate @getSelectedPath e.target

the css selector had to be corrected in the lib/controller/atom-sync* file also
Here seems to be the cause of the trouble:
https://github.com/atom/tree-view/commit/a654ac76f8600e75e1bb2a9bb41879e0bb08dc9c
somebody changed the structure of the context-menu and the query-selectors need to be changed:
.tree-view is now a parent div, and was a class on the context-menu- items itself before it seems.
adding a space inside ".tree-view .full-menu" solves all the troubles.
/s Good thing it was clearly documented in the update-log of Atom 1.17 and 1.18 ;)
...
also the place where the "Sync" Menu appears seems totally random. i have no idea how to move it near the "New File / New Folder" like in 1.16. its a bit annoying to search for it all the time.
...
TL/DR: the "configure" command was not executed because the query-selector in controller was wrong, the rest of the code was fine.

oh there was a solution already here:
https://github.com/dingjie/atom-sync/issues/61
but never added to this "atom-sync" master. seems @dingjie is on a long vacation.

I've submitted a pull request for 1.19.1 to enable the menus as well.
see pull request #74, just needed to update the css selectors.

seems @dingjie is not maintaining the project any more.
you can switch to https://atom.io/packages/atom-sync-cygwin which is a fork of this plugin, but also working with windows (cygwin) and has small bugfixes.


[FEATURE] Preview of XML Files

Is your feature request related to a problem? Please describe.
We would like to enhance the Visualization feature to support the preview of attached XML files.
Describe the solution you'd like
We would like to implement functionality that allows users to view XML files directly within the AAS-web-ui.
Additional context
This feature would significantly improve usability for users who frequently work with the DEXPI submodel.

Hello @JS-Aibel,
can you elaborate on what the preview of XML files should look like? Is there an example?
We will consider a feature like this. For now, I will add this to our backlog.

Hi @aaronzi,
I think for now it is sufficient that it displays the XML itself as text.
We can consider style and interactivity later.


PubSub AWS SnsSqs - Conformance Tests

Describe the feature
Conformance tests for PubSub AWS SnsSqs
Release Note

RELEASE NOTE:

/assign @amimimor

After PR is merged, please, cut a PR in docs to mark this component as Beta.


Refactor sim service to include TDS integration

Changes:

Scheduler has been renamed to SimulationService

This applies to the docker-related stuff as well

forecast is now called simulate
petri is now called model for both simulate and calibrate
The arguments to calibration have significantly. See examples/ to compare
Optional TDS integration: If it's turned on, you can use an integer id instead of a string for model and dataset

simulate example modified for tds integration
calibrate example modified for tds integration

TODOs:

[ ] Document new modules and functions
[ ] Ensure environment booleans are working correctly

note that this will change the github package name @Tom-Szendrey @mwdchang


Add image/gif validation support for ProductOptionValueImage

Hi,
Is it possible to add the support of gif image to the ProductOptionValueImage entity please ?
                       - "image/gif"
Thanks in advance :D

Hello @jacquesbh,
Thanks to did it as fast as possible, but when will it be merged please ?

Hi, we will first make the CI run correctly.
I made the PR #6 for that.
When we will merge it, we will rebase the #5 and check everything is OK before merge.

@Loocos the new tag v1.0.0-RC.2 is available


CY-50 Adding Panel for Disk Statistics

Adding panel which displays disk usage of Splunk Instances.

Closing PR as already covered in - https://github.com/VatsalJagani/Splunk-Cyences-App-for-Splunk/pull/88/files


Habits need a title, reason, and start date.

1.1) Habits need a title (20 Chars long), reason (30 Chars long), and start date (Year Day Month), and if it is public or not.

Done #7


Inheritdoc now available in Visual Studio 2019 16.4

What needs to be documented:

[ ] Overridden methods automatically inherit. Release Notes
[ ] How to use the inheritdoc tag. Inheritdoc is a standard existing outside of Msft and we should link to the official inheritdoc documentation.

The inheritdoc tag in a doc summary like the example below will be inherited:
/// <summary>

/// <item><inheritdoc/></item>

/// </summary>

The class inheriting the doc will need this comment:
///<inheritdoc cref = "MyClass" />

@kendrahavens -- Kendra, thanks for your feedback; we appreciate it. To make sure we have all the information needed to triage your issue, please let us know which topic you’re referencing.

Thanks!
This could be captured in a few places:

https://docs.microsoft.com/en-us/dotnet/csharp/codedoc
https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/xmldoc/
https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/xmldoc/recommended-tags-for-documentation-comments

Since each tag has there own page in the 'Recommended-tags-for-documentation-concepts' it be good to have a page with inherit doc examples.

@kendrahavens -- Kendra, thank you for your considered suggestions.
@BillWagner -- Bill, please look into this issue, or re-assign it to this domain's author.

@mikadumont for review/visibility.

@mikadumont had written this article which covers the new <inheritdoc> tag.
I'll close this as resolved.


/proc/net/arp: No such file or directory

/proc/net/arp: No such file or directory  when arp -n is running
I installed a linux bash to use linux without installing a vm and every time I try to have my ARP table I have this message /proc/net/arp: No such file or directory
Solution
type : sudo apt update && sudo apt upgrade && arp.exe -a

/dupe #2279


Migrate site from SF to GitHub

At the moment documentation is still hosted on legacy SF website [http://dozer.sourceforge.net/]
It should be migrated to GitHub Pages.

@buzdin ; thats the plan :)

Guys, if you could create a repo for Dozer on GitHub Pages, I would be happy to help with migration.
BTW, some parts of the website are duplicated in wiki section of GitHub repo:
https://github.com/DozerMapper/dozer/wiki/Why-Map%3F
https://github.com/DozerMapper/dozer/wiki/Getting-Started
https://github.com/DozerMapper/dozer/wiki/General-Usage
Probably it makes sense to remove them from wiki to maintain only one single source of documentation.
Another closely related topic is the PDF guide. If documentation is migrated to GitHub Pages, do you have ideas how to deal with this?

@orange-buffalo ; i'd prefer if the wiki was removed, since it duplicates content (dont think i have permission to remove though).
I would like all docs to be kept in:

https://github.com/DozerMapper/dozer/tree/master/docs/asciidoc

And then built using gitbooks, which can be pushed to github pages. That way we only have 1 place for everything and its in an easy viewable / editable format, i.e.: ascidoc

@buzdin ; can you create a pages repo? or give me repo creation rights.

https://pages.github.com/

https://github.com/DozerMapper/DozerMapper.github.io

Added first draft:

https://dozermapper.github.io/

TODO: Push docs from dozer repo to gh-pages.

I see one complication with automatic publishing to GitHub Pages and wanted to ask your opinions.
There are two use-cases for documentation updates:

Improvements / fixes for documentation of existing functionality.
This would be perfectly safe, useful and convenient to automatically push updated docs to GitHub Pages.
Documentation of new functionality.
This is exactly where I see some issues. If documentation is automatically pushed to GitHub Pages, users will see new features before they are available in any released version. So the online doc will not be aligned with code available to Dozer users.

Do you have ideas how to deal with the second use case?

One more thing. I am currently trying to automate doc build with Maven. I have generated html pages from adoc files in website folder using gitbook, but got different result from what is currently deployed to GitHub Pages. I used gitbook because found scripts for this tools in docs/build-gitbook.sh. @garethahealy, have you generated website using gitbook? if yes, could you please share the config so I can embed it into automatic build?
Also I am wondering about two folders with documentation - one is website, another is docs. Would it make sense to merge them to have one source for online documentation?

@orange-buffalo ; anything else needed for this issue? or can it be closed?

@garethahealy I believe initial goal of this issue is achieved. I do not know if you plan to fully automate the process of documentation generation and publishing. Probably if yes, this will be a topic for another issue (btw, here is a quite nice Travis integration script https://gist.github.com/domenic/ec8b0fc8ab45f39403dd).
Another thing is that once all the relevant documentation is on GitHub Pages, the Wiki section of GitHub project could be disabled to not confuse users.
Other than that, nothing else from my side.

Have removed the wiki pages as the content was just duplicate of GH-pages.

Thanks!

The SF site at http://dozer.sourceforge.net/ should be removed, or a big deprecation notice added, in favor of the github repo landing page.  This old site leads to believe the project is dead!

@buzdin needs to do that @jbq, as i dont have access to SF

Should I completely remove that project on SF @garethahealy ?

@buzdin ; if you still have access to the FTP, can you download the site, zip it, and attach? I'll modify so that all pages redirect via JS to github.
If not, then, yes, remove please.

How about to migrate documentation to GitHub Pages and use MkDocs as a template for documentation?

@vadeg ; we already use GH pages and ascidoc with gitbook.

https://dozermapper.github.io/gitbook/index.html


Firefox Add on Options

Name of Defective Feature
System Settings (Required)
ToolKit Version: 12.0
Operating System (Mac/Windows/Etc): Windows 10 & Windows Server 2016
Browser (Chrome/Firefox/Safari): Firefox
Browser Version:  56.0 (32-bit) & 56.0.2 (64-bit)
Other Extensions: Roboform
Export of Toolkit Settings (Bottom Left of Settings Page)

Settings Export

[paste settings here]

Steps to Reproduce the Behavior

Selected Add-On from Firefox Menu
Clicked 'Options' next to 'Toolkit for YNAB'
It opens a new tab that directs to: chrome://kango-4f1fb113-d7d8-40ae-a5ba-9300eaea0f51/content/options.html
However, the tab is blank on both systems and I cannot change any of the settings.

Expected Behavior

I was able to successfully install the Chrome add on on the Windows Server 2016 system and successfully change the Toolkit for YNAB settings.  I expected to be able to similarly change the settings for the Firefox add on like I did for the Chrome add on on both systems.
Actual Behavior

The screen opened to a blank chrome://kango-4f1fb113-d7d8-40ae-a5ba-9300eaea0f51/content/options.html page and I was not able to change any Toolkit settings.

That version sounds quite old, which doesn't surprise me because Firefox hasn't been accepting our updates for quite a while. They've discontinued support for the way we're built, and we're currently in the process of moving over to Web Extensions, which will let us run on newer versions of Firefox.
My suggested workaround as we continue to work towards this is to use Chrome. If you can't do that, there should definitely be a later version than v0.12.0 available for Firefox.

Kevin,
I appreciate your reply.  I retrieved version 12.0 from the Firefox Add-on
Repository https://addons.mozilla.org/firefox/addon/toolkit-for-ynab/.
Please see the attached screenshot for your reference.  It appears that it
was last updated in the repository on Sep 14, 2017.
Unfortunately, my company blocks the installation of Chrome extensions,
but, oddly enough, allows the installation of Firefox Add Ons.  The server
platform that I was utilizing was my home computer, but I utilize my work
computer about 99.8% of the time and am very rarely ever on my home
computer.  That's why I was attempting to get this to work on my work
laptop.  Do you happen to have any other recommendations?  Specifically,
where I obtain a newer version of the Firefox Add-on that you had mentioned?
Very Respectfully,
Shawn McGraw
MBA MIS, CISSP, MCSA 2012, ITIL
(918) 899-9811
<EMAIL_ADDRESS>Linkedin.com/in/ShawnMcGraw
http://www.ShawnMcGraw.com
On Thu, Nov 2, 2017 at 6:10 PM, Kevin Brown<EMAIL_ADDRESS>wrote:

That version sounds quite old, which doesn't surprise me because Firefox
hasn't been accepting our updates for quite a while. They've discontinued
support for the way we're built, and we're currently in the process of
moving over to Web Extensions, which will let us run on newer versions of
Firefox.
My suggested workaround as we continue to work towards this is to use
Chrome. If you can't do that, there should definitely be a later version
than v0.12.0 available for Firefox.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/toolkit-for-ynab/toolkit-for-ynab/issues/1029#issuecomment-341584859,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AfuIQX0hjwBqoZinQPUhPDlW39DEKmoLks5sykvmgaJpZM4QQByY
.

Ah, that's version 1.2.0, not version 0.12.0, which is definitely much newer!
The challenge is that we need to upgrade to WebExtensions in order to run on the newest versions of Firefox, which we're working towards, but I don't have a firm timeframe for when it'll be completed. Once we complete the transition we should be able to run on Chrome, Firefox, and Edge.
In the meantime there are two things I'd recommend trying:

It's quite possible that even though they've blocked installing extensions from the Chrome store, you might be able to install them in development mode. To give this a shot, you can download the latest build from this link, then follow these steps to install the extension in development mode. You may need to unzip the downloaded file before you can follow those steps.

It's worth noting if you do this that the extension won't auto update, but if it works it should get you through in the meantime while we're working on the transition.

I'm not sure about the politics around this at your work, but it might be worth reaching out to the IT department at your work to ask them if they can whitelist the Toolkit.

Other than those, the only other thing I can think of is helping us transition to WebExtensions if you're a developer.

Wish I could be more helpful here, but unfortunately Firefox put us in a tight spot with the quick transition over to WebExtensions when we're not a commercial team, so we can't just put engineers on the refactor full time or anything, so it's slow going.
Does that help at all?


expose Stats types

Fixes #13014
What kind of change does this PR introduce?
Typescript types improvement.
Did you add tests for your changes?
There don't appear to be tests for asserting typescript exports. if someone can point me at some, happy to update them.
Does this PR introduce a breaking change?
No
What needs to be documented once your changes are merged?
N/A

As @alexander-akait mentioned, StatsAsset has so far been considered internal.
However, any type referenced by an exported type is also indirectly public/exposed, so we already had access to this type anyway and needed it when dealing with assets:
const fn = (asset: StatsAsset) => { ... };

stats.assets.map(fn);

One suggestion I had to reduce the surface we expose was to introduce a subset of the interface like so:
interface ExposedStatsAsset {
  type: string;
  name: string;
  size: number;
  chunkNames?: Array<string|number>;
}

However, this would mean introducing a "fake" type for the sole purpose of being exported, which feels worse than just exposing the type itself.
IMO there's nothing wrong with exposing it as even if you consider it fairly internal, any changes must happen in a new version anyway so it could be an acceptable risk for those of us consuming it.
This will also allow us to fix bundlephobia's webpack 5 integration.
If you'd rather me tackle this another way, just let me know and i will update.

Thanks


Don't export dependency on ucx

Until https://github.com/openucx/ucx/issues/9614 is resolved ucxx cannot export its dependency on ucx and must instead rely on consumers to specify this.

/merge


vino: fix dependencies again

Motivation for this change
My previous pull request was for release-15.09,
and as such has not been reapplied in master (and thus release-16.09).
Previous message:
The libXtst dependency was missing, which was required to enable remote control.
The other dependencies have been updated as well to reflect the dependencies stated at:
https://wiki.gnome.org/Projects/Vino
https://git.gnome.org/browse/vino/tree/configure.ac
Things done

[ ] Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] OS X
[ ] Linux

[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

Can you also backport to 3.20?

Whoops, sorry about that, I did not update my dev branch in a while. I have forced pushed the branch, fixing the dependencies for 3.20 as well. I have not tested it for 3.20, but there seem no real differences between the two, so it is unlikely to break.

i merged manually and i removed 3.18, since it is not present in nixpkgs anymore

@msteen tnx!


Display progress when parsing files (x/n files)

starting flow in my $HOME, it then hangs for long minutes with:
flow is still initializing; this can take some time. [parsing]

the reason is that my $HOME contains thousands of folders and files, and I suppose that flow recursively traverses them. By moving .flowconfig in a subfloder with nothing else, the issue is fixed.
To avoid this confusing message, I suggest to check what the current folder contains and outputs an appropriate error message.

This problem isn't specific to $HOME, but any directory that contains a very large amount javascript source code. I'm not sure there's much value in checking for $HOME.

I agree. Changed the issue title accordingly.

What kind of error would be helpful here? Some projects (e.g. Facebook) intentionally have many thousands of folders and files, so this isn't an error IMO.

Not necessarily an error but some feedback.
Something like a progress bar:
Listing JS files .....
Found 6547 JS files
Indexing x/6547 ....
--Martin

Seems reasonable to me. Thanks for this feature request!

I like this idea.
one interesting wrinkle is that we start parsing files in worker processes while we're still listing them in another process, so we don't actually know how much work there is to do when we would need to start showing this progress bar. we should still show as much progress as we know, but the denominator will increase too.

Some progress bar would be really good to make sure flow hasn't frozen or something else funky. Flow has been running now for me tens of minutes in a project and still claiming that "flow is still initializing; this can take some time. [processing] "
We have relatively large project where some of the files uses flow and some not.

We do display the progress in CLI now


nanodet keras model fixed to default 80 coco classes

Issue Type
Bug
Source
source
MCT Version
1.10.0
OS Platform and Distribution
Ubuntu 20.04
Python version
3.10.12
Describe the issue
re: tutorials/mct_model_garden/models_keras/nanodet/nanodet_keras_model.py

nanodet_keras_model.py can only handle 80 classes

Can be made more general by replacing the value of 80 with a class parameter
- see def nanodet_box_decoding()

Moreover, in nanodet_plus_head(n, feat_channels=128)
h[idx] = Conv2D(112, 1, name='head.gfl_cls.' + str(idx))(h[idx])
should probably be replaced with
h[idx] = Conv2D(32+num_of_classes, 1, name='head.gfl_cls.' + str(idx))(h[idx])
where num_of_classes corresponds to a new input parameter (default is 80 as in coco)

Expected behaviour
expected to be able to load pth file with a different number of classes than in coco (80)

Code to reproduce the issue
# see https://github.com/sony/model_optimization/blob/main/tutorials/notebooks/keras/ptq/example_keras_nanodet_plus.ipynb
PRETRAINED_WEIGHTS_FILE = <pth file with num of classes != 80>
# Parameters for nanodet-plus-m-1.5x_416
INPUT_RESOLUTION = 416
INPUT_SHAPE = (INPUT_RESOLUTION, INPUT_RESOLUTION, 3)
SCALE_FACTOR = 1.5
BOTTLENECK_RATIO = 0.5
FEATURE_CHANNELS = 128

pretrained_weights = torch.load(PRETRAINED_WEIGHTS_FILE, map_location=torch.device('cpu'))['state_dict']
model = nanodet_plus_m(INPUT_SHAPE, SCALE_FACTOR, BOTTLENECK_RATIO, FEATURE_CHANNELS)
load_state_dict(model, state_dict_torch=pretrained_weights)  # ValueError
...

Log output
(tested with 5 classes instead of 80)
...torch2keras_weights_translation.py", line 15

ValueError: Cannot assign value to variable ' head.gfl_cls.0/kernel:0': Shape mismatch.The variable shape (1, 1, 128, 112), and the assigned value shape (1, 1, 128, 37) are incompatible

Nanodet keras model in MCT Mode Garden supports configurable number of classes #1000


Checkout failure due to missing starting point of cp

I'm trying to run a build task with CLI on macOS but, unfortunately, I've faced with the followings:
$ circleci build
Docker image digest: sha256:525c91e01875050fbf65cdb4bc83a45744c54f9027fa05f76f88713f3d37f4e3
====>> Spin up Environment
Build-agent version 0.0.7725-9681bc36 (2018-08-09T15:10:50+0000)
Starting container circleci/buildpack-deps:latest-dind
  using image circleci/buildpack-deps@sha256:f9dea38e6583c663bbe2d950b8c9abdb771f9dd587fdb18460da2f4253c3a70a

Using build environment variables:
  BASH_ENV=/tmp/.bash_env-localbuild-1534833691
  CI=true
  CIRCLECI=true
  CIRCLE_BRANCH=feature/ci-upgrade
  CIRCLE_BUILD_NUM=
  CIRCLE_JOB=build
  CIRCLE_NODE_INDEX=0
  CIRCLE_NODE_TOTAL=1
  CIRCLE_REPOSITORY_URL=git@github.com:***
  CIRCLE_SHA1=*******************
  CIRCLE_SHELL_ENV=/tmp/.bash_env-localbuild-1534833691
  CIRCLE_WORKING_DIRECTORY=~/project

====>> Checkout code
  #!/bin/bash -eo pipefail
mkdir -p /home/circleci/project && cp -r /tmp/_circleci_local_build_repo/. /home/circleci/project
cp: cannot stat '/tmp/_circleci_local_build_repo/.': No such file or directory
Error: Exited with code 1
Step failed

I believe I'm using the latest version.
$ circleci version
0.1.1418+de70cdc

My .circleci/config.yml is absolutely simple for now, but always failed to run.
version: 2
jobs:
  build:
    machine:
      enabled: true

    working_directory: ~/project

    environment:
      SYMFONY__AWS__KEY: ****
      SYMFONY__AWS__SECRET: ****

    steps:
      - checkout
      - run:
          name: Install Docker Compose
          command: |
            pip install --upgrade pip
            pip install docker-compose

I'm not sure it is an issue about local-cli or the docker image. Should I go to  https://github.com/CircleCI-Public/example-images?

It looks like you are using the new cli from https://github.com/circleci-Public/circleci-cli so you may want to report the issue there.  You can try updating the docker image used for build by running circleci update build-agent.
I see your example config uses the machine executor. Note that only the docker executor works locally.


Make --archive save the links to the downloaded songs as soon as the songs are downloaded. Closes #2196

Title
Write downloaded songs to the archive file incrementally to prevent data loss on interruption
or
Make --archive save the links to the downloaded songs as soon as the songs are downloaded #2196
Description
This PR modifies the download/downloader.py and utils/archive.py modules to ensure that the archive file is created as soon as the script runs and updated incrementally. The initialize and add_entry methods have been added to archive.py to create the archive file if it does not already exist and add an entry to the archive file while flushing it immediately, respectively. Additionally, the download_multiple_songs method in the Downloader class has been updated to call add_entry as soon as each song is successfully downloaded. This change ensures that even if the download process is interrupted, the archive file will contain the URLs of all songs downloaded up to that point.
Related Issue
This change addresses issue #2196 where the archive file was not being updated during the download process, causing data loss if the process was interrupted.

Motivation and Context
This change is necessary to prevent data loss when using the --archive flag. Previously, the archive file was not created at the beginning of the script and if the download was interrupted, the archive file remained empty, even if most of the songs had been downloaded. By writing to the archive file incrementally, we ensure that users have an up-to-date record of all successfully downloaded songs, even in case of an unexpected interruption.
How Has This Been Tested?
Manually tested by downloading a playlist with songs that would not be present on the download source and interrupting the process at various points to ensure the archive file contains all URLs of downloaded songs.
Verified that the archive file is created at the start of the download process and updated immediately after each successful download.
Screenshots (if appropriate)

Types of Changes

[x] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

[x] My code follows the code style of this project
[ ] My change requires a change to the documentation
[ ] I have updated the documentation accordingly
[x] I have read the CONTRIBUTING document
[x] I have read the  CORE VALUES document
[ ] I have added tests to cover my changes
[ ] All new and existing tests passed

Thanks @Shajal-Kumar - I've changed the base to dev branch, can you resolve conflicts in downloader.py?

@Silverarmor To resolve conflicts, I'm just supposed to compare and remove the code and code markers from the dev branch, right?

@Silverarmor I've resolved the conflicts in downloader.py. Can you please check it?

Hi @Silverarmor, I wanted to follow up and ask if there are any other changes that I need to make.

I've requested a code review from @xnetcat who will look into it when they are free :)

This won't save the data incrementally. You would have to move the add entry code to the search and download function, while making sure that only one thread has access to the file at the time.

Hello @xnetcat, I'll make the required changes to the search_and_download function. Could you please tell me how I'm supposed to ensure that only one thread has access to the file at a time? Should I separately create a thread lock or do I have to use a part of the existing code while implementing the incremental updates to the archive?


Refusing to run with the root privileges - Aborted (core dumped)

Describe the Bug
Upon installing the server it'll come right to the end, "Success! App '1690800' fully installed.", then:
2023-12-26T19:29:40.978812100Z Success! App '1690800' fully installed.
2023-12-26T19:29:41.817764190Z Refusing to run with the root privileges.
2023-12-26T19:29:41.817823286Z libc++abi: __cxa_guard_acquire detected recursive initialization
2023-12-26T19:29:43.026621866Z Aborted (core dumped)

Your Runtime Command or Docker Compose File
version: '3'
services:
  server:
    image: 'wolveix/satisfactory-server:latest'
    volumes:
      - /mnt/gluster/games/satisfactory/config:/config
    environment:
        SERVERGAMEPORT: "7777"
        SERVERBEACONPORT: "15000"
        SERVERQUERYPORT: "15777"
        MAXPLAYERS: "4"
        PGID: "1000"
        PUID: "1000"
        ROOTLESS: "true"
        STEAMBETA: "false"
        AUTOPAUSE: "true"
        AUTOSAVEINTERVAL: "300"
        AUTOSAVENUM: "5"
        AUTOSAVEONDISCONNECT: "true"
        SKIPUPDATE: "false"
    restart: unless-stopped
    deploy:
      placement:
        constraints: [node.id == swarm-worker-01]
      resources:
        limits:
          memory: 12G
        reservations:
          memory: 8G
      labels:
        - traefik.enable=true
        - traefik.docker.network=traefik
        - traefik.constraint-label=traefik
        # Game port
        - traefik.udp.routers.satisfactory.entrypoints=udp7777
        - traefik.udp.services.satisfactory.loadbalancer.server.port=7777
        # Beacon port
        - traefik.udp.routers.satisfactory.entrypoints=udp15000
        - traefik.udp.services.satisfactory.loadbalancer.server.port=15000
        # Query port
        - traefik.udp.routers.satisfactory.entrypoints=udp15777
        - traefik.udp.services.satisfactory.loadbalancer.server.port=15777

System Specs (please complete the following information):
===== START ISSUE REPORT =====
OS:  Linux swarm-worker-01 5.14.0-362.8.1.el9_3.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Nov 7 14:54:22 EST 2023 x86_64 x86_64 x86_64 GNU/Linux
CPU: Intel(R) Core(TM) i3-9100 CPU @ 3.60GHz
RAM: 16GB (more or less dedicated to Satisfactory but I've limited the container to 12G at the moment)
HDD: Not represented well because of cluster storage. About 200GB dedicated for Satisfactory.
===== END ISSUE REPORT =====

Just in case it's not clear I'm running AlmaLinux (SELinux enabled)
Additional Context
Docker runs everything as user 1000 on my servers and I would appreciate being able to run this as rootless as well

Hey. I'm unfamiliar with that error in particular. Could you try applying the user directive in your Compose file in addition to what you already have?

Have had the same problem, even with a different user set within the ENV's.
Only runable as long as root privileges are given.
Is there a way to add a few debug break points into the code which we could enable to log stuff and send it over to you?

@Chaoschaot232 yeah, I'm not surprised to hear it. Could you fill out the issue template, please? It doesn't need to be in a new issue, I just need to know your current system environment, as well as your Docker Compose/run command

@Chaoschaot232 yeah, I'm not surprised to hear it. Could you fill out the issue template, please? It doesn't need to be in a new issue, I just need to know your current system environment, as well as your Docker Compose/run command

Sure, give me a moment and I am collecting everything you need :)

@Chaoschaot232 thanks! Could you try adding the user directive to your Docker Compose declaration for your corresponding user (for 1006)?

@Chaoschaot232 thanks! Could you try adding the user directive to your Docker Compose declaration for your corresponding user (for 1006)?

you mean smth like - USER=satisfactory right? So the name of the user behind my ID 1006.

@Chaoschaot232 no, it'd be using the user directive, rather than as an array item. Please see this Stack Overflow link :)
https://stackoverflow.com/questions/48727548/how-to-configure-docker-compose-yml-to-up-a-container-as-root
Your Compose should look something like this:
version: '3'

services:
  sf-server:
    container_name: sf-server
    restart: unless-stopped
    image: 'wolveix/satisfactory-server:latest'
    ports:
    - 15000:15000/udp
    - 7777:7777/udp
    - 15777:15777/udp
    volumes:
    - /home/data/marcel-gameserver/satisfactory/sf-micro:/config
    environment:
    - MAXPLAYERS=4
    - PGID=1006
    - PUID=1006
    - ROOTLESS=true
    - STEAMBETA=false
    - MAXOBJECTS=3000000
    user: YOUR_USERNAME_HERE
    deploy:
      resources:
        limits:
          memory: 16G
        reservations:
          memory: 8G

@Chaoschaot232 any luck with this?

I had the same issue, adding the user: worked
services:
  satisfactory-server:
    container_name: satisfactory
    hostname: satisfactory
    image: wolveix/satisfactory-server:latest
    user: 1000:1000
    network_mode: host
    #ports:
    #  - 7777:7777/udp
    #  - 15000:15000/udp
    #  - 15777:15777/udp
    volumes:
      - /home/satisfactory/config:/config
    restart: unless-stopped
    environment:
      - PGID=1000
      - PUID=1000
      - STEAMBETA=false
      - AUTOPAUSE=false
      - AUTOSAVEINTERVAL=3600
      - AUTOSAVENUM=30
      - AUTOSAVEONDISCONNECT=false
      - MAXOBJECTS=5120000
      - MAXPLAYERS=3
      - MAXTICKRATE=20
      - NETWORKQUALITY=3
      - ROOTLESS=true
      - TIMEOUT=30
networks: {}

Thanks for confirming this @williamoverstreet! In which case, given the lack of response from the original poster, I'll close this out. Please feel free to re-open this if needed


Add status implementation

CRI adds a new Status rpc interface, we should implement it.
cc/ @resouer

This change is

LGTM


Make Kubernetes EOL info available via the Management API

User story

As an admin, I want to know when Kubernetes ends their support for a minor version used by Giant Swarm.

Background
The Kubernetes project sets end of life (EOL) dates for each minor version. Once a minor version has passed the EOL date, no more patch releases are delivered for this version.
The REST API makes Kubernetes EOL dates available via the info endpoint. Our web UI and gsctl display that information in the context of a workload cluster release. Example:

With the move to the Management API and deprecation of the REST API, we need a new way to store and access this type of information.

Part of this story should be to bring the EOL info to basically every place where we show the Kubernetes version of a release or a workload cluster.

All done ✅


Define Process for OSE Generation of Docs from Templates/Sample/Example/Sandbox code into Publishing and Maintenance

Write up workflow for surfacing Sample Apps / Examples / Sandbox applications and code to publishing and long-term maintenance w/ DevRel and Programs
Automation?
Is this a different process for content type?
Does each content type end up in a new repo? For instance: tbd-sample-apps, tbd-sandboxes?
How does publishing these relate to the documentation templates describing them?
Can we align all these into repos that have a dependency on one version of Web5, therefore allowing us to version all the examples/samples/tutorials together and using the same version of Web5?

Probably is made more clear by nature of picking something like Yeeter or the VC Sandbox and working through how it's documented and potentially found a new home. Right now all these excellent apps are scattered throughout the TBD54566975 org.

@kirahsapong and i's hackMD on this process...https://hackmd.io/@mfCHHFWDRcaHJX3OqGUxLQ/r1dKQLUln/edit


WFLY-19550 Distributed timer service implementation should not schedule timers while server is suspended.

https://issues.redhat.com/browse/WFLY-19550
Requires https://github.com/wildfly/wildfly-core/pull/6117

<--- THIS SECTION IS AUTOMATICALLY GENERATED BY WILDFLY GITHUB BOT. ANY MANUAL CHANGES WILL BE LOST. --->

Wildfly issue links:

WFLY-19891

<--- END OF WILDFLY GITHUB BOT REPORT --->
More information about the wildfly-bot[bot]

Blech - something broke in the last rebase.

Rebased to resolve conflicts.

Thanks @pferraro and @rachmatowicz


Add main README

Purpose

Add main README file content

Merging as test case failures are unrelated to the Documentation changes

Merging as test case failures are unrelated to the Documentation changes


Reduce manual indexing in for_each in TileBlocks

Gives about a 1.5% speedup.

https://beta.arewecompressedyet.com/?job=master_425c5ca9_rerun%402020-07-02T20%3A16%3A44.032Z&job=reduce_for_each_overhead%402020-07-02T23%3A46%3A44.159Z

Coverage increased (+0.1%) to 81.812% when pulling adca7cb89b190e38fda16c7422716448a9ffd200 on KyleSiefring:slice_for_each_tile_blks_review into f25edcc50d8d4a885469ea452c88e7e5c3d9448b on xiph:master.


simplify implementation of R.omit

This implementation is simpler in the terms of the number of direct and indirect internal dependencies. If we're to promote the use case outlined in #829 we should avoid defining functions in terms of other functions where practical. I realize this is the antithesis of functional programming!

LGTM :cow2:

I realize this is the antithesis of functional programming!

implementation is orthogonal to interface


Fix capture during matching

Currently patterns/switch_expressions_variable_capture fails because the joint variable is captured instead of the declared variable.

@chloestefantsova This might be related to the issues you mentioned.

It seems so! That's my next target. Thanks!

Related to https://github.com/dart-lang/sdk/issues/50898


Home Indicator is Missing on Launch

Description
The indicator in the sidebar is missing when Files is launched.
Steps To Reproduce

Launch Files

Expected behavior
The sidebar indicator should be visible and on Home when the app is launched.
Files Version
Version: <IP_ADDRESS> OS Version: 10.0.22000.434
Windows Version
Windows 11 22000.434
Relevant Assets/Logs
Observed:

Expected:

It looks like it has been fixed already.


Now using Alpine Linux 3.3 and GO 1.5.3

Now using Alpine Linux 3.3 and GO 1.5.3.
I have removed the "edge" package repo for the "production" image build script. Even though "edge" offers us GO 1.6.0 (at the moment) I believe its best to use the same GO version in the "dev" and "production" images.
Images build with this patch:
logspout                  dev                 1dad5bc0db51        4 seconds ago       276.4 MB
logspout                  v3.1-dev            0a8f9c9009f7        18 minutes ago      15.2 MB

Output for dev image:
$ ROUTE=tcp://<IP_ADDRESS>:5555 make dev
# github.com/gliderlabs/logspout
link: warning: option -X main.Version dev may not work in future releases; use -X main.Version=dev
# logspout dev by gliderlabs
# adapters: udp tls raw syslog tcp
# options : debug:true persist:/mnt/routes
# jobs    : http[logs,routes]:80 pump
# routes  :
#   ADAPTER	ADDRESS		CONTAINERS	SOURCES	OPTIONS
#   raw+tcp	<IP_ADDRESS>:5555			map[]
2016/03/31 12:15:33 pump: 93bb4c2e3a0c started
2016/03/31 12:15:33 pump: 4d0f921c481e started
2016/03/31 12:15:33 pump: event: 85983efd5dff create
2016/03/31 12:15:33 pump: event: 85983efd5dff attach
2016/03/31 12:15:33 pump: event: 85983efd5dff start
2016/03/31 12:15:33 pump: 85983efd5dff started
2016/03/31 12:15:33 pump: event: 85983efd5dff die
2016/03/31 12:15:33 pump: event: 85983efd5dff destroy
^CMakefile:5: recipe for target 'dev' failed

Output for pro image:
$ docker run --rm -e DEBUG=true -v /var/runocker.sock:/var/run/docker.sock -e ROUTE_URIS=tcp://<IP_ADDRESS>:5555 logspout:v3.1-dev
# logspout v3.1-dev by gliderlabs
# adapters: raw syslog tcp udp tls
# options : debug:true persist:/mnt/routes
# jobs    : http[logs,routes]:80 pump
# routes  :
#   ADAPTER	ADDRESS		CONTAINERS	SOURCES	OPTIONS
#   raw+tcp	<IP_ADDRESS>:5555			map[]
2016/03/31 12:16:06 pump: e8b7778d4676 started
2016/03/31 12:16:06 pump: 4d0f921c481e started
2016/03/31 12:16:07 pump: event: 3f7e99e40d28 create
2016/03/31 12:16:07 pump: event: 3f7e99e40d28 attach
2016/03/31 12:16:07 pump: event: 3f7e99e40d28 start
2016/03/31 12:16:07 pump: 3f7e99e40d28 started
2016/03/31 12:16:07 pump: event: 3f7e99e40d28 die
2016/03/31 12:16:07 pump: event: 3f7e99e40d28 destroy
...

hi @mattaitchison, what do you think about this pr? regards.

@rtoma Something seems wrong, your dev image is over 18x as large as v3.1-dev...
logspout                  dev                 1dad5bc0db51        4 seconds ago       276.4 MB
logspout                  v3.1-dev            0a8f9c9009f7        18 minutes ago      15.2 MB

hi @tmornini, the dev build does no cleaning of the image. This is the size of the dev image of the current master:
logspout                                               dev                 3aa463885f61        About a minute ago   199 MB

The final image size is what you need to focus on.

@rtoma I see, thanks for schooling me. :-)


Better Questing

https://goo.gl/EfL1jQ
Crashing when loading the world using the Latest Better Questing.

I also have this problem


vb.net web application - Type 'AudioFileReader' is not defined ( NAudio.dll )

Hello All
(Designed in Visual Studio 2017/2019)
It seems I have run into a snag.
My sample code is this.
Imports NAudio.Wave
Partial Class _Default
    Inherits System.Web.UI.Page
        Dim filename As String
        filename = "G:\VS_Lessons\Media\Read\01. Got to Choose.mp3
        Dim reader As New AudioFileReader(filename)
        Dim duration As String = reader.TotalTime.ToString("mm\:ss")
        reader.Dispose()
        Label1.Text = duration
    End Sub
End Class

This only happens when I am running the site LIVE and NOT from VS.
Running it through VS, it works as it supposed to.
Live, I get the following error.
Type 'AudioFileReader' is not defined
On this line
Dim reader As New AudioFileReader(filename)
Now, I tried to register the NAudio.dll using.
gacutil -i NAudio.dll
However, I get the following line
Failure adding assembly to the cache: Attempt to install an assembly without a strong name
I followed the steps in this article here
How to add Strong Name to an existing DLL and install to the GAC
But I get a FAILURE when trying to create the DLL file. (Maybe I am doing it wrong because I am not the owner of the file. Not really sure, this is the first time I have ever done anything like this and was trying everything I could find online)
Any assistance on this would be mighty grateful.
Wayne

The website has to be converted into an Application.
IIS | Web Site |
Right-click on the web site name or sub-folder name
And choose [Convert to Application]
Run the script and it works.

I wrote a knowledge base article on this issue as well.
NAudio.dll - Type 'AudioFileReader' is not defined
I hope it helps others out like it helped me.
Wayne


Simulation

PR not ready. Here for tracking changes

Sure, one question before we do: can we get a (super barebones) RosTest that works using this branch and multiple-roscore.
nothing that needs to be merged in, but a sanity check that they will interact well.

About the test, I can write something that will literally spin up the launch file, and assert true.
I can't guarantee that we'll be able to write a test that will actually check if actuators work though, because of #161

Not merging 180 in is fine. As long as it passed locally on some branch that's fine. I didn't even necesarilly want the test written on here, just confirmation that it works.

Oh Alright. Just tested locally in a new merged branch and it works. I'll revert the file in this PR and we can merge.
Make a note somewhere in a readme for test utils that when issues arise its a good idea to clean pyc files. I was having trouble with a test hanging (because a message wasnt being received so it just waited on stdin input forever) and cleaning pyc files fixed it.

noting that this closes #182, #160, and #159


[MongoDB Connector] Improve update operation return type

Suggested improvement
Discord: https://discord.com/channels/957996897782616114/1088153674204860426/1088153674204860426
Currently it returns only the modified count for update operation. https://github.com/ballerina-platform/module-ballerinax-mongodb/blob/71b73c27314fd2e61efdad82d08c7cd6c716a821/mongodb-native/src/main/java/org/ballerinalang/mongodb/MongoDBCollectionUtil.java#L182
We can improve the return type with more detail
Ref:
https://mongodb.github.io/mongo-java-driver/3.8/javadoc/com/mongodb/client/result/UpdateResult.html

Suggested Labels (optional):

Suggested Assignees (optional):

Duplicate of https://github.com/ballerina-platform/ballerina-library/issues/4988


Extend tf.keras.Model.evaluate with class_weight

System information

TensorFlow version (you are using):
TF 2.0.0
Are you willing to contribute it (Yes/No):
Yes

Describe the feature and the current behaviour/state.
Currently tf.keras.models.Model.fit method allows the user to pass either 'sample_weight' and 'class_weight' parameters. These are used to compute at some point a standardised 'sample_weights' and used later on while calculating the loss.
This feature request is about extending the 'tf.keras.Model.evaluate' API so that is permits using class_weight directly. The evaluate function already permits for sample_weight.
Will this change the current api? How?
current API
evaluate(
    x=None,
    y=None,
    batch_size=None,
    verbose=1,
    sample_weight=None,
    steps=None,
    callbacks=None,
    max_queue_size=10,
    workers=1,
    use_multiprocessing=False
)

new API
evaluate(
    x=None,
    y=None,
    batch_size=None,
    verbose=1,
    sample_weight=None,
>>>    class_weight=None,
    steps=None,
    callbacks=None,
    max_queue_size=10,
    workers=1,
    use_multiprocessing=False
)

Who will benefit with this feature?
Those users of the API who would like to perform the evaluation of a model that was trained with bespoke class weights.
Any Other info.

Since this issue is opened for a long time, let me work on this.

Is there any solution please to add class_weight to the evaluate() method?

Sorry for the long wait. I checked with Francois about the issue, and seems that it is expected to not having class_weights for model.evaluate.
Quote from Francois:
"The reason we don't support class weights in evaluate is that the class_weight argument represents sample weights that are computed from the labels, but the labels should not be an input to the model during evaluation. During training this is fine, but during evaluation this represents a data leak from the labels to your metrics. If you used class weighting in evaluate, your score would not be reproducible on real test data (when you don't have the labels).
So this is conceptually wrong."
I am closing this issue since it is working as intended.


Feature request - Container linking

Hi, newbie question here, but I am not seeing anything in the settings to be able to link two containers trough the UI ?

@runseb, you can currently do container linking through the use of environment variables.
Screenshot attached.

is it full linking or its just like connecting from host? If its first option can you post some details? Eventualy is that functionality on some roadmap?

@mchiang0610 ?

The linking functionality needs some more work. It's a common use-case to link containers, and a vague reference to ENV variables is not that helpful to be honest. The user interface SHOULD clearly indicate how to link containers intuitively, and the documentation is really lacking fundamental information on this subject.

Yea I think this should be re-opened because if you're downloading an official image, lets say an app that requires a database, the app container isn't going to know about the database container. Linking containers is what the container publisher is expecting you to do so environment variables do not help you in this case.

@gkawka Not yet. At the moment you have to use environment variables.
There is a pull request for container linking https://github.com/kitematic/kitematic/pull/479.
Reopening this issue

This is actually a duplicate of issue 14. Can we move discussion there? https://github.com/kitematic/kitematic/issues/14
Sorry for the confusion on my part. :(

I'd really appreciate some explanation regarding linking containers through the use of environment variables in the meantime. I can't find any solid information on this aspect.

Would also appreciate any help on linking containers using environment variables. Anything published by now?

+1... how do you use env variable for linking?

Yup, would be nice to know what the ENV VAR mechanism is.

+1 for info on linking containers with environment variables, anyone?

For me, i needed to link my Wordpress container to my sql container, i did this in the end by adding some environment variables which i obtained from the settings area within the sql container.

This could be solved via the networking feature discussed here:
https://github.com/docker/kitematic/issues/1249

In the meantime you can do the linking in a docker compose yml file and fire it up via the kitematic Docker cli.

+1
It's clearly missing!
A management of the "links", and a management of "composes"
Here is an example of the Synology Docker:


Adds config tempaltes

This is draft PR, which will bring easier way to create task configs, which are language-aware.
The original TaskConfig is kept and can be still used when one needs to be very fancy or his use-case is not supported (chat templates, perplexity etc.., I have only created what I need for now).
Plan is to create following templates:

MultiChoiceTaskConfig (implemented and we can discuss if it make sense)
QuestionAnswerTaskConfig
NLITaskConfig
CopaTaskConfig
BoolQATaskConfig
ContinuationTaskConfig (Will find better name, should be used for hellaswag/xcodah/winogrande type of tasks)

Here is example task definition: https://github.com/huggingface/lighteval/compare/config_templates?expand=1#diff-e849815a8af98441f28b2bc603d2d90dd5b00ea2b029358b5cd5af1e392e9fbcR77

The templates folder mixes up a lot of items of different logic level, we'll have to disuss the structure IRL. Do we really need "NLITaskConfig, CopaTaskConfig, BoolQATaskConfig", etc? I think it would be better to aim for a general structure instead of having lots of small ones

I did remove the BoolQATask, for the rest I think they have their place as they have some logic in them.
If you would be super strict only the continuation and multichoice would make the cut. The thing is I will need to for the multilang tasks, so I will create it one way or another. If you insist on greater seperation we can have general/task templates.
Secondly I decided that I won't create a question-answer template as one can just use multichoice with cf to achieve just that


Variables in config

Describe the feature
I would like to have variable in the config.py file. One use case is a custom dataset path that may be known only ad runtime. Currently, there is no way to inject a variable in the config file.
Thank you!
Cheers,
Francesco

The config is modifiable. How did you inject the variable? Any use case?

Yes, it is :) Imagine for example that I don't want to call train.py from the cl.

@FrancescoSaverioZuppichini Very good, but you can also directly add variables in the configuration and read them in the program yourself, which would be better.

@hhaAndroid No I can, what if I need to add an instance of a class? or anything that is not a literal


Fix next warnings

This is fixing this warning when running the app in dev mode:
warn  - Invalid next.config.js options detected:
  - The value at .i18n has an unexpected property, keySeparator, which is not in the list of allowed properties (defaultLocale, domains, localeDetection, locales).
  - The value at .i18n has an unexpected property, nsSeparator, which is not in the list of allowed properties (defaultLocale, domains, localeDetection, locales).
  - The value at .i18n has an unexpected property, defaultNS, which is not in the list of allowed properties (defaultLocale, domains, localeDetection, locales).
  - The value at .i18n has an unexpected property, loadLocaleFrom, which is not in the list of allowed properties (defaultLocale, domains, localeDetection, locales).

@testerez the PR should be done to canary, not to master. Anyway thank you so much for your contribution

It's my pleasure @aralroca , thanks for maintaining this library!


AKS - Autoscaler General Availability

Is there any ETA on General availability of this feature (AKS - autoscaler)

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 077f1bb5-2ba9-490c-2c6e-0de292266539
Version Independent ID: 0fd87b4f-88a4-c2eb-3ebc-f2b113ac203c
Content: Use the cluster autoscaler in Azure Kubernetes Service (AKS)
Content Source: articles/aks/cluster-autoscaler.md
Service: container-service
GitHub Login: @iainfoulds
Microsoft Alias: iainfou

@Sethu2488 Thanks for your question. We are checking on this and will respond to you soon.

@Sethu2488 At this time we do not have any ETA information we can share. As new updates are available we will be sure to update the documentation.

How about now?


SWARM-1135 - Unable to selectively enable loggers using -D

Motivation
It is useful to enable some logging categories (org.wildfly.swarm.config
and org.wildfly.swarm.metrics) from the command-line during development.
Modifications
Promote the -Dswarm.logging.= properties to actually
adjust the related loggers early, and ensure that they are propagated
to the logging subsystem if applicable.
Result
-Dswarm.logging.org.wildfly.swarm.config=DEBUG works.

[X] Have you followed the guidelines in our Contributing document?
[X] Have you created a JIRA and used it in the commit message?
[X] Have you checked to ensure there aren't other open Pull Requests for the same issue?
[X] Have you built the project locally prior to submission with mvn clean install?

Dang it.
On Mon, Feb 27, 2017 at 5:43 PM George Gastaldi<EMAIL_ADDRESS>wrote:

@gastaldi commented on this pull request.
In
fractions/wildfly/logging/src/main/java/org/wildfly/swarm/logging/LoggingFraction.java
https://github.com/wildfly-swarm/wildfly-swarm/pull/387#discussion_r103332187
:

             .rootLogger(level, CONSOLE);

   Properties allProps = System.getProperties();

   for (String name : allProps.stringPropertyNames()) {

       if (isSimpleLoggerName(name)) {

           System.err.println("name-->" + name);

Is this meant to be here or was just used for debugging purposes?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/wildfly-swarm/wildfly-swarm/pull/387#pullrequestreview-24115512,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAA-T_1dsqk1Hjify6NwN-Z3TiRDSzuYks5rg1GKgaJpZM4MNo3y
.


Modularization and testing

The current structure is not testable. I would suggest decomposing all steps in our workflow into individual modules (essentially functions that take some input and generate some output). These should go be placed into individual folders and should be tested separately. The pydoit workflow would then by an integration test that uses these individual modules. Note that this integration test should include the dpendencies and targets (maybe that is the best thing to start with, since that is essentially the definition of the interfaces of the individual modules.

Can we close this issue?
Although the complete thing is not yet running, currently all working steps are modular and being tested.

We could probably close that issue since we are working on it iteratively and have at least started.


Addon API: proposal

Hey everyone, i'll try to make a resume(we talked a lot) of the conversation in the #addon channel, do not hesitate to add your thoughts and comments about the idea talked here or new one :)
@tmeasday made a quite overview about addons that resume the thing quite well i think, so i think it's a good start :
There are basically three different kinds of "decorators" that you might want.
Depending on where in the "composition" of the story they live:

Decorators that alter the story in a real way (think knobs, or the prop-generator one).
Really, these are classic HOCs

Decorators that want to inspect the "real" story and report the result somewhere
(typically to the channel but it could be somewhere else i guess) (info, jsx, etc).
These have been called "middleware" although I think "inspectors" or "connectors" or "probes" would be more descriptive.

Decorators that provide some kind of rendering context to the story, be it a container class, fonts/bg colors, or react context (i.e. providers).
Some of that is visual, some of it is needed for the story to even execute, but it is not "part" of the story

Some goals of the new/reworked API are :

Make a clear and simple API to create addon
Make addon clear about their use (type of decorators describe above)
Avoid the current problem with the overwrite of .add() by addons like infos on jsx (the main is that they can't be use together)
Avoid the current pattern where addons "pollute" the stories
Create "best practices" and "guides" about creating addons for the community
Respond to the needs from the developer (addons and storybooks evolves and we face new challenges like a playground for example)

You can find some gist/branches of proposed API below (some are more ideas rather than concept, but i think it's worth listing them here anyway) :

proposal 1
proposal 2
proposal 3 and 4
proposal 5
proposal 6

For my proposals (1 and 2)
the idea behind is this one (comes from slack channel conversation too):
An other reason to separate decorators and "middleware"(addons like infos, jsx, ...)
decorators modify or provide context to your component and allow them to render.
Your want them applied in the right order.
Middleware are side effect for storybook, they do not need to alter your story, only get informations. They should receive the story rendered with the decorators and that's it. Each middleware should receive the same story.
This what i mean by pollute the story (it return the story with new JSX wrapping your story) :

export default story => {
    return (
        <Provider renderer={getRenderer()} mountNode={getMountNode()}>
            <ThemeProvider theme={theme}>
                <Container>
                    {story()}
                </Container>
            </ThemeProvider>
        </Provider>
    );
};

// OR storybook addon

import React from 'react';

import { storiesOf } from '@storybook/react';
import { WithNotes } from '@storybook/addon-notes';

import Component from './Component';

storiesOf('Component', module)
  .add('with some emoji', () => (
    <WithNotes notes={'A very simple component'}>
      <Component></Component>
    </WithNotes>
  ));

the problem with this is the fact that addons like storyshots will render the decorator in the snapshot and not just the story.
If we use decorators/connectors we can avoid that by sending the final story to the connectors without the extra code from other connectors.

One of the text from the conversation that's worth mentioning and keep a trace of from @ndelangen

So.. I want to move away from using context at some point.. by making connectors responsible for modifying the props-object, and decorators for just wrapping things for decorative purposes, I think this will become quite easy to do.
Both decorators and connectors can have connections to panels.
Decorators should be renderable things (class extending from components, or just a function)
Connectors should be pure functions (possibly async)
Decorators cannot set props on a story. (this is actually current behaviour!) But this is circumvented by using context.

So the use case for connectors is to provide a clean and robust way to provide props (over time) to a story.
As a bonus we get less components rendered in the preview, possibly making debugging easier for users.

This compose is just a reduce over all functions… applying them in order… Which is exactly what we currently do. Am I missing something or is this just a alternative-method for setting which connectors/decorators a user wants on a story?
I think an array will be a whole lot LESS confusing to people. It’s very similar to how webpack is configured.

Here's a pretty simple proposal that moves the onus of categorizing and ordering decorators onto the addon author and avoids the user having to understand what the different types are and even think to much about the order they apply:
https://gist.github.com/tmeasday/775c5572271121bb2a71153ac5f2cd7a

Hi, I'd like to present my proposal which is compatible with current api.
https://gist.github.com/Stupidism/4f40f3a9688cb09b3a7cb995831d3be9

Are we still debating this proposal? If not, let's close this and open a new issue for new API proposals.


idea: disable cmd+Q

Hi,
don't remembering all shortcuts by heart (and being lazy and irresponsible -- by just trying keystrokes), happened to me a couple of times that I've killed completely Mastodon (and Fiji) by pressing Cmd+Q. I did that in an attempt to increase radius of a spot, which is decreased with Q and I did too much, and I was hoping that something like alt+q, shift+q, cmd+q might do the opposite...
BTW: Cmd+, opens also the Preference dialog of Fiji (the single check-box one with "Clear preferences").
Was on Mac.
Vlado

Firstly: shouldn't this issue be in imagej/imagej, since you're asking for a global disable of Cmd+Q for all of ImageJ?
Secondly: I was going to argue that we shouldn't override default Mac behavior. But then Chrome recently implemented a thing where you have to press Cmd+Q twice in order to really quit. So I guess we could do this... if you can figure out how to override that shortcut on the Java side.
Otherwise, you can always override the Cmd+Q shortcut globally on your system using some hack like this.

Please no hack....
As much as possible we do not want to mess with existing UIs.
If Shift+Q does not work for changing the spot size, please change the keymap now that it is possible in Mastodon.

To my defense :) It has not occurred to me that the fact that Fiji closes upon Cmd+Q is not due to Fiji's own will (which explains to me also why similar short cut does not close FIji in other OSes), but it's an Apple's goodie.
Shift+Q works, I guess, it is probably slow/fast radius decrease.. I wanted the radius increase. I now remember that E is the answer.

PS: If you would be interested in starting a discussion about changing a few default shortcuts (before Mastodon gets widespread), here's a couple of ideas:

avoid using Cmd+whatever, I would stay with omnipresent shift, ctrl, alt only
swap ctrl+A (add linked spot) with A (add spot), this saves one key stroke per timepoint per cell... at least I would always do: ctrl+A to start a track and follow cell with just A (not ctrl+A) and space/q/e to adjust, then A again...

swap ctrl+A (add linked spot) with A (add spot), this saves one key stroke per timepoint per cell... at least I would always do: ctrl+A to start a track and follow cell with just A (not ctrl+A) and space/q/e to adjust, then A again...

Normally this is the case: to create a linked spot, just press 'A' inside an existing spot. Ctrl A should not do anything no?

Hmm, I think I took the screen shot from already altered project..  sorry. With Default (built in) keymap, there is A for both add linked spot and add spot, and it works just fine (don't know what I was doing days back so that I adopted this habit of changing this shortcut). My bad, case closed.
Your favourite trouble maker,
V.


Rocm Docker Image Fails to Load Dynamic Libraries (0.1.27-rocm)

TLDR: The 0.1.27-rocm cannot find the correct version of rocm libraries.
I start the docker image using the following command:
sudo -H -u ollama /usr/bin/podman --runtime /usr/bin/crun run --gpus all --rm -v /usr/share/ollama/.ollama:/root/.ollama -p 11434:11434 --name ollama 'ollama/ollama:0.1.27-rocm'

Ollama appears to identify the AMD GPU without issue
...omitted for brevity...
msg="Extracting dynamic libraries..."
time=2024-02-24T00:19:07.462Z level=INFO source=payload_common.go:146 msg="Dynamic LLM libraries [cpu_avx cuda_v11 rocm_v5 cpu cpu_avx2 rocm_v6]"
time=2024-02-24T00:19:07.462Z level=INFO source=gpu.go:94 msg="Detecting GPU type"
time=2024-02-24T00:19:07.462Z level=INFO source=gpu.go:265 msg="Searching for GPU management library libnvidia-ml.so"
time=2024-02-24T00:19:07.486Z level=INFO source=gpu.go:311 msg="Discovered GPU libraries: []"
time=2024-02-24T00:19:07.486Z level=INFO source=gpu.go:265 msg="Searching for GPU management library librocm_smi64.so"
time=2024-02-24T00:19:07.492Z level=INFO source=gpu.go:311 msg="Discovered GPU libraries: [/opt/rocm/lib/librocm_smi64.so.5.0.50701 /opt/rocm-5.7.1/lib/librocm_smi64.so.5.0.50701]"
time=2024-02-24T00:19:07.504Z level=INFO source=gpu.go:109 msg="Radeon GPU detected"
time=2024-02-24T00:19:07.504Z level=INFO source=cpu_common.go:11 msg="CPU has AVX2"

Then I attempt to run ollama from the client via:
echo 'test' | ollama run llama2

And observe the following errors:
time=2024-02-24T00:19:40.892Z level=INFO source=dyn_ext_server.go:90 msg="Loading Dynamic llm server: /tmp/ollama1271109174/rocm_v5/libex
t_server.so"
time=2024-02-24T00:19:40.892Z level=INFO source=dyn_ext_server.go:150 msg="Initializing llama server"
time=2024-02-24T00:19:40.892Z level=WARN source=llm.go:162 msg="Failed to load dynamic library /tmp/ollama1271109174/rocm_v5/libext_serve
r.so  Unable to init GPU: invalid device ordinal"
time=2024-02-24T00:19:40.893Z level=WARN source=llm.go:162 msg="Failed to load dynamic library /tmp/ollama1271109174/rocm_v6/libext_serve
r.so  Unable to load dynamic library: Unable to load dynamic server library: libhipblas.so.2: cannot open shared object file: No such fil
e or directory"
time=2024-02-24T00:19:40.894Z level=INFO source=dyn_ext_server.go:90 msg="Loading Dynamic llm server: /tmp/ollama1271109174/cpu_avx2/libe
xt_server.so"
time=2024-02-24T00:19:40.894Z level=INFO source=dyn_ext_server.go:150 msg="Initializing llama server"
llama_model_loader: loaded meta data with 23 key-value pairs and 291 tensors from /root/.ollama/models/blobs/sha256:8934d96d3f08982e95922
b2b7a2c626a1fe873d7c3b06e8e56d7bc0a1fef9246 (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
...omitted for brevity...

Note the error:
Unable to load dynamic library: Unable to load dynamic server library: libhipblas.so.2

If I grab a shell on the image, I can see that the libhipblas.so.2 library does not exist, only .1 and .0 do:
$ ls /opt/rocm/lib/libhipblas* | cat
/opt/rocm/lib/libhipblaslt.so
/opt/rocm/lib/libhipblaslt.so.0
/opt/rocm/lib/libhipblaslt.so.0.3.50701
/opt/rocm/lib/libhipblas.so
/opt/rocm/lib/libhipblas.so.1
/opt/rocm/lib/libhipblas.so.<IP_ADDRESS>701

I ran into a similar issue with a mismatch between library versions when running outside of docker which I was able to mitigate as described here: https://github.com/ollama/ollama/issues/2685#issuecomment-1961666228 (TLDR: just symlinking new versions to old versions). I believe even if I fixed the libhipblas.so issue, the other libraries would also need to be fixed as they were in the linked comment. Additionally, the issue here appears to be the opposite of the scenario described in the linked comment where the old versions exist but ollama wants to new versions (I believe).
I've looked at the Dockerfile and won't quite understand how the 0.1.27-rocm image is built so am not able to offer guidance on a fix.

Following @GabrielHaeusler 's approach (Thanks for the info!), I was able to successfully get ollama in podman to use the GPU via the following command:
sudo -u ollama -H /usr/bin/podman --runtime /usr/bin/crun run --rm -v /usr/share/ollama/.ollama:/root/.ollama -p 11434:11434 --name ollama  --device=/dev/kfd:/dev/kfd --device=/dev/dri:/dev/dri 'ollama/ollama:0.1.27-rocm'

I tried all variants of using HSA_OVERRIDE_GFX_VERSION, passing in kfd, and pssing in dri.  I would have thought --gpus all would handle access to these devices but I guess not. Therefore, I removed --gpus all in the above. HSA_OVERRIDE_GFX_VERSION did also not appear to be required for my setup.
Should probably have noted that I'm using linux-hardened with a 6900xt but not selinux.

To clarify some of the error messages, our current build uses the v5 ROCm image base, so the actual error was
time=2024-02-24T00:19:40.892Z level=WARN source=llm.go:162 msg="Failed to load dynamic library /tmp/ollama1271109174/rocm_v5/libext_serve
r.so  Unable to init GPU: invalid device ordinal"

Since v5 fails, it attempts the v6 variant, and the second failure to find the v6 dependencies is ~expected since the base image is v5, not v6.
It sounds like you got it working with permission fixes to expose the drivers.  I'll keep this issue open to track improving the docs for radeon container usage.

I'm seeing behavior which falls back to CPU no matter what. Granted, I am definitely in unsupported territory as I want to see if I can get inference to run on the 760m inside this 7640hs. /dev/kfd is exposed and ollama seems to recognize it, but fails on the same invalid device ordinal as above:
$ podman run --rm --env HSA_OVERRIDE_GFX_VERSION=gfx1103 --device /dev/kfd --device /dev/dri -p 11434:11434 --name ollama ollama/ollama:rocm
[snip]
time=2024-03-16T02:53:16.858Z level=INFO source=amd_linux.go:88 msg="detected amdgpu versions [gfx1103]"
time=2024-03-16T02:53:16.859Z level=INFO source=amd_linux.go:246 msg="[0] amdgpu totalMemory 2048M"
time=2024-03-16T02:53:16.859Z level=INFO source=amd_linux.go:247 msg="[0] amdgpu freeMemory  2048M"
time=2024-03-16T02:53:16.859Z level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-03-16T02:53:16.859Z level=WARN source=amd_linux.go:53 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers: amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
time=2024-03-16T02:53:16.859Z level=INFO source=amd_linux.go:88 msg="detected amdgpu versions [gfx1103]"
time=2024-03-16T02:53:16.859Z level=INFO source=amd_linux.go:246 msg="[0] amdgpu totalMemory 2048M"
time=2024-03-16T02:53:16.859Z level=INFO source=amd_linux.go:247 msg="[0] amdgpu freeMemory  2048M"
time=2024-03-16T02:53:16.859Z level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-03-16T02:53:16.888Z level=INFO source=dyn_ext_server.go:90 msg="Loading Dynamic llm server: /tmp/ollama2880729444/runners/rocm_v60000/libext_server.so"
time=2024-03-16T02:53:16.888Z level=INFO source=dyn_ext_server.go:150 msg="Initializing llama server"
time=2024-03-16T02:53:16.900Z level=WARN source=llm.go:170 msg="Failed to load dynamic library /tmp/ollama2880729444/runners/rocm_v60000/libext_server.so  Unable to init GPU: invalid device ordinal"
time=2024-03-16T02:53:16.901Z level=INFO source=dyn_ext_server.go:90 msg="Loading Dynamic llm server: /tmp/ollama2880729444/runners/cpu_avx2/libext_server.so"
time=2024-03-16T02:53:16.901Z level=INFO source=dyn_ext_server.go:150 msg="Initializing llama server"

Got it fully running! The key for me was sudo setsebool container_use_devices=1 and HSA_OVERRIDE_GFX_VERSION=11.0.0:
podman run --rm --env HSA_OVERRIDE_GFX_VERSION=11.0.0 --device /dev/kfd --device /dev/dri -p 11434:11434 --name ollama ollama/ollama:rocm
========================================= ROCm System Management Interface =========================================
=================================================== Concise Info ===================================================
Device  [Model : Revision]    Temp    Power     Partitions      SCLK  MCLK     Fan  Perf  PwrCap       VRAM%  GPU%
        Name (20 chars)       (Edge)  (Socket)  (Mem, Compute)
====================================================================================================================
0       [0xb002 : 0xc3]       63.0°C  53.055W   N/A, N/A        None  2800Mhz  0%   auto  Unsupported   64%   99%
        Phoenix1
====================================================================================================================
=============================================== End of ROCm SMI Log ================================================


Bump all versions of extensions to do a publish with the proper workflow

Bump all versions of extensions to do a publish with the proper workflow which should enable annotation propagation to the UIplugin resource, which is needed for some checks with the shouldNotLoadPlugin method in Dashboard

Should we not change minor version instead of fix one to stick to semver or is there a reason we do not want to?


python312Packages.sentence-transformers: 2.7.0 -> 3.1.0

Description of changes
As #341872 was stalling, I am putting up a new PR to address the update.
cc @happysalada
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 342752 run on aarch64-linux 1

6 packages marked as broken and skipped:

khoj
khoj.dist
open-webui
open-webui.dist
python311Packages.private-gpt
python311Packages.private-gpt.dist

16 packages built:

aider-chat
aider-chat.dist
python311Packages.langchain-huggingface
python311Packages.langchain-huggingface.dist
python311Packages.llama-index-embeddings-huggingface
python311Packages.llama-index-embeddings-huggingface.dist
python311Packages.manifest-ml
python311Packages.manifest-ml.dist
python311Packages.sentence-transformers
python311Packages.sentence-transformers.dist
python312Packages.langchain-huggingface
python312Packages.langchain-huggingface.dist
python312Packages.manifest-ml
python312Packages.manifest-ml.dist
python312Packages.sentence-transformers
python312Packages.sentence-transformers.dist

Result of nixpkgs-review pr 342752 run on aarch64-darwin 1

6 packages marked as broken and skipped:

khoj
khoj.dist
python311Packages.llama-index-embeddings-huggingface
python311Packages.llama-index-embeddings-huggingface.dist
python311Packages.private-gpt
python311Packages.private-gpt.dist

14 packages built:

open-webui
open-webui.dist
python311Packages.langchain-huggingface
python311Packages.langchain-huggingface.dist
python311Packages.manifest-ml
python311Packages.manifest-ml.dist
python311Packages.sentence-transformers
python311Packages.sentence-transformers.dist
python312Packages.langchain-huggingface
python312Packages.langchain-huggingface.dist
python312Packages.manifest-ml
python312Packages.manifest-ml.dist
python312Packages.sentence-transformers
python312Packages.sentence-transformers.dist

Result of nixpkgs-review pr 342752 run on x86_64-darwin 1

20 packages marked as broken and skipped:

khoj
khoj.dist
open-webui
open-webui.dist
python311Packages.langchain-huggingface
python311Packages.langchain-huggingface.dist
python311Packages.llama-index-embeddings-huggingface
python311Packages.llama-index-embeddings-huggingface.dist
python311Packages.manifest-ml
python311Packages.manifest-ml.dist
python311Packages.private-gpt
python311Packages.private-gpt.dist
python311Packages.sentence-transformers
python311Packages.sentence-transformers.dist
python312Packages.langchain-huggingface
python312Packages.langchain-huggingface.dist
python312Packages.manifest-ml
python312Packages.manifest-ml.dist
python312Packages.sentence-transformers
python312Packages.sentence-transformers.dist


How to accelerate Kinectv2 using Jetson TX2?

Hi, thanks for sharing. Here are some questions:
When running the Libfreenect2/build/bin/Protonect, I noticed that "Environment variable LIBFREENECT2_PIPELINE can be set to cl, cuda, etc to specify the pipeline".
And when running the kinect2_bridge on ROS, the "depth_method" could be chosen among "cpu, opencl, opengl, cuda...", but the "reg_method" could be only set to "cpu ". When "reg_method" set to "opencl", the problem "OpenCL registration is not available!" occurred.
Also the frame rate of HD/QHD point clouds was very low.
So, is there a way to accelerate the HD/QHD points cloud acquisition on ROS for Jetson TX2?
Looking forward for your reply.

There's something up with OpenCL acceleration on (at least) L4T, I've been digging into it a bit myself as I want the same thing you do ;)
I've noticed specific other apps that use OpenCL also have similar issues, we just need to figure out what's up here.


chore(deps): update go

This PR contains the following updates:

Package
Type
Update
Change

github.com/DataDog/datadog-go/v5
require
minor
v5.3.0 -> v5.4.0

github.com/argoproj/gitops-engine
require
patch
v0.7.1-0.20230906152414-b0fffe419a0f -> v0.7.3

github.com/coreos/go-oidc/v3
require
minor
v3.8.0 -> v3.9.0

github.com/golang-jwt/jwt/v5
require
minor
v5.0.0 -> v5.2.0

google.golang.org/api
require
minor
v0.152.0 -> v0.153.0

google.golang.org/grpc
require
minor
v1.59.0 -> v1.60.0

k8s.io/api
replace
minor
v0.24.2 -> v0.28.4

k8s.io/apiextensions-apiserver
replace
minor
v0.24.2 -> v0.28.4

k8s.io/apimachinery
replace
minor
v0.24.2 -> v0.28.4

k8s.io/apimachinery
require
minor
v0.27.3 -> v0.28.4

k8s.io/apiserver
replace
minor
v0.24.2 -> v0.28.4

k8s.io/cli-runtime
replace
minor
v0.24.2 -> v0.28.4

k8s.io/client-go
replace
minor
v0.24.2 -> v0.28.4

k8s.io/cloud-provider
replace
minor
v0.24.2 -> v0.28.4

k8s.io/cluster-bootstrap
replace
minor
v0.24.2 -> v0.28.4

k8s.io/code-generator
replace
minor
v0.24.2 -> v0.28.4

k8s.io/component-base
replace
minor
v0.24.2 -> v0.28.4

k8s.io/component-helpers
replace
minor
v0.27.3 -> v0.28.4

k8s.io/controller-manager
replace
minor
v0.24.2 -> v0.28.4

k8s.io/cri-api
replace
minor
v0.24.2 -> v0.28.4

k8s.io/csi-translation-lib
replace
minor
v0.24.2 -> v0.28.4

k8s.io/kube-aggregator
replace
minor
v0.24.2 -> v0.28.4

k8s.io/kube-controller-manager
replace
minor
v0.24.2 -> v0.28.4

k8s.io/kube-proxy
replace
minor
v0.24.2 -> v0.28.4

k8s.io/kube-scheduler
replace
minor
v0.24.2 -> v0.28.4

k8s.io/kubectl
replace
minor
v0.24.2 -> v0.28.4

k8s.io/kubelet
replace
minor
v0.24.2 -> v0.28.4

k8s.io/legacy-cloud-providers
replace
minor
v0.24.2 -> v0.28.4

k8s.io/metrics
replace
minor
v0.24.2 -> v0.28.4

k8s.io/mount-utils
replace
minor
v0.24.2 -> v0.28.4

k8s.io/pod-security-admission
replace
minor
v0.24.2 -> v0.28.4

k8s.io/sample-apiserver
replace
minor
v0.24.2 -> v0.28.4

k8s.io/utils
require
digest
cf03d44 -> b307cd5

sigs.k8s.io/kustomize/api
replace
minor
v0.12.1 -> v0.16.0

sigs.k8s.io/kustomize/kyaml
replace
minor
v0.13.9 -> v0.16.0

Release Notes

DataDog/datadog-go
v5.4.0
Compare Source
See the Changelog for the details.

argoproj/gitops-engine
v0.7.3
Compare Source
v0.7.2
Compare Source

coreos/go-oidc
v3.9.0
Compare Source
What's Changed

build(deps): bump github.com/go-jose/go-jose/v3 from 3.0.0 to 3.0.1 by @dependabot in https://github.com/coreos/go-oidc/pull/399

New Contributors

@dependabot made their first contribution in https://github.com/coreos/go-oidc/pull/399

Full Changelog: https://github.com/coreos/go-oidc/compare/v3.8.0...v3.9.0

golang-jwt/jwt
v5.2.0
Compare Source
What's Changed

Exported NewValidator by @oxisto in https://github.com/golang-jwt/jwt/pull/349
Improve ErrInvalidKeyType error messages by @Laurin-Notemann in https://github.com/golang-jwt/jwt/pull/361
Update MIGRATION_GUIDE.md by @jbarham in https://github.com/golang-jwt/jwt/pull/363

New Contributors

@Laurin-Notemann made their first contribution in https://github.com/golang-jwt/jwt/pull/361
@jbarham made their first contribution in https://github.com/golang-jwt/jwt/pull/363

Full Changelog: https://github.com/golang-jwt/jwt/compare/v5.1.0...v5.2.0
v5.1.0
Compare Source
What's Changed

Using jwt's native ErrInvalidType instead of json.UnsupportedTypeError by @oxisto in https://github.com/golang-jwt/jwt/pull/316
Fix typos in comments and test names by @alexandear in https://github.com/golang-jwt/jwt/pull/317
Format: add whitespaces, remove empty lines by @alexandear in https://github.com/golang-jwt/jwt/pull/319
Refactor example: use io.ReadAll instead of io.Copy by @alexandear in https://github.com/golang-jwt/jwt/pull/320
Refactor code by using switch instead of if-else by @alexandear in https://github.com/golang-jwt/jwt/pull/318
A quick way to validate token string by @dcalsky in https://github.com/golang-jwt/jwt/pull/302
Refactor: remove unnecessary []byte conversion to string by @alexandear in https://github.com/golang-jwt/jwt/pull/330
Refactor: compare strings with strings.EqualFold by @alexandear in https://github.com/golang-jwt/jwt/pull/329
Avoid use of json.NewDecoder by @craigpastro in https://github.com/golang-jwt/jwt/pull/313
Update ParseUnverified godoc by @duhaesbaert in https://github.com/golang-jwt/jwt/pull/341
Update ci workflows (add go1.21) by @mfridman in https://github.com/golang-jwt/jwt/pull/345
Bump actions/checkout from 3 to 4 by @dependabot in https://github.com/golang-jwt/jwt/pull/346
Key rotation with VerificationKeySet by @mfridman in https://github.com/golang-jwt/jwt/pull/344
Add explicit ClaimsValidator implementation check for custom claims by @epelc in https://github.com/golang-jwt/jwt/pull/343
feat: allow making exp claim required by @tareksha in https://github.com/golang-jwt/jwt/pull/351
Add error handling to examples by @craigpastro in https://github.com/golang-jwt/jwt/pull/312

New Contributors

@alexandear made their first contribution in https://github.com/golang-jwt/jwt/pull/317
@dcalsky made their first contribution in https://github.com/golang-jwt/jwt/pull/302
@craigpastro made their first contribution in https://github.com/golang-jwt/jwt/pull/313
@duhaesbaert made their first contribution in https://github.com/golang-jwt/jwt/pull/341
@epelc made their first contribution in https://github.com/golang-jwt/jwt/pull/343
@tareksha made their first contribution in https://github.com/golang-jwt/jwt/pull/351

Full Changelog: https://github.com/golang-jwt/jwt/compare/v5.0.0...v5.1.0

googleapis/google-api-go-client
v0.153.0
Compare Source
Features

all: Auto-regenerate discovery clients (#2276) (510f0f9)
all: Auto-regenerate discovery clients (#2279) (d07fe32)
all: Auto-regenerate discovery clients (#2280) (e8e2895)
all: Auto-regenerate discovery clients (#2283) (bdc87de)
all: Auto-regenerate discovery clients (#2286) (7897739)
all: Auto-regenerate discovery clients (#2287) (4148872)
all: Auto-regenerate discovery clients (#2290) (9f55397)
all: Auto-regenerate discovery clients (#2291) (0a6ebe4)
Expose experimental universe-related options (#2264) (e648a9d)
Update workloadmanager api manually (#2282) (4aaad57)

grpc/grpc-go
v1.60.0: Release 1.60.0
Compare Source
Security

credentials/tls: if not set, set TLS MinVersion to 1.2 and CipherSuites according to supported suites not forbidden by RFC7540.

This is a behavior change to bring us into better alignment with RFC 7540.

 Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

⚠ Artifact update problem
Renovate failed to update an artifact related to this branch. You probably do not want to merge this PR as-is.
♻ Renovate will retry this branch, including artifacts, only when one of the following happens:

any of the package files in this branch needs updating, or
the branch becomes conflicted, or
you click the rebase/retry checkbox if found above, or
you rename this PR's title to start with "rebase!"


Add warning that you can't reuse a Steam account name

You should probably indicate the other changes you have made in your PR title.

That's the only change.

Well, I see this commit you added to this PR: Commit, which looks like you added some websites.

They are already merged commits

My bad.