use enum to implement log_level in wholememory

In the absence of a generic logging tool, this all looks good

/merge


Fix dropdown destroy method

This fixes issue #9339. When destroying a dropdown menu, the dropdown classes wouldn’t be removed if using a different class structure (e.g.:, using custom class names instead of .menu). Previous versions selected all child elements of the menu, but on the latest version only direct li child elements  or  elements with .menu class are targeted.
This commit ensures ul elements also get the classes removed, for those cases where custom classes are used. Not sure if this is the best way, but seems to cover most cases.

Thanks for submitting a PR!  You're right that it doesn't necessarily seem like the best way... the reason this was changed to be less broad-reaching was because folks were running into problems with nesting stuff inside menus.
That said, looking at the rest of the Nest code there is already an assumption that uls are going to be submenus as a part of this menu.
Hmm... the one thing I would raise then is that this will only work in the single-nested case, but deeply nested menus will still break.  Given we have the ul assumption in Feather, maybe instead of matching against .menu we can match against ul and catch all of our cases?
Would be interested in thoughts from other @zurb/yetinauts on this as well.

@kball So > li, > li > ul is probably the safest way, no?  Even if you add a second level of nesting, this would only target one specific dropdown (.menu on the other hand would destroy all).

That looks really ugly. Shouldn't the selector for finding do a lot less assumptions about the structure of the menu?

I agree with @DaSchTour, we shouldn't target > li. But unfortunately, this is the current structure of the menu for the CSS.
So or we remain consistant and we can make this ugly assumption in the Js. Or we switch to something like .menu-item.

@ncoden I'd personally love to see the menu system redone, as its a bit of a nightmare to overwrite styles at the moment. That and get rid of table-cell, but thats another issue entirely 😉

@brettsmason I remember that on my previous project, I preferred reimplement a menu by myself rather than using the Foundation menu and overwriting everything. :laughing:

@ncoden relying on class names seems more problematic.
I stumbled upon the problem because I'm building my own menu structure and class names using Foundation's mixins. So targeting a basic menu structure (lists) or just going back the previous way (selecting *) seems less prone to errors.

@diniscorreia I didn't mean that .menu-item would be used in the JS, but only to apply CSS styles (the styles we want, where we want it, when we want it). That's a bad practice to target classes or tags (<li>) in the Js. We should use [data-*] attributes instead.
I thought about it, but I really don't know. I don't think I seen data attributes used for sub-elements before: (poke @kball)
<ul data-menu class="menu">
    <li data-menu-item class="menu-item">
    ...
</ul>

Note: If we care about perf issues, we can use .js-* classes the same way

@ncoden from what I have heard recently the performance gap between class and attribute selector isn't that big and that there are a lot of operations that decrease performance a lot more.
I also don't like the idea of adding .js-* classes while all other components use data attributes.

@DaSchTour I was talking generally. Anyway, the performance gap between classes and data attributes is for the browsers that don't support querySelector, so IE <= 8.
My main concern is about using data attributes for the sub-elements of a component. 🤔

This gets to a really tricky line around how verbose we want to force the markup to be vs not, and it is something we're going to need to figure out moving forward...
We have a couple of examples where we use data attributes on sub items before (magellan targets, equalizer watches)...  We could start standardizing that.
Another thing that we could consider (not sure if this is a good idea or not) is making it configurable via a javascript flag... e.g. for folks using the "default" approach we auto-apply the data attributes for children... this lets us have the 'base case' be still minimal markup.  But if you want to do more complicated things or get more control, you set a 'data-manual-children' attribute on the parent and then you mark it up... making it easier to explicitly control component nesting etc.

@kball 👍  totally agree

@kball @DaSchTour @brettsmason So, what do we do for this PR ?


Correct way to implement background service

Hi,
I have to pull data from an external source on a regular basis.
Right now I included this service on site startup
registrar.RegisterInstance(new OrderManagement.ShopwareOrderSynchronizer());
Well, I start a thread and insert new Orders. What is the best way? Use the connection.ForTable directly, use the Repository with UnitOfWork? Or with SqlInsert(...)?
I'm a bit lost here, for example I don't know who to validate the user (I get "Access denied" when using the Repository).
Is there an example I can follow? I also have to include a settings page and so on...
Thank you!

In a separate thread, as there is no http request, user is null.
You need to register an ImpersonatingAuthenticationService encapsulating your own AuthenticationService, and call its Impersonate and UndoImpersonate methods to fake a user that has rights.
Serenity has a background task system. There is not a sample yet, but on todo list.

Ok, thank you, I'm going to try it. Regarding the background system - is it used somewhere in serene?

No, not yet

Thank you very much! Really appreciate your work

If others have the same requirement... There's already an ImpersonatingAuthorizationService in Serenity. I just did the following:
In SiteInitialization:
var registrar = Dependency.Resolve();
registrar.RegisterInstance(new ImpersonatingAuthorizationService(new Administration.AuthorizationService()));
And in my background service:
(Dependency.Resolve() as ImpersonatingAuthorizationService).Impersonate("admin");
....
(Dependency.Resolve() as ImpersonatingAuthorizationService).UndoImpersonate();

@MrPec I am still getting validation / permission issues after using above code
can you please suggest what else you might have done to make it work?


Tailwind media queries no longer working on create-nuxt-app v3.7.1

The issue is that Tailwind media queries used in PostCSS blocks in components are not compiling correctly.
To replicate, I have generated a new nuxt app with the following settings:

To test a Tailwind media query, I have added one to the NuxtLogo component like so:
<style lang="postcss">
.nuxt-logo {
  height: 180px;

  @screen md {
    height: 280px;
  }
}
</style>

Expected CSS generated:
.nuxt-logo {
  height: 180px;
}
@media (min-width: 768px)
  .nuxt-logo {
    height: 280px;
  }
}

Resulting CSS generated:
.nuxt-logo {
    height: 180px;
    @media (min-width: 768px) {;
    height: 280px;
    }: ;
}

Chrome dev tools screenshot:

If anyone has a quick fix or workaround for me while a fix is rolled out in create-nuxt-app I would be eternally grateful too...

Have you tried using this format instead?
@media screen(md) {   .nuxt-logo {     height: 280px;   } }

I can confirm that the above format still works, yes. Thanks


fixed small errors

Looks like the typo fixes in this PR was corrected on the following commits before checking this PR, I'll close it as it's no longer relevant.


Is the body tracking model closed-source?

Hello -- I was unable to locate the ML code and training data that the Azure Kinect uses for body tracking. I am wondering if it is closed-source or not, and if so, whether or not there are plans to release it?
I am interested in full-hand pose and facial expression pose estimation for the application of 3D modeling, and would like to try using the body tracking implementation as a reference if possible.

Yes, body tracking is closed source.

@qm13 I was testing out the body tracker and realised that in most of the samples, the colour camera is turnt off. Is it safe to assume that the body tracker only utilises the depth camera for detection & tracking?

Yes, body tracking only uses the depth and IR images. RGB camera is not used.


Idea: functional API for Pipelines

I came across this yesterday: https://github.com/alegonz/baikal
It's essentially the functional API from Keras applied to scikit learn pipelines. i.e. you supply an actual pipeline step to your next step (instead of duplicating output-of-last is input-of-next etc.). For konduit that would mean to establish

an Input abstraction
Output
the actual chaining of steps

that could be tested with Python first in an experimental version, but could work just as well in Java

In theory this should be completely doable when you think about it:
Input abstraction InputType
Output abstraction OutputType
Step becomes PipelineSteps
Then instead of a Model you create a Service which would contain the remaining info we need.  With the Service you could

start() it in place
save('config.json') to save the resulting config.json (magic needed here)
create_server('folder') to create a konduit-serving bundle with the config.json and all python files that you can just bin/konduit-server start
deploy('http://server:1234') to upload and deploy the pipeline on that server (More magic needed here)

I'm imagining it could look like:
import numpy as np
from keras.models import load_model
from konduit.functional import Input, Output, Service, Step, Top5ClassesStep

def my_op(x, y):
    return x * y / 2

my_model = load_model('foo.h5')

x = Input(dtype=np.array)
y = Input(dtype=float)

op = Step(my_op, output=Output(dtype=np.array)
net = Step(my_model, output=Output(dtype=np.array)

pipeline = op(x, y)
pipeline = net(res)
pipeline = Top5ClassesStep(nclasses=10)(res)

service = Service(pipeline)

service.save("backup_config.json")
service.deploy("http://localhost:1234")

Should we not reuse the names Input, Output and Step though?  Since many other things use those names I wonder....

yeah, along those lines. following the keras spec, the equivalent design would be to make a Pipeline inherit from PipelineStep. Then in Python you can implement __call__ methods on each implementation of PipelineStep that return a PipelineStep (and take one as input, of course). This allows you to do do roughly this:
from konduit import Input, Output, Pipeline, ModelStep, PythonStep

x = Input(input_name='x', ...)

ps = PythonStep(python_code='x * 2')(x)
ms = ModelStep(type='KERAS', model_file='model.h5)
out = Output(type='RAW')(ms)

server = Server(input=x, output=y, serving_config=...)

You can almost do the same thing in Scala with apply, in Java we'll have to have a specific step method or so to make it work. Actually, I'd like to see this implemented in Java and generate the Python stuff (minus the syntactic sugar).

the key thing here is that a lot of internals can be inferred from the former steps. e.g. in the above example, why would a user need to specifically output names for a step and then make sure the input names of the next step exactly match? nobody should care about the internal naming conventions of a pipeline, as long as you can make sure what comes out of it makes sense to you.

So the ModelStep would take ps from PythonStep as ms = ModelStep(type='KERAS', model_file='model.h5)(ps)?

Also, maybe we can already have information from the out object and just do server = Server(pipeline=out, serving_config=...)


FIlebrowser creating invalid share links

links currently say "ext-onion://" instead of "http://". I'm guessing .local needs to be updated as well.

This is not happening anymore for me. @ProofOfKeags why was this issue reopened?

Given that I closed this and immediately reopened, my guess is that it was a mistake. Close at your discretion.


Testing for pywrencli.py

We have no tests for pywrencli.py.

@ericmjonas Is this fixed with the new interactive script or do we want to keep this open to add more tests ?

I mean I'd always love for there to be more tests but I think the new interactive script is going to be the preferred way going forward.


fix(eventable): don't execute events in wrong order on every 2nd call

Added (at first) failing regression test.


503 - Service Unavailable

Hi,
Can you please help me resolve 503 Service Unavailable error when I hosted it a domain:
http://xyz.net/apps/ArrestDB//
None of these Sqlite Path didn't work in index.php:
$dsn = 'sqlite://./test.db'
$dsn = 'sqlite://./public_html/apps/ArrestDB/test.db.db'
$dsn = 'sqlite:///home/user/public_html/apps/ArrestDB/test.db.db'
Empty client:
$clients = [];
All other files in that folder has full(755,777) permissions.

Same problem here

what PHP version are you using?

Using 5.5+ version.

Me too

try with $dsn = 'sqlite://test.db'
when the test.db file and the index.php file are in the same directory
easy test

most probable:
https://stackoverflow.com/questions/8803728/pdo-sqlite-could-not-find-driver-php-file-not-processing
look what PHP version your server is running

The test.db file is in same folder along with index.php, but with your option(sqlite://test.db), the complete service/app is down or broken, the webpage shows blank or disconnected page.
I tried again with "sqlite://./test.db", now got the page, but the service is broken with below error message:
{
"error": {
"code": 503,
"status": "Service Unavailable"
}
}
The exact version of PHP is 5.6.30.

Finally, got resolved.
For SQLite, I had to force write the mod_rewrite rules in the .htaccess file to get it fixed.  For MySQL, it worked without mod_rewrite.
For both issues, I had to access the URLs in the format "/index.php//.." to get them working.


Remove redundant u strings

Description
The goal of this PR is to serve as a followup to #2023 and remove a bunch of lingering u""
GitHub Issues
#1994
#2000
#2022

Coverage decreased (-0.03%) to 72.832% when pulling f60618e42a974d2c4e18f6f1daa7f9421274b93a on mbeacom:python3-1 into 60f6b86dae6e7df7ce82150aef77958017c76af7 on Miserlou:master.


⚠ DBA1 has degraded performance

In 21830cb, DBA1 (https://dba1.jenshamann.solutions) experienced degraded performance:

HTTP code: 200
Response time: 4175 ms

Resolved: DBA1 performance has improved in 4b57b97 after 7 minutes.


[Feature Request] Gamepad Support

It'd be nice to have gamepad support in DOME.

This will be coming in #25 once the API design settles.

This has landed in #25.


Version 0.2.0

[x] option parse by optparse
[x] add subcommand
[x] robocop on CI
[x] fix some by rubocop
[x] actionlint
[x] Docker Image
[x] Container Image Build on GitHub Actions
[x] Container Image Vulnerability Scan on GitHub Actions

ク　ソ　デ　カ　マ　ー　ジ


"Too many open files" using per-test-case In-Memory Realms

Goals
I'd like to stand up a new Realm for each test case as described in the documentation.
Expected Results
Each test case creates a new in-memory Realm instance to be used in isolation for only that test case.
Actual Results
After ~500 test cases, subsequent tests fail with RLMException: open() failed: too many open files.
Steps to Reproduce

Implement a subclass of XCTestCase that stands up an in-memory Realm for each test case using the name of the test as the inMemoryIdentifier.
Write more than 500 test case methods.
Run test suite

Code Sample
This is our test case setup method:
- (void)setUp
{
    [super setUp];

    RLMRealmConfiguration *config = [RLMRealmConfiguration new];
    config.inMemoryIdentifier = [self name];
    [RLMRealmConfiguration setDefaultConfiguration:config];

    _realm = [RLMRealm realmWithConfiguration:config error:NULL];
}

(That Realm instance is then injected generally throughout the codebase)
I know the documentation on in-memory Realms says:

In-memory Realms create several files in a temporary directory for coordinating things like cross-process notifications. No data is actually written to the files unless the operating system needs to swap to disk due to memory pressure.

So I assume those are the files that are building up. Is there some kind of tear-down I need to perform to clean up any files associated with the in-memory Realms?
Version of Realm and Tooling
ProductName:    Mac OS X
ProductVersion: 10.10.5
BuildVersion:   14F1605

/Applications/Xcode.app/Contents/Developer
Xcode 7.2.1
Build version 7C1002

/Users/alexleffelman/.rbenv/shims/pod
0.39.0.beta.5
Realm (0.98.3)

/bin/bash
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin14)

/usr/local/bin/carthage
0.6.3
(not in use here)

/usr/bin/git
git version 2.5.4 (Apple Git-61)

I've set up a test project that reproduces this issue with 1600 test cases that are all just XCTAssertTrue(YES), but with the same general setUp logic.
https://github.com/LeffelMania/Realm-Unit-Test-Repro

In Realm's unit tests we call the private +[RLMRealm resetRealmState] method after each test to clean up any cached realm instances. If you add a call to that method in -tearDown does the problem you're seeing go away?

That method doesn't appear to exposed in the public interface of RLMRealm, but when I do add it in an extension for testing, the tests in my sample project do indeed complete. :+1:

Thanks for confirming. We should consider exposing +resetRealmState, or something like it, for sake of people writing unit tests that use Realm.

Any updates on this one? We are experiencing the exact same issue and not sure how to tackle this at the moment. We have over 500 tests and most of them are using Realm with the in-memory configuration. The tests are flaky and we can't rely on them at the moment. Its a biggie for us.

Here are more issues that are related to this:

https://github.com/realm/realm-cocoa/issues/6033
https://github.com/realm/realm-cocoa/issues/4435

Same problem here after adding some more test cases and apparently passing the threshold now. Is my assumption correct that there is no Swift way to achieve a workaround similar to the performSelector with resetRealmState approach in Objective-C that bdash mentioned? Any help would be greatly appreciated.

Clearing the tmp folder that is getting flooded with DB files periodically (e.g. in a tearDown call) has no effect since the error seems to originate from an internal threshold or cache limit I guess?

Hi, do have any updates on this one?
Keen to create a fresh inMemory realm while unit testing.

I've also faced that issue with over 500 unit tests. Are there any solution here?

You can increase the rlimit on your system.

@nirinchev thanks for answering
I tried it before, and it's unlimited as I can see. Maybe I do something wrong?

The file descriptor limit is the determining factor here. I'm not sure how many realm files you open during tests, but every open realm will consume several file descriptors, so the 2.5k number there doesn't correspond 1:1 to the number of open realms you can have.

Alright, just increasing ulimit for file descriptors didn't work for me, at least if I run it on terminal and then try to launch unit tests in xCode
What helps on my local machine is sudo launchctl limit maxfiles 65536 200000
I found it here: https://wilsonmar.github.io/maximum-limits/


The status of pull requests

There are about 15 open pull requests waiting a response right now –– some of them are very essential to make Jest useful for us, like making jest aware of NODE_PATH, and some have been waiting for months for an official response. Are there any plans to formally review and maybe integrate these requests into the master branch? Please?

Hi, yes this is on me. I don't have a great excuse other than that I've taken on quite a bit more work than I've been able to keep up with. You've prompted me to actively start reaching out to a few people and see if I can get some more hands on deck here to help keep Jest moving forward...so hopefully we can make that happen very soon.
In the meantime: I'm on a plane all day tomorrow -- so barring plane wifi issues, I will try to go through all of the pending PRs and hopefully we can get them sorted out as soon as possible.

Thanks. This is for you. :beer:


Doesn't Function

I launch the program. It does see my client location and PBE location and the boxes next to them are checked. However, once I launch a replay nothing happens. I have tried starting it both before and after starting a replay. I have also tried the same with running as admin. Any thoughts?

same for me

We can probably fix whatever is going wrong but I need some help from you guys to help debug it.
Startup a replay on PBE making sure that league director has the checkbox ticked next to where you PBE is installed. Then open the following URL in your browser:
https://<IP_ADDRESS>:2999/replay/game
You will need to skip past the security warnings, but then it should return something like this:
{     "processID": 9596 }
Let me know if that works for you. You can also come and chat to me in the league of editing discord and we can try a few things out.
https://discord.gg/aXRWmQ

I can try this out this afternoon when I am home from work. That discord invite doesn't work for me as well.

Sorry try this one: https://discord.gg/7j5fdRp

That one worked great thank you. Will be in touch a bit later!

I have a problem running leaguedirector either by using source code or by using the installer of v0.1.
Leaguedirector opens and does not connect to the game

Output when I run the run.bat
Requirement already satisfied: pipenv in c:\python37\lib\site-packages (2018.11.26)
Requirement already satisfied: virtualenv-clone>=0.2.5 in c:\python37\lib\site-packages (from pipenv) (0.5.1)
Requirement already satisfied: setuptools>=36.2.1 in c:\python37\lib\site-packages (from pipenv) (40.6.2)
Requirement already satisfied: pip>=9.0.1 in c:\python37\lib\site-packages (from pipenv) (19.0.2)
Requirement already satisfied: certifi in c:\python37\lib\site-packages (from pipenv) (2018.11.29)
Requirement already satisfied: virtualenv in c:\python37\lib\site-packages (from pipenv) (16.4.0)
Installing dependencies from Pipfile.lock (9a6b12)…
  ================================ 3/3 - 00:00:01
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
qt.network.ssl: QSslSocket: cannot call unresolved function d2i_X509
qt.network.ssl: QSslSocket::connectToHostEncrypted: TLS initialization failed
qt.network.ssl: QSslSocket::connectToHostEncrypted: TLS initialization failed
qt.network.ssl: QSslSocket::connectToHostEncrypted: TLS initialization failed
qt.network.ssl: QSslSocket::connectToHostEncrypted: TLS initialization failed
qt.network.ssl: QSslSocket::connectToHostEncrypted: TLS initialization failed
qt.network.ssl: QSslSocket::connectToHostEncrypted: TLS initialization failed
qt.network.ssl: QSslSocket::c/replay/gameonnectToH ostETLS initialization failedncrypted: TLS
initialization failed
qt.network.ssl: QSslSocket::connectToHostEncrypted: TLS initialization failed
qt.network.ssl: QSslSocket::connectToHostEncrypted: TLS initialization failed
...

Same here on normal client.

League Director only works on PBE right now - but 9.4 is going live this week after which it will work everywhere.

Trying it on EUW now but I have the same issue, it's not connecting to my replays. Maybe it only works if the directory is in C drive?

Well it doesnt work for me but should, right?  If you have a specific issue that you can describe and reproduce feel free to open another issue.


[README] Add DebugBundle to the list

Q
A

Bug fix?
[no]

New feature?
[no]

BC breaks?
[no]

Deprecations?
[no]

Tests pass?
[yes]

Fixed tickets

License
MIT

Doc PR

This commits mentions DebugBundle in the readme. I believe this should also be ported to 2.7, 2.8 and master branches.
Have a nice weekend!

I have updated it according to comments. If there is anything else - please let me know.

Thank you @edvinasme.


Added TextAnalyticsError to HealthcareJobState

MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[x] I commit to follow the Breaking Change Policy of “no breaking changes
[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

[ ] Ensure to check this box if one of the following scenarios meet updates in the PR, so that label “WaitForARMFeedback” will be added automatically to involve ARM API Review. Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs, all “removals” and “adding a new property” no more require ARM API review.

Adding new API(s)
Adding a new API version
Adding a new service

[ ] Please ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If there are following updates in the PR, ensure to request an approval from API Review Board as defined in the Breaking Change Policy.

[ ] Removing API(s) in stable version
[ ] Removing properties in stable version
[ ] Removing API version(s) in stable version
[ ] Updating API in stable version with Breaking Change Validation errors
[ ] Updating API(s) in preview over 1 year

Please follow the link to find more details on PR review process.

/azp run automation - sdk

not a change to ARM api - removed checked items from PR checklist and removing label

/azp run automation - sdk

@varunkumta @yangyuan can you review this ?

@varunkumta Varun Kumta FTE @yangyuan Yuan Yang FTE can you review this ?

@suhas92 please look at my comment above - this is not a change to an ARM api and hence does not need Arm api review.

@varunkumta Can you get it merged?

@varunkumta Varun Kumta FTE Can you get it merged?

the PR assignee @anuchandy can help you with that


[BUG] Poor error handling & made up error headers

HTTP response error handling is not correct or helpful.
Expected Behavior

errors are handled according to the API documentation / specification
if there is a response body available it should be printed out / thrown in the error
no made up headers should be present in the map

Current Behavior
Maps contain hallucinated headers x-error-code x-error-message but it do NOT contain error response bodies!
if (response.status !== 200) {
  const error = {
    object: 'error',
    status: response.status,
    code: response.headers['x-error-code']?.[0] ?? '',
    message: response.headers['x-error-message']?.[0] ?? '',
  };
  throw new std.unstable.MapError(error);
}

Possible Solution

at the very least report the response body, if available ❗
do not make up headers that do not exists

Ideal Solution

handle common HTTP error codes
respect the API documentation when it comes to error handling

Steps to Reproduce
Easiest way is to try to execute against any API unauthenticated. Most likely you will end up with the almost useless error:
ERROR: {
  "name": "PerformError",
  "errorResult": {
    "code": "",
    "message": "",
    "object": "error",
    "status": 401
  }
}

Your Environment

@superfaceai/cli/4.0.0-beta.5 darwin-x64 node-v18.13.0 (with<EMAIL_ADDRESS><EMAIL_ADDRESS>macOS
OneSDK Beta0
Node 18.13

@zdne  Can you please check again? This is related to #319 because the engine pulled up incorrect docs for the prompts. Thus it might have hallucinated some of the maps, too.
Also, this partly is AI issue.

I will check it again today.


Made the BIDS example more general for the audience

Made the BIDS example more general for the audience

Pull Request Test Coverage Report for Build 444631179

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 90.059%

Totals

Change from base Build 436863558:
0.0%

Covered Lines:
5979

Relevant Lines:
6639

💛  - Coveralls

I'm not sure if this is beyond the scope of the pull request, but it might be worth explaining a bit about what is in these folders/files and what sub-01 means, for example?

thank you, does https://github.com/ivadomed/ivadomed/pull/597/commits/30f826f3ce607ce65e0ec6e1849b37a133a8c2f2 help?

I'm not sure if this is beyond the scope of the pull request, but it might be worth explaining a bit about what is in these folders/files and what sub-01 means, for example?

thank you, does https://github.com/ivadomed/ivadomed/pull/597/commits/30f826f3ce607ce65e0ec6e1849b37a133a8c2f2 help?

Yes, https://github.com/ivadomed/ivadomed/pull/597/commits/30f826f3ce607ce65e0ec6e1849b37a133a8c2f2 helps clarify what is in the folders.

Yes, https://github.com/ivadomed/ivadomed/pull/597/commits/30f826f3ce607ce65e0ec6e1849b37a133a8c2f2 helps clarify what is in the folders.


Add closure to read_line()'s arguments to enable syntax highlighting

Reviews would be great!

In my opinion it'd be great if you could keep https://github.com/MovingtoMars/liner/pull/83 as well (if not too much work with for example merge conflicts of course), since upstreaming is good in general. Although that might just be me.

@Sag0Sag0 I resolved a conflict in your branch after my PR #2 was merged.

Thanks!


community:  Add Cloudflare AI Gateway option for Cloudflare workers-ai LLM

Description:
Added support for Cloudflare AI Gateway in the Cloudflare workers-ai integration. This enables users to call Cloudflare AI models through their AI Gateway endpoint, providing more flexibility for analytics, caching, and managing AI requests.
Before:
self.endpoint_url = f"{self.base_url}/{self.account_id}/ai/run/{self.model}"

After:
...

ai_gateway: str = ""
gateway_url: str = "https://gateway.ai.cloudflare.com/v1"

...

if self.ai_gateway:
    self.endpoint_url = f"{self.gateway_url}/{self.account_id}/{self.ai_gateway}/workers-ai/run/{self.model}"
else:
    self.endpoint_url = f"{self.base_url}/{self.account_id}/ai/run/{self.model}"

Dependencies: N/A

Twitter handle: @YimingShen5

closing as stale


Peers not reaching each other

Hi,
My peers cannot connect to each other, despite default configurations. On the admin page they result connected but they can not ping each other. It is a problem of my selfhosting attempt, the same peers on the official server work perfectly.
docker logs -f tmp-signal-1 shows connection and disconnection correctly and no errors.
I have no idea how to further diagnose this issue, but i am considering another possibility.
Is there any know issue in selfhosting netbird on Oracle Cloud? netmaker for instance warns against hosting there.
Could be some networking quirk of the cloud provider?
Thanks!

Hey @asamarco
Would you mind joining our slack and pinging us there in the general chat?
Please also refer to this issue in your message there. Let's figure out the issue together!
https://join.slack.com/t/netbirdio/shared_invite/zt-vrahf41g-ik1v7fV8du6t0RwxSrJ96A

Any update on this? I´m having some issues with some peers

hey @risturiz
Could you please ping me in Slack? (Misha)

Hi,
My peers cannot connect to each other, despite default configurations. On the admin page they result connected but they can not ping each other. It is a problem of my selfhosting attempt, the same peers on the official server work perfectly.
docker logs -f tmp-signal-1 shows connection and disconnection correctly and no errors.
I have no idea how to further diagnose this issue, but i am considering another possibility. Is there any know issue in selfhosting netbird on Oracle Cloud? netmaker for instance warns against hosting there. Could be some networking quirk of the cloud provider?
Thanks!

Oracle Cloud having this problem.maybe arm cpu or network.

Hi, I tried again netbird just yesterday and I have found the cause of my
problems.  Once this was corrected I could finally connect
all the nodes. I am using an AMD VM though not the ARM ones.
On Mon, Sep 25, 2023, 14:44 htc10 @.***> wrote:

Hi,
My peers cannot connect to each other, despite default configurations. On
the admin page they result connected but they can not ping each other. It
is a problem of my selfhosting attempt, the same peers on the official
server work perfectly.
docker logs -f tmp-signal-1 shows connection and disconnection correctly
and no errors.
I have no idea how to further diagnose this issue, but i am considering
another possibility. Is there any know issue in selfhosting netbird on
Oracle Cloud? netmaker for instance warns against hosting there. Could be
some networking quirk of the cloud provider?
Thanks!
Oracle Cloud having this problem.maybe arm cpu or network.
—
Reply to this email directly, view it on GitHub
https://github.com/netbirdio/netbird/issues/1094#issuecomment-1733639146,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABIHMBECWCH4OZUFYL4KE3TX4F4DDANCNFSM6AAAAAA3V34X3Q
.
You are receiving this because you were mentioned.Message ID:
@.***>


Sync call

Hi,
Is there a way to call synchronously the find method such as?
var found =  Model.find().exec();
I have something like this in my bootstrap.js (sails.js config).
global.sync = require("synchronize");
_.each(sails.models,function(model){
_.map(model,function(attrValue,attrKey){
if(typeof attrValue=="function"){
model[attrKey+"Sync"]=sync(attrValue);
}
});
});
But when I call var found =  Model.findSync() the property values is always empty. I guess it's because .exec is not called. But if I call with .exec() it fires an error. exec must have a callback.
Is there a way to do it easily?
Do you support it by default?
Thanks,

@ameilland, this is not supported and never will. Node.js is single threaded and doing something like that would make Node.js block while waiting for the DB (ages in CPU time). I suggest you use callbacks or promises. You can read more about this in http://stackoverflow.com/questions/9884418/how-can-i-make-this-call-to-request-in-nodejs-synchronous. If you really insist in making an async method sync'ed (not recommended) you can use https://github.com/abbr/deasync.

Ok it makes sense. But sometimes people can use in a loop a query and wait for the results before going to the next action. And callback can complicate a lot the code. But deasync makes the job perfectly. Thanks,

But sometimes people can use in a loop a query and wait for the results before going to the next action. And callback can complicate a lot the code.

@ameilland if you're using node, you should get use to programming asynchronously. node.js is asynchronous by design. This is what "event-driven, non-blocking I/O means". From nodejs.org:

Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.


docs: add design docs for including additional objects in bundles

Description of the change:
Docs for including new objects in the bundle.

PodDisruptionBudget (policy/v1beta1)
PriorityClass (scheduling.k8s.io/v1alpha,v1beta1,v1)
VerticalPodAutoScaler

Docs were written in a generic way to target both upstream and downstream OpenShift audiences. The underlying assumption is that we will allow users to include these objects in the bundle with no limitations on the actual content of these objects.
Motivation for the change:
Reviewer Checklist

[ ] Implementation matches the proposed design, or proposal is updated to match implementation
[ ] Sufficient unit test coverage
[ ] Sufficient end-to-end test coverage
[ ] Docs updated or added to /docs
[ ] Commit messages sensible and descriptive

As this is doc-only we should be OK to merge without CI pending approval.

/approve

/lgtm

manually merging as this is doc-only.


Build of Hydrafw fails

Hi,
I am trying to flash the newest Firmware 0.10 on to the Hydra Bus. I failed to accomplish my goal via the guide (https://github.com/hydrabus/hydrafw/wiki/Getting-Started-with-HydraBus-flash-and-use-hydrafw-on-windows). As an alternative I tried to flash via the Cube IDE. I followed https://github.com/hydrabus/hydrafw/wiki/Getting-Started-with-HydraBus-and-STM32CubeIDE for this option. This options fails for me too.
When Building the Project (Step 6) the Build fails.
The Error I received from the Build Console is:
10:52:09 **** Incremental Build of configuration Debug for project hydrafw ****
make -j4 all
make: Makefile:15: pipe: No such file or directory
Compiling crt0_v7m.S
Creating ./common/hydrafw_version.hdr
Compiling vectors.S
Compiling chcoreasm_v7m.S
make (e=3): Das System kann den angegebenen Pfad nicht finden.
make: *** [Makefile:323: common/hydrafw_version.hdr] Error 3
make: *** Waiting for unfinished jobs....
"make -j4 all" terminated with exit code 2. Build might be incomplete.
10:52:10 Build Failed. 2 errors, 1 warnings. (took 708ms)
In the c File common a File is called with #include "hydrafw_version.hdr but there is no such file in the repositories.
I am probably the problem, but I would appreciate any help.

Hi !
Looks like the hydrrafw_version.hdr cannot be generated correctly.
From the src/ folder, are you able to run the following command ?
python build-scripts/hydrafw-revision.py

This should return the current git version number (0.10 in this case).
If that's not the case, please follow the steps indicated here : https://github.com/hydrabus/hydrafw/wiki/how-to-build-flash-and-use-hydrafw-on-windows#prerequisites-for-windows-install-git-python-and-clone-hydrafw
Let me know if you have any other issues.

Closing since it's been more than a week with no update

It seems the error you have is due because you have not followed the steps and you have not cloned the repository and probably just extracted the zip archive and so the python build-scripts/hydrafw-revision.py cannot work as it does not find git information...
Please confirm what steps you have done.

Yesterday I was able to flash the device successfully after reinstalling all tools. It seems that I failed at structuring the Data in the right way. So my guess is that your assumption is correct.

I would like to apologize for the long latency period and thank you for the help.

No worries ;) Glad you were able to flash the device.


Bug: Days since last test shows zero days but should be more when test only test is backdated

Steps:

Create a new user.
Observe: in the People Tab, is "Days since last test" is "N/A"
In the Conduct Tests tab, start a new test for this new user.
Select a different Test Date, (e.g. 7 days in the past).
Submit test
Go back to the People Tab
Result: "Days since last test" is 0

Expected: "Days since last test" should be based on the backdated date and be 7 days.

Just checked this and it's fixed so closing.


Delay parsing of putative ASTs

Overview
In light of the desire to upload putative tool definitions without errors being thrown in the case of incorrectly structured JSON, this PR removes any attempts to parse the a tool definition's JSON AST into a Map Algebra AST from the API. The changes in this PR, therefore, largely consist in reversion of features from PR #1192
Thinking through when, precisely, JSON should be parsed can likely be avoided until more work is done to flesh out how we will apply a Map Algebra AST as a GeoTrellis operation.

@moradology, thanks for your PR! By analyzing the history of the files in this pull request, we identified @kshepard, @alkamin, @notthatbreezy, @jisantuc and @Lknechtli to be potential reviewers.


Add useDataGrid Hook Documentation and Example

Test me! 'MUI'
Link to USE-DATA-GRID-DOC
Hey, I'm client! Test me! 'MUI'
Link to USE-DATA-GRID-DOC
Test me! 'MUI'
Link to USE-DATA-GRID-DOC
Hey, I'm client! Test me! 'MUI'
Link to USE-DATA-GRID-DOC
Test me! 'MUI'
Link to USE-DATA-GRID-DOC
Add useDataGrid hook documentation for @pankod/refine-mui package.
Test documentation! 'MUI'
Link to USE-DATA-GRID-DOC

Closing this PR and reopening in #1877 for a clean commit tree


Updated "Health Report" tab

Added survey menu and questionnaire for each category

@vchu22 I approved, up to you if you want to resolve my feedback within this PR or in a follow-up PR.


Add RBXScriptSignal:Once (again)

#118
#123

Still fails 🤔

try rebasing and rerunning the tests

No success


Allow crafting schematics

It'd be nice if you could allow creating schematics using tool parts and paper, instead of having to find them all as loot drops.
This way, you could replace vanilla tools completely like in Tinker's Construct.
I can kind of do this using CraftTweaker/KubeJS, but it's quite a hassle, and doesn't show up properly in EMI.
And having an option to disable vanilla/modded tool recipes in favor of Forgero tools (or maybe a better solution would be to set vanilla/modded tools' durabilities to 1, like Tinker's Construct could do) would be nice.
I'd really love to use this in a full survival playthrough without normal tools with some friends, and get a Tinker's Construct 1.7.10 like experience.
I'm up to help if wanted!

Hiya!
Crafting schematics from parts is a supported feature, but there might have been some issues with the recipes in the currently released version. The issue has been fixed, and every schematic should be craftable with paper + tool, when 0.10.5 releases sometime soon. I am considering changing the schematic system, to only make basic schematics craftable with normal paper, while introducing another mechanic for crafting higher tiered schematics. I am still not sure what the best approach is, so I'll keep it like this until I have a concrete plan.
I have considered creating a settings toggle that can disable/hide normal tools, but haven't prioritized it yet. It should not be too hard, so I might include this with 0.10.5 when it releases. I have been pretty cautious when it comes to making changes to tools from vanilla/other mods, in case it causes compatibility issues, or breaks normal features, but if its an opt-in feature I think it would be a nice addition. If you would like to help me test this feature, that would be awesome, and I could experiment with different implementations.
To disable recipes, I can add two entries to the config file, one is a toggle to disable selected recipes, and the other is a list with recipe identifiers that we would like to disable. I'll test if setting the durability to 1 is simple to add, and if so, I can add the feature behind a configuration toggle.

Hey, that's cool, thank you!
Making the changes to normal tools (be it vanilla or modded), should definitely be opt-in, I agree.
For the higher tiered schematics, maybe have something like Tinker's Construct's smeltery + metal casts?
Though it might be easier integrate with Alloy Forgery or Smithee for the smeltery, if you plan to go that route.

Integrating some form of smeltery is definitely an option, but its still far off, so I won't lock in any decision yet. Its probably a matter of figuring how much time I have for development of the features, and what makes the most sense gameplay wise. I am still looking for great ideas, so if you want to take part in the discussion, be sure to join the Discord.
I'll try to see if I can get something done tonight regarding the disabling of vanilla tools. The issue has been up for a long time, so it would be great to get it resolved. See #155

Alright, thanks! Will take a look at the Discord server then. Good luck with it!

Updated the link now

Ay nice!


Fix for Issue #107: User Deletion

Bug Fix
The bug lies in the following:
execute "delete user: #{user}" do

user does not exist, so it was replaced with new_resource.username.
Other Changes
Created a user deletion recipe and test suite
Recipe creates a user in an execute block with sysadminctl to avoid using macos_user (in case there are future issues in it), and also to simulate a previously existing user.

Thanks!

Thanks!


Make PostgreSQL INTERVAL format not case sensitive.

Problem Statement
I think this could be considered to be a bug, please advice.
As per docs we support postgres interval format, but it's case sensitive:
Crate 5.5.2
SELECT INTERVAL '1 day'; -> 1 day 00:00:00
SELECT INTERVAL '1 DAY'; -> SQLParseException[Cannot cast `'1 DAY'` of type `text` to type `interval`]

In postgres:
SELECT INTERVAL '1 day'; -> 0 years 0 mons 1 days 0 hours 0 mins 0.0 secs
SELECT INTERVAL '1 DAY'; -> 0 years 0 mons 1 days 0 hours 0 mins 0.0 secs

Possible Solutions
No response
Considered Alternatives
No response

This is fixed by https://github.com/crate/crate/pull/15585, works on master now:
cr> SELECT INTERVAL '1 DAY';
+-----------------+
| 'P1D'::interval |
+-----------------+
| 1 day 00:00:00  |
+-----------------+
SELECT 1 row in set (0.142 sec)

I see, thanks for letting me know


[v3] Carousel enhancement: Better accessibility for dot buttons

For what version of Nuxt UI are you suggesting this?
v3-alpha
Description
I just did a lighthouse scan of my website using the v3-alpha. It complained about missing button labels for the scroll snap dot buttons.
https://github.com/nuxt/ui/blob/v3/src/runtime/components/Carousel.vue#L301
Just saw that there are aria-labels for the prev and next buttons, but missing for the dot buttons.
If you agree, I can send in a PR to add automatic labels like Slide 1 of 3.
Additional context
No response

@madebyfabian Thanks for the report! PRs are always welcome 😊


Configuration

I feel completely stupid about asking this, but I can't find any documentation on pointing mebla at a different host. What is the recommended way of doing this?

Hi wed :D sorry for not replying sooner, you can change the host in config/mebla.yml

facepalm Thanks :)


[chip/dv] Add keymgr testplan for chip-level

We can go over this in the review meeting
Signed-off-by: Weicai Yang<EMAIL_ADDRESS>

Hi all,
Thanks for joining the review meeting. I have updated the testplan as we discussed. PTAL. Thanks


Clean up processing history table(s)

Create a clinvar_ingest_v1 and clinvar_ingest (v2) dataset in .
Backfill the processing_history table in each with the entries from the mixed clinvar_ingest dataset

v2 vcv and rcv runs we want to keep:
-- CREATE TABLE `clingen-dev.clinvar_ingest_backup_20241118.processing_history_v2` AS
SELECT
  -- *
  release_date,
  file_type,
  pipeline_version,
  xml_release_date,
  processing_started,
  processing_finished
FROM `clingen-dev.clinvar_ingest.processing_history`
WHERE pipeline_version not like '%_gh_%'
AND pipeline_version not like '%_local%'
AND xml_release_date <> '2023-10-07'
AND xml_release_date <> '2024-06-10'
AND not (file_type = 'vcv' and pipeline_version like 'v1%')
AND xml_release_date >= (select min(xml_release_date) from `clingen-dev.clinvar_ingest.processing_history` where pipeline_version like 'v2%')
ORDER BY xml_release_date;

Query creating table with just v1 runs we want to keep.
CREATE TABLE `clingen-dev.clinvar_ingest_backup_20241118.processing_history_v1` AS
SELECT
  *
  -- release_date,
  -- file_type,
  -- pipeline_version,
  -- xml_release_date,
  -- processing_started,
  -- processing_finished
FROM `clingen-dev.clinvar_ingest.processing_history`
WHERE pipeline_version not like '%_gh_%'
AND pipeline_version not like '%_local%'
AND xml_release_date <> '2023-10-07'
AND xml_release_date <> '2024-06-10'
AND not (file_type = 'vcv' and pipeline_version like 'v2%')
ORDER BY xml_release_date;

I think the v2 set we want to seed with is this:
-- CREATE TABLE `clingen-dev.clinvar_ingest_backup_20241118.processing_history_v2` AS
SELECT
  -- *
  release_date,
  file_type,
  pipeline_version,
  xml_release_date,
  processing_started,
  processing_finished
FROM `clingen-dev.clinvar_ingest.processing_history`
WHERE pipeline_version not like '%_gh_%'
AND pipeline_version not like '%_local%'
AND xml_release_date <> '2023-10-07'
AND xml_release_date <> '2024-06-10'
AND not (file_type = 'vcv' and pipeline_version like 'v1%')
AND xml_release_date >= (select min(xml_release_date) from `clingen-dev.clinvar_ingest.processing_history` where pipeline_version like 'v2%')
AND xml_release_date >= '2024-10-27'
ORDER BY xml_release_date;

Next steps are replacing the old clinvar_ingest.processing_history and moving the v1 processing history to a clinvar_ingest_v1 dataset.

[ ] delete clinvar_ingest.processing_history (it is backed up to clingen-dev.clinvar_ingest_backup_20241118.processing_history)
[ ] create clinvar_ingest.processing_history as the contents of the above clinvar_ingest_backup_20241118.processing_history_v2 table.
[ ] create dataset clinvar_ingest_v1
[ ] create clinvar_ingest_v1.processing_history as the contents of the above clinvar_ingest_backup_20241118.processing_history_v1

v2 processing history:

Creating v1 processing_history_pairs view in clinvar_ingest_v1.
CREATE VIEW `clingen-dev.clinvar_ingest_v1.processing_history_pairs` AS
SELECT
        -- Use the release date from the VCV file as the final release date
        vcv.release_date as release_date,
        -- VCV fields
        vcv.file_type as vcv_file_type,
        vcv.pipeline_version as vcv_pipeline_version,
        vcv.processing_started as vcv_processing_started,
        vcv.processing_finished as vcv_processing_finished,
        vcv.release_date as vcv_release_date,
        vcv.xml_release_date as vcv_xml_release_date,
        vcv.bucket_dir as vcv_bucket_dir,
        vcv.parsed_files as vcv_parsed_files,
        vcv.bq_ingest_processing as vcv_bq_ingest_processing,
        -- RCV fields
        rcv.file_type as rcv_file_type,
        rcv.pipeline_version as rcv_pipeline_version,
        rcv.processing_started as rcv_processing_started,
        rcv.processing_finished as rcv_processing_finished,
        rcv.release_date as rcv_release_date,
        rcv.xml_release_date as rcv_xml_release_date,
        rcv.bucket_dir as rcv_bucket_dir,
        rcv.parsed_files as rcv_parsed_files
    FROM
    (SELECT * FROM `clingen-dev.clinvar_ingest_v1.processing_history` WHERE file_type = "vcv") vcv
    INNER JOIN
    (SELECT * FROM `clingen-dev.clinvar_ingest_v1.processing_history` WHERE file_type = "rcv") rcv
    ON (
        vcv.xml_release_date >= DATE_SUB(rcv.xml_release_date, INTERVAL 1 DAY)
        AND
        vcv.xml_release_date <= DATE_ADD(rcv.xml_release_date, INTERVAL 1 DAY)
    )

The VCV v2 processing history table is there but we may want to backfill further and process the files we got while the v2 files were just running through the copy_only workflow. Our processed v2 VCV files only goes back to 2024-10-27 but we have files back to 2024-07-24.

Closing as complete. Further backfilling can be a separate ticket


[WIP] convert results into jUnit format

Tossed this together super-quick, it's incomplete, decidedly not pretty, and it's untested.
WIP: Push early, push often 😀

--fixup the "not working" part - this will successfully collect results

note-to-self: https://github.com/ansible/ansible/blob/stable-2.3/lib/ansible/plugins/callback/junit.py

Ahh, this one seems much better designed and active:  https://github.com/kyrus/python-junit-xml.git

All - annotating code...feedback welcome.

@cevich Should we test this?

Split the collection-aspect off into a separate PR: #945
This one will just focus on producing the canonical jUnit XML for consumption by ci-automation.

...replies to @stevekuznetsov from other PR, moved here:

Neither our Jenkins nor the Gubernator pages need extra information

Possibly, it depends on how the results are presented.  Unfortunately, the producer of the results can't always know where/how they will be consumed.  In most cases, it's very easy to "assume" the source, especially when it's not recorded in the data.
Since I'm parsing results anyway, it's just as easy to stick an extra XML attribute into the elements that proves where they came from.  Pinning the results to a specific hostname for example.
I would consider not having that extra detail as a very important near-term goal, if not a blocker.
Results of questionable or unverifiable origin are as good as a soup-sandwich.  Esp. since this group already had a related run-in, that was very scary.
What is this Gubernator thing, such a sexy name, I simply must know more 😀

Is this separate from the jUnit pull request to BATS

I could easily be wrong about BATS.  By my read/investigation, it appears as an after-through addition, conditional on xunit == True.  A leftover vestige from a bygone Jenkins era that's probably over-complicating what a simple script can do.  Effectively dead-code once this PR goes in.  Though if my understanding is wrong, it would be nice to know early 😀

Possibly, it depends on how the results are presented.

The three main ways engineers use these results are:

in legacy jobs, through the Jenkins UI
using Gubernator
through TestGrid

In all cases, there is a base assumption that the execution environment is not salient to the test failure and that test runs are independent. Therefore, seeing test X fail 10 times in a row indicates something wrong with the code under test (or framework executing it) not an issue with e.g. a Jenkins node.

Pinning the results to a specific hostname for example.

Tests are run on ephemeral VMs in EC2 so I'm not sure what this gains us, but sure, there's nothing wrong with putting more information into the file as long as that is not a blocker for collecting it.

By my read/investigation, it appears as an after-through addition, conditional on xunit == True

I have no knowledge of this, but from the quick look I did, there is a PR/fork of BATS with jUnit support. Seems like that is an easier place to integrate this sort of thing than parsing the output file from a BATS run if the PR is something that will be accepted. If it's dead and the project doesn't want it, your approach here sounds good.

execution environment is not salient to the test failure

Yep, that's exactly the issue that arises when tabulating multiple jUnits.  They silently become dependent on both the execution environment and run-independence (among other factors).  Here's an example.  Consider what happens when these two (somewhat fabricated) e2e jUnits are tabulated into a single "result" (I'm assuming worst-case, one-run, multiple hosts running the e2e tests):
Host A:
<?xml version="1.0" encoding="UTF-8"?>
  <testsuite tests="1" failures="0">
      <testcase name="[k8s.io]..." classname="Kubernetes e2e suite" time="0"></testcase>
</testsuite>

Host B:
<?xml version="1.0" encoding="UTF-8"?>
  <testsuite tests="1" failures="1">
      <testcase name="[k8s.io]..." classname="Kubernetes e2e suite" time="0">
             <failure message="test failure">Assertion failed</failure>
      </testcase>
</testsuite>

The tabulated result either shows the "[k8s.io]..." testcase passed or failed depending on who/what/how the the testing system handled the files.
The fix is super-simple, add something unique (like a hostname or machine-id or whatever) to all the results:  Like this:
Host A:
<?xml version="1.0" encoding="UTF-8"?>
  <testsuite tests="1" failures="0">
      <testcase name="(Host A)[k8s.io]..." classname="Kubernetes e2e suite" time="0"></testcase>
</testsuite>

Host B:
<?xml version="1.0" encoding="UTF-8"?>
  <testsuite tests="1" failures="1">
      <testcase name="(Host B)[k8s.io]..." classname="Kubernetes e2e suite" time="0">
             <failure message="test failure">Assertion failed</failure>
      </testcase>
</testsuite>

Now the tabulated result shows that one host passed but the other failed.  Since the identifying information is tied to the result, debugging is made that much simpler.

as long as that is not a blocker for collecting it.

Oh no, not for collection.  That's why I split up / re-purposed these PRs 😀

I need this to keep my sanity.  Grogging through logs to find out if something failed is not fun.

@cevich strong NACK to changing the NAME of a test ... upstream tools like testgrid absolutely need to be able to consider a test run on host A and host B as the same test and look at the results together. Again, we run with ephemeral VMs. There is never a case where a specific host is broken or causes tests to fail. Te architecture does not allow for it. Upstream tooling to track test failures over time has a strong pre-requisite that test runs are independent. If we cannot deliver that with CI (we can), we should not be running it. I don't see an issue with putting new XML attributes on the tests as long as they fit the schema, but changing things like name fields is a bad idea.

@stevekuznetsov I think that's okay, we can put the distinguishing detail somewhere else, like the testsuite name or 'classname'.  To be clear, the issue here isn't mixing up results from one run to the next, it's about not letting results from one host get masked by another from the same run.  So, confusing a RHEL 7.4 pass a 7.3 pass instead, or (god-forbid) a Fedora pass.  When the jUnits are combined, unless there's some distinguishing feature to each testcase or testsuite, they all get mixed up, leaving @rhatdan searching for hairs to pull 😀
My secondary motivation here is it gives us an interpretation-layer where we could (later) massage RHEL z-stream results.  For example, if in 7 years there's a bug that won't ever be fixed in 7.3.z. It's easier to mask the bogus failure at this layer, rather than try and fiddle with the tests - which work fine everywhere else.

@stevekuznetsov I think that's okay, I can put the distinguishing detail somewhere else, like the testsuite name or classname or wherever.  This is what the gubinator(?) thing does:
https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/16408/test_pull_request_origin_extended_conformance_crio/15/
The issue I'm trying to solve here isn't mixing up results from one run to the next, it's about not letting results from one host get masked by another from the same run.  Let's sync-up and see what we can do here, maybe there's a have-cake-eat-cake option.  For example, could gubinator consume a different jUnit from Jenkins?  Maybe there's another jUnit key I can fuss with that gubinator ignores but xUnit does not.

None of our tools combine jUnit between jobs. A 7.3 job will only have data fr builds of that job, for instance. See the test grid.. for instance
https://k8s-testgrid.appspot.com/kubernetes-presubmits#pull-kubernetes-e2e-gce-etcd3
https://k8s-testgrid.appspot.com/kubernetes-presubmits#pull-kubernetes-e2e-gce-bazel
Running the same tests, but in different contextx

@cevich needs a rebase

@stevekuznetsov Yes, context, that's what I want to preserve 😀
I'm assuming gubinator made those?  (I still dunno what it is 😕)  So looking at the first one, how is a developer to know those results came from a Fedora host vs a RHEL host, and which release it was?
What about the idea of feeding Gubinator a different jUnit file from what Jenkins consumes?  Could that work?  Smells like an easy solution to me.

Developers in k8s are not interacting with Jenkins. The tests in question assume there is no difference between them running on Fedora, or RHEL, or whatever -- I believe those tests specifically run on a Debian-based distro.
Gubernator shows jUnit output for a specific build of a specific job. TestGrid aggregates results per job. If you wanted to test both Fedora and RHEL, you'd have separate jobs to do it.

Thanks for the extra detail @stevekuznetsov so in the TestGrid, there's no association of the results to a platform or release, other than a job-name or number?  When viewing a grid for a build, how much digging (no. clicks) would a developer need to make to find out what distro/release it ran on?
The TestGrid is neat, and seems useful for giving a good overall picture of what's going on, I don't want to step on that at all.  The problem I'm trying to solve in this PR, is helping a CRI-O PR author to quickly find the source to any specific test result on a playbook run (possibly across more than one host).
IIRC, you/someone mentioned that the gubinator/grid only consumes jUnits of a particular name?
Is there some reason you can think of why we can't simply tabulate the results how we need them, into a different jUnit file from what gubinator consumes?  That would allow developers to mold the results as needed without affecting the testgrid.
i.e. The latest rebase/push (@rhatdan: code-complete but un-tested) would leave the following for archiving / further processing.
$WORKSPACE/artifacts/main.log                  # playbook log
$WORKSPACE/artifacts/canonical_results.xml     # junit_01.xml + testout.txt w/ hostnames & date/time
$WORKSPACE/artifacts/<hostname>/testout.txt    # raw integration testing output
$WORKSPACE/artifacts/<hostname>/junit_01.xml   # raw unmodified output

My preference would be to also add hostnames and date/time to the junit_01.xml but I can live without that as long as Jenkins is consuming (only) canonical_results.xml.

Note to me:  Waiting for #945 so I can get my hands on a real testout.txt and junit_01.xml to play with.

When viewing a grid for a build, how much digging (no. clicks) would a developer need to make to find out what distro/release it ran on?

If it's an important part of the job, it will be in the job name usually. For many of these tests, this is not an important axis of variance.

Is there some reason you can think of why we can't simply tabulate the results how we need them, into a different jUnit file from what gubinator consumes?

This approach sounds good to me.

rebase, --autosquash and --force push.

sigh...travis, travis, travis, what are you trying to say?

Travis-translation (according to jhonce): "I suspect your using <1.8.1"
Since that build was suppose to be for "tip", and this PR has nothing to do with building stuff...that sounds bad...need to dig more 😖

OIC the problem:  The integration build failed trying to find the e2e output (and vis versa).
@stevekuznetsov when you get a sec. (it's not urgent).  I see the results.yml playbook is being called w/o the tags used on the main job, e.g.
ansible-playbook -vv --become -i localhost, --tags all --become-user root --connection local /go/src/github.com/kubernetes-incubator/cri-o/contrib/test/integration/results.yml

Is it a terrible pain to have that use the same tags passed to main.yml?  If so, then I think maybe the easier (if less ideal) thing to do is make results.yml an include file from main.yml.  Alternatively, if you want to direct me toward the job definition, I can attempt the change myself 😀

@mrunalp FYI- the "Red X of failure" some jobs may be showing, probably has to do with my comment above to Steve.  This doesn't really affect running the tests at all, just collecting the results.

Right now the results are called in a common post-step for all jobs. Can we write the playbook instead to do as much as possible?
Also, some comments I had about the results.yml playbook from the other PR:

the playbook should expect to run on the same host as the tests did, not on the Jenkins infrastructure
the playbook should not be moving files around or renaming/duplicating them

The framework around your test is logging your output and gathering artifacts that you let us know where you are leaving them. We can handle the fact that Gubernator needs junit_xx.yml naming, etc, not something the job needs to deal with.

Can we write the playbook instead to do as much as possible?

Meaning main.yml?  Yes, I can re-write the results collection to happen there instead.

the playbook should expect to run on the same host as the tests did, not on the Jenkins infrastructure

There's no "control" host, these all run locally?  Crap, totally missed that one.  Okay, jeeze, right, well I guess it's got to run both ways then, otherwise we build-in that expectation (which is very un-ansible like).  Okay, I'll test that out, thanks for the heads up.

the playbook should not be moving files around or renaming/duplicating them

"No Moving", that's fine.  Please explain duplicating further, is that because the s3 artifacts loose all sub-directories so the filenames would clash?
@stevekuznetsov Thanks for the advice, I'll re-tool as I encounter the above issues while manual testing.

No, I think your concept of a follow-up playbook that is run as a post-job task is correct. I just meant that this playbook (results.yml) could do as much work as it can determine it needs to.

There's no "control" host, these all run locally?

There is a control host, but the domain of the test is on the target VM only.

Please explain duplicating further, is that because the s3 artifacts loose all sub-directories so the filenames would clash?

If the artifact already exists in the directory we are going to upload, having e.g. the jUnit there twice with different file-names leads to what we're seeing right now where they don't clash and both get uploaded, and we see results x2.

@stevekuznetsov until I get this PR shaped up, can you kill running that post-task results.yml playbook?  I think that's an easy short-term fix (so the build-status matches test results).  Let me know if not, and I'll get my arse moving quickly on another PR to fix it.  Thanks again.

No, I think your concept of a follow-up playbook that is run as a post-job task is correct.

Oh okay, I can roll with that too (after fixing it)

There is a control host, but the domain of the test is on the target VM only.

Excellent, I'm glad to hear that.  Okay that should make  #972 easier to swallow.

If the artifact already exists in the directory we are going to upload, having e.g. the jUnit there twice with different file-names leads to what we're seeing right now where they don't clash and both get uploaded, and we see results x2.

Thanks, that makes more sense and I understand the breakage better.  Okay, I'll re-tool the playbook here (or in another PR) so it's correct.  Sorry about the miscommunication, it seems I was making a lot of assumptions, so it's certainly my bad.

Yeah, I was totally misreading the results.yml when I reviewed it as well, sorry! I will now remove the call to results.yml.

Thanks Steve, good thing is we found the problem quickly and the fix(es) seems to be understood all around.

Rebased from master.  Changed behavior of results.py playbook:

Doesn't rely on / require use of any tags, doesn't assume it's running on any particular host.

Distinguishes between collecting results on the subject and copying them back to the control-host

When the control-host is the same as the subject, files are only copied around, into {{ artifacts }}.

When the control-host is different from the subject, additional behaviors occur:

A local directory {{ collection_dirpath }}/{{ inventory_hostname }}/ is created[*]
The contents from {{ artifacts }} on each subject are copied there.
The results.py script is run over specific files, producing a single {{ collection_dirpath }}/canonical_results.xml output file, safe for Jenkins xUnit consumption.

[*] I guess this needs to be changed so the {{ inventory_hostname }} sub-directory is only used if groups.all > 1?  (i.e. PB is running against an inventory of more than one subject).

okay, I rebased and --autosquash from master again and --force pushed after some manual testing.  Everything is working, testout.txt from the integration tests is being processed, and combined with junit_01.xml from the e2e tests.  I sent an example of the output to @stevekuznetsov for checking before running the CI tests.
There are some necessary followup changes needed in the jobs themselves (like using the wrapper-script), but I think I may be able to manage that on my own.  If you want to see this working locally it's pretty easy:

Stick some files named testout.txt and/or junit_01.xml into /tmp/artifacts/
cd contrib/test/
export WORKSPACE=$PWD
./venv-ansible-playbook.sh -i localhost, -l localhost -c local --verbose integration/results.yml
Examine /tmp/artifacts/junit_01.xml
Eat cake

Plan B (if steve isn't happy) is to change the output filename (one-line fix) to something other than junit_01.xml, and convince jenkins to consume it but gubinator/grid to not (should be possible).

Okay, steve says "okay" to combine them.  Let's see whatsheedoo...
/test all

Rebased onto #972 (which was rebased onto master).  Added a fudging commit for testing purposes.

(rebase from master & --force push)

/test ami

/test ami

@mrunalp @rhatdan check it out: Integration and e2e results all in one report!
(click on the 'Show 630 Passed Tests' link)
https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/kubernetes-incubator_cri-o/939/test_pull_request_crio_ami_fedora/69/
/me shakes his fist at travis

DO NOT MERGE: FUDGING JENKINS CHANGES HERE FOR TESTING

Note: I'm working with Steve on the job-changes needed to remove the above commit.

rebase from #972 & --force push

rebase from #972 & --force push.
Includes some more robust handling of jUnit input files under various conditions.
When input and output are the same file, make a backup before modifying.
Tweaked the way the subject's hostname is included into the results.
Tweaked the way additional ID information is included (PR number, commit, etc).

N/B: I'm also testing this manually using an Ansible inventory file (/tmp/hosts) containing IPs for a RHEL 7.4 and Fedora 26 VM using:
$ cd ./contrib/test
$ export WORKSPACE=$PWD
$ ./venv-ansible-playbook.sh -vv --tags all -i /tmp/hosts integration/main.yml
$ ./venv-ansible-playbook.sh -vv -i /tmp/hosts integration/results.yml

Since this is the whole point of this PR and #972 - being able to run properly from an inventory of remote hosts.

rebase and --force push.

rebase from master & --force push

/test ami

/test ami

Note to me: Broke b/c what https://github.com/kubernetes-incubator/cri-o/pull/1119 fixes?

note to me: testing again w/ 1119 fix (assuming it works)

/test ami

Moved into #957


Revert #661

This PR

reverts the changes in composer.lock by the composer update in #661;
applies composer update nothing to only add the platform changes (ext-filter + ext-json).

Just noticed that the infection run is called "Psalm" :D
Sorry about that

You are forgiven ;) Many thanks again!


Web Socket Support Check

upgrade notice for browsers that don't support web sockets
better error messages on web socket errors
log cleanup

LGTM


Typo on step 3.7 of the article.

[Enter feedback here]
Hello,
On this section: https://docs.microsoft.com/en-us/power-platform/admin/connect-gmail-oauth2#step-3-configure-oauth-consent
Step 7:
Authorized redirect URIs
There is a typo on the environment URL:
Current value:
Your environment's URL with "/_grid/cmds?dlg_gmailoauth.aspx"
Expected:
Your environment's URL with "/_grid/cmds/dlg_gmailoauth.aspx"
The URL example and the screenshot is correct but this value is wrong and confuses the customer and they submit tickets for troubleshooting. Please correct this.
Thank you
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 450ab446-3f0a-aabb-82d5-b57938d12def
Version Independent ID: b7159cad-3458-6cbf-e915-fe77e1f2b372
Content: Connect Gmail accounts using OAuth 2.0 - Power Platform
Content Source: power-platform/admin/connect-gmail-oauth2.md
Service: power-platform
GitHub Login: @jimholtz
Microsoft Alias: jimholtz

@v-iliva
Thanks for bringing this to my attention. The content has been corrected.

@v-iliva
Thanks for bringing this to my attention. The content has been corrected.


Trunk build deletes all files in path given without asking

To my surprise, trunk build --release -d . permanently deleted the repo (My CWD at that moment) that I was working on. Thankfully it was (mostly) backed up, but ideally the user should be aware of this when running the command (Maybe by warning them before or by prompting whether to remove files or not).

Sadly this was over 6 months ago so I can't fully remember, but I think it was only the target folder, sorry for the confusion

Thanks for sharing that trunk build --release --dist . is another rm -rf .
Next Tuesday, 2023-04-04,  becomes this issue a half year old.  Hopefully is at that time Added the missing information skeleton merged. So there is a place where the rm -rf _topdir_ can be documented.

Rather than documented, I think it would make more sense to fix the issue by warning the user about it, or even change the default behavior. But until then I guess documentation could do.

Still pretty relevant I'd say, thank you GH Actions bot

Yeah i just nerfed a whole configuration

Bump


[Spike][Bug] Fix Serialisation issues with stash and withdraw commands

Due to an unresearched issue something is failing to serialise within the player inventory during the use of /stash. This causes an error internally and also deletes the players inventory, these commands cannot be used for the time being.

Using NBTAPI (Used within flex command) rather than ItemStack.serialize() to avoid "Cannot be serialized issues" as NBT data is stored as a string, serializing the ReadWriteNBT object created rather than the ItemStackItself. Also unifies the object to a single format so certain items wont fail to serialize like before.


None no psk

Building on my PR #38, this removes :psk from the settings passed
to WpaSupplicant if using no encryption, thus avoiding the error
as described in #39.

Thanks!
I had to manually merge this branch, but it's in master now.


Use separate file for access logs

Commit 789b3e00 switched HTTP access logs to use JSON format so we could get
richer context information about requests. It put the access logs in the same
log file as the main server logs. This worked after the server started, but
when logs were rotated, the access logs didn't switch over to the newly-created
server logfile.
Switch to a separate file for access logs. The new file will be rotated on the
same schedule as the main server log.

#2608  👈 (View in Graphite)
main

This stack of pull requests is managed by Graphite. Learn more about stacking.


Skip Azure tests

Queued the tests with long running on enabled just to verify that it won't break tonight


refactor: refactor the logs of lvmcluster controller

Signed-off-by: Nitin Goyal<EMAIL_ADDRESS>

Can you post the before and after logs for the operator so we can see what the changes look like?

after-lvm.log

Can the following be done?

Remove "reconciler group":"lvm.topolvm.io","reconciler kind":"LVMCluster" from the messages. I think they are unnecessary.
Rework the messages to remove duplicate fields. For instance, the req name and namespace already have the information captured in name and namespace in the messages below:

{"level":"info","ts":1648710461.5450764,"logger":"controller.lvmcluster.lvmcluster-controller","msg":"openshiftSCC found, setting cluster type to openshift.","reconciler group":"lvm.topolvm.io","reconciler kind":"LVMCluster","name":"lvmcluster-sample","namespace":"lvm-operator-system","Request.Namespace":"lvm-operator-system","Request.Name":"lvmcluster-sample"}
{"level":"info","ts":1648710461.6462138,"logger":"controller.lvmcluster.lvmcluster-controller","msg":"Finalizer not found for LvmCluster. Adding finalizer.","reconciler group":"lvm.topolvm.io","reconciler kind":"LVMCluster","name":"lvmcluster-sample","namespace":"lvm-operator-system","Request.Namespace":"lvm-operator-system","Request.Name":"lvmcluster-sample"}

Uniform capitalization for messages. The older ones do not start with a capital letter.
uniform capitalization for field names. Older ones use all smallcase letters for the most part.
Reconsider some of the existing log fields. For instance:
{"level":"info","ts":1648710462.1055522,"logger":"controller.lvmcluster.lvmcluster-controller","msg":"already exists","reconciler group":"lvm.topolvm.io","reconciler kind":"LVMCluster","name":"lvmcluster-sample","namespace":"lvm-operator-system","Request.Namespace":"lvm-operator-system","Request.Name":"lvmcluster-sample","SecurityContextConstraint":"odf-lvm-vgmanager"}
may be better as
{"level":"info","ts":1648710462.1055522,"logger":"controller.lvmcluster.lvmcluster-controller","msg":"already exists","reconciler group":"lvm.topolvm.io","reconciler kind":"LVMCluster","name":"lvmcluster-sample","namespace":"lvm-operator-system","Request.Namespace":"lvm-operator-system","Request.Name":"lvmcluster-sample", "type","SecurityContextConstraint", "name", "odf-lvm-vgmanager"}
or
{"level":"info","ts":1648710462.1055522,"logger":"controller.lvmcluster.lvmcluster-controller","msg":"SCC odf-lvm-vgmanager already exists","reconciler group":"lvm.topolvm.io","reconciler kind":"LVMCluster","name":"lvmcluster-sample","namespace":"lvm-operator-system","Request.Namespace":"lvm-operator-system","Request.Name":"lvmcluster-sample"}

Can the following be done?

Remove "reconciler group":"lvm.topolvm.io","reconciler kind":"LVMCluster" from the messages. I think they are unnecessary.
Rework the messages to remove duplicate fields. For instance, the req name and namespace already have the information captured in name and namespace in the messages below:

{"level":"info","ts":1648710461.5450764,"logger":"controller.lvmcluster.lvmcluster-controller","msg":"openshiftSCC found, setting cluster type to openshift.","reconciler group":"lvm.topolvm.io","reconciler kind":"LVMCluster","name":"lvmcluster-sample","namespace":"lvm-operator-system","Request.Namespace":"lvm-operator-system","Request.Name":"lvmcluster-sample"}
{"level":"info","ts":1648710461.6462138,"logger":"controller.lvmcluster.lvmcluster-controller","msg":"Finalizer not found for LvmCluster. Adding finalizer.","reconciler group":"lvm.topolvm.io","reconciler kind":"LVMCluster","name":"lvmcluster-sample","namespace":"lvm-operator-system","Request.Namespace":"lvm-operator-system","Request.Name":"lvmcluster-sample"}

Uniform capitalization for messages. The older ones do not start with a capital letter.
uniform capitalization for field names. Older ones use all smallcase letters for the most part.
Reconsider some of the existing log fields. For instance:
{"level":"info","ts":1648710462.1055522,"logger":"controller.lvmcluster.lvmcluster-controller","msg":"already exists","reconciler group":"lvm.topolvm.io","reconciler kind":"LVMCluster","name":"lvmcluster-sample","namespace":"lvm-operator-system","Request.Namespace":"lvm-operator-system","Request.Name":"lvmcluster-sample","SecurityContextConstraint":"odf-lvm-vgmanager"}
may be better as
{"level":"info","ts":1648710462.1055522,"logger":"controller.lvmcluster.lvmcluster-controller","msg":"already exists","reconciler group":"lvm.topolvm.io","reconciler kind":"LVMCluster","name":"lvmcluster-sample","namespace":"lvm-operator-system","Request.Namespace":"lvm-operator-system","Request.Name":"lvmcluster-sample", "type","SecurityContextConstraint", "name", "odf-lvm-vgmanager"}
or
{"level":"info","ts":1648710462.1055522,"logger":"controller.lvmcluster.lvmcluster-controller","msg":"SCC odf-lvm-vgmanager already exists","reconciler group":"lvm.topolvm.io","reconciler kind":"LVMCluster","name":"lvmcluster-sample","namespace":"lvm-operator-system","Request.Namespace":"lvm-operator-system","Request.Name":"lvmcluster-sample"}

made most of the changes, pls take a look.


MAINT: stats.truncnorm/stats.betaprime: fix _munp for higher moments, add tests

Reference issue
Closes gh-18634
What does this implement/fix?
gh-18634 reported

When using the stats.truncnorm.moment, I always get an error, if I try to compute a moment greater than [or equal to] 5.

This fixes the bug and adds a distribution-specific tests.
Since there was no existing test coverage, I wrote a generic test for all distributions that override _munp. This discovered that betaprime raised a NotImplementedError for higher moments. Now, rather than raising an error, it falls back to a generic implementation.
Additional information
I'd like to backport at least the truncnorm fix; this does not require any specialized stats knowledge to review. The _munp function was relying on _pdf to accept a list; the fix was as simple as turning the list into an array.
I'd be happy to remove the betaprime part and generic test to make review easier, since nobody has complained about betaprime.

Thanks @tylerjereddy. Besides these distributions, all other distributions supported arbitrary moment order. The limitation in truncnorm was a clear bug because the _munp override passed a list to the private _pdf method, which always expects the input to be an array. The limitation in betaprime seemed intentional, but was undocumented and unnecessary. By replacing the four separate moment formulas with a general formula, we remove the limitation and simplify the code. The new distribution-specific tests confirm the implementations against outside references, and the generic test ensures that bugs like this don't arise again. I think it would be good to get these in 1.11.0. @tupui would you double check and confirm?


Prognostic run configuration hard codes cloud-based 79-level fv_core.res.nc asset

By way of fv3kube.c48_initial_conditions_overlay, the prognostic run currently hard codes the vertical coordinate file asset to: https://github.com/ai2cm/fv3net/blob/766778a1534d864e542a73c74575fb749abeb933/external/fv3kube/fv3kube/config.py#L22-L26
This presents two possible issues (issue two being the primary current motivation for making this post):

If we choose to restart from files with a different number of vertical levels, this hard code will prevent it.
Even when restarting from files with 79 vertical levels, this file asset is hard coded to be cloud-based, which is not convenient when running on HPC.

These can be worked around by patching the config file produced by the prepare-config tool with the appropriate path, but this is not a very clean solution.  Would we be open to adding another optional configuration attribute to the InitialCondition dataclass that allowed the user to specify the path to this file?
cc: @mrudko

Hi Spencer,
Thanks for sending me this email. I have been working on submitting ML  training and ML corrected run. I was looking at InitialCondition dataclass before you send me the email. I am right now learning through online tutorial (Real Python website) about dataclasses. Please give me time untill tomorrow and/or the day after tomorrow before I can report to you about the results.
Mike

From: Spencer Clark @.>
Sent: Monday, March 27, 2023 12:02 PM
To: ai2cm/fv3net @.>
Cc: Mykhailo Rudko @.>; Mention @.>
Subject: [ai2cm/fv3net] Prognostic run configuration hard codes cloud-based 79-level fv_core.res.nc asset (Issue #2197)
By way of fv3kube.c48_initial_conditions_overlayhttps://github.com/ai2cm/fv3net/blob/766778a1534d864e542a73c74575fb749abeb933/workflows/prognostic_c48_run/runtime/segmented_run/prepare_config.py#L88-L91, the prognostic run currently hard codes the vertical coordinate file asset to: https://github.com/ai2cm/fv3net/blob/766778a1534d864e542a73c74575fb749abeb933/external/fv3kube/fv3kube/config.py#L22-L26
This presents two possible issues (issue two being the primary current motivation for making this post):

If we choose to restart from files with a different number of vertical levels, this hard-code will prevent it.
When restarting from files with 79 vertical levels, this file asset is hard coded to be cloud-based, which is not convenient when running on HPC.

These can be worked around by patching the config file produced by the prepare-config tool with the appropriate path, but this is not a very clean solution. Would we be open to adding another optional configuration attribute to the InitialCondition dataclasshttps://github.com/ai2cm/fv3net/blob/766778a1534d864e542a73c74575fb749abeb933/workflows/prognostic_c48_run/runtime/segmented_run/prepare_config.py#L69-L91 that allowed the user to specify the path to this file?
cc: @mrudkohttps://github.com/mrudko
—
Reply to this email directly, view it on GitHubhttps://github.com/ai2cm/fv3net/issues/2197, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A5HHP47WCVDOAMW3UDMJNFDW6G23HANCNFSM6AAAAAAWJK5I5Q.
You are receiving this because you were mentioned.Message ID: @.***>

No worries, @mrudko!  I mostly just wanted to post this here to document the issue and discuss a possible solution before we work on addressing it.  I tagged you so that you would be kept in loop and could engage in the discussion.  I’d be happy to talk more about dataclasses offline.

@oliverwm1 FYI this is really the only cloud-file-related issue that came up in doing simulations on HPC.  @mrudko has implemented the solution I suggested.  He is just now working on adding some tests.
Feel free to weigh in if you have you thoughts, though this approach felt pretty simple and natural, so I didn't necessarily want to bother you right away.

Thanks Spencer, your approach of adding an attribute to the InitialCondition class sounds good to me.


Improve build tool to make it easier to maintain and more pleasant to use

Following #20332 I've spent some time trying to improve the build process by switching to npm scripts only. See comment.
My analysis is that npm scripts, while looking incredibly simple in theory, present several constraints and that the problems of the build tool are not necessarly related to the tools but mostly the way tasks are organized and orchestrated.
So I decided to evaluate a solution that would make the build tool simpler to use, easier to maintain, robust and more importantly pleasant to use for developers, while building as much as possible on the existing that works.
It turns out it was not that hard and really fun to do :-), so after a couple hours the evaluation turned something that was almost there. So I decided to polish it, update the doc and propose a PR so anyone can can have a look.
Image speak more than words so this is what I have now:

Here is the improvement that this PR brings.
Easy setup
npm install and that's it.
Simple to use
Only 3 commands for the user/developer:

dist: build JS, CSS and Docs.
test: build JS, CSS, Docs, test and lint.
watch: Start everything to dev efficiently.

Only 4 commands for the maintainer:

prep-release: prepare release.
publish: publish doc on GitHub.
change-version: update version in source files.
update-shrinkwrap: update shrinkwrap dependencies management.

Pleasant to develop with
npm run watch, Open your browser and voila! All changes made in scss, js and docs are automatically reflected in live on the local doc website. See gif above :)
Fast

build runs in 6s (it used to be 24s)
test runs in 20s
watch react to a modification of js, scss or docs files, handle everything
and reload the docs on the browser in about 1s (it used to be up to 10s just for sass + time for Jekyll to process the change)

Easier to maintain

Gruntfile is now 10 line of code and shouldn't have to be modified anymore.
For each task there is a short declarative config file under build/grunt.
Grunt tasks have been reorganized in a more logical way and can be found in build/grunt/aliases.js.
No more custom JS script.
No more Javascript function here and there (like in current Gruntfile.js).
Most of the Javascript code has been eliminated. Only the meaningful configuration remains in build/grunt and build/grunt/aliases.js and is  written in a more declarative way than it would in an npm script.

Stable

Most of the individual tasks are the one Bootstrap has been running for a while now, so that limit the risk of regression.
All the Grunt plugins are reliable and well maintained plugins (often more up to date than some npm cli equivalent).

How do I use it?

git clone -b build-rework https://github.com/vanduynslagerp/bootstrap
cd bootstrap
npm install
npm run watch
Open you browser to localhost:9001
Start coding! Each modification is applied within 1s and the doc website is reloaded automatically. Just code and watch the result live.

I understand the Bootstrap team, like many others, seems to desire to move to pure npm scripts. That said, I think I reached a pretty satisfying result, that worth to be evaluated, which is the point of this PR.

This is interesting, but a heads-up that I can't guarantee we'll use this.

@bardiharborow that's alright. I continue to improve it whenever I see something because I use it when I work on Bootstrap. The gain in comfort and productivity is really appreciable.
So even if Bootstrap decide to not go that way, it's not lost as I would have used it for a while.
Btw, let me know if you want me to improve/change something that would help this build tool to be adopted.

Hi @vanduynslagerp!
Thanks for wanting to contribute to Bootstrap by sending this pull request!
Unfortunately, your pull request seems to have some problems:

Changes must be made to the original Sass or Less source code file(s), not just the compiled CSS file(s).

You'll need to fix these mistakes and revise your pull request before we can proceed further.
Once you've fixed these problems, you can either ask the maintainers to re-open this pull request, or you can create a new pull request.
Thanks!
(Please note that this is a fully automated comment.)


Please add an example for using classes (with deps) as processors.

I am not sure if this is even possible, but it'd be nice to be able to define processors as regular NestJS service classes, with dependencies and all. Is that even possible?
Thanks.

Sure this is possible :slightly_smiling_face:
You can inject Bull queues in your injectables :
import { Injectable, OnModuleInit } from '@nestjs/common';
import { DoneCallback, Job, Queue } from 'bull';
import { InjectQueue } from 'nest-bull';

@Injectable()
export class MyProcessor implements OnModuleInit {

  constructor (
    @InjectQueue('my_queue') private readonly queue: Queue
  ) {}

  onModuleInit() {
    this.queue.process((job: Job, done: DoneCallback) => {
      done(null, job.data);
    });
  }

}

Don't forget to provide your MyProcessor injectable.
I haven't tested this code so let me know if it works properly :wink:

Perfect, that works! Didn't know about onModuleInit.
Thank you!


Indentation Issue

GitHub's tab indentation is 8 characters, but mine is 4. Need to make sure how to setup.

add ts={{number}} after uri.
ex)
{{uri}}?ts=4


Set-Cookie header to write two cookies

I want to write accessToken cookie and csrfToken as well - at one response.
I tried many things but nothing works.

@razb-viola this repository is for the specification itself, not any particular implementation.
As far as the specification is concerned, you can do this by defining two cookie parameters, one named accessToken and one named csrfToken.  If the implementation you are using does not do what you want with those, you should raise that issue with whoever makes the implementation.

// Declaration
class Rectangle {
constructor(height, width) {
this.height = height;
this.width = width;
}
}
// Expression; the class is anonymous but assigned to a variable
const Rectangle = class {
constructor(height, width) {
this.height = height;
this.width = width;
}
};
// Expression; the class has its own name
const Rectangle = class Rectangle2 {
constructor(height, width) {
this.height = height;
this.width = width;
}
};


Support for MJML scanner

When using MJLM templates with snippets, snippets are not detected because a MJML scanner is currently not supported.
This can be very useful when using the Shopware plugin FroshPlatformTemplateMail

hi
thanks for this great idea
so i think frosh also uses twit. that means snippets have a different syntax? or do you need same snippet twig syntax with a different file extension?
thanks

The FroshPlatformTemplateMail plugin supports MJML formats which makes styling emails much easier. In those templates, the MJML extension is used instead of twig.
Example folder structure with MJML files.

MJML content with snippets.
<mjml>
    <mj-body mj-class="body">
        <mj-wrapper mj-class="body__wrapper">
            <mj-section mj-class="content">
                <mj-column>
                    <mj-text>

{{ 'email.contact.subject'|trans|sw_sanitize }}

{{ 'email.contact.intro'|trans|sw_sanitize }}

                        ...
                    </mj-text>
                </mj-column>
            </mj-section>
        </mj-wrapper>
    </mj-body>
</mjml>

In this example the snippets in the MJML file are not detected by the twig-scanner.

email.contact.subject
email.contact.intro

Hi
a perfect
thats amazing
so we have a
a) a new extension
b) new snippet pattern in that files
great
ill do it
thanks a lot :)

you can test dev-master with the new scanner
should work now, QA not started yet but unit tests are all passing

Thanks for the fast response!
I've tested the feature on the dev-main branch but it still does not recognize the .mjml files? Do I have to activate to MJML-scanner?
#1 [EMPTY_CONTENT] Found empty translation
   - Locale: en-GB
   - File: /.../.../.../.../src/xyz/Resources/snippet/storefront.en-GB.json
   - Line: 30
       [x]: email.contact.intro

you need to use the scanner:usage command
the detection in files is integrating with an external system from the frameworks point of view (so twig files)
and its not 100% correct...because snippets might not appear inside templates but could be used inside PHP scripts....
so the basic idea on scanners is to get some quick insight in what MIGHT not be used :)
php bin/console scanner:usage .... --scanner=mjml --dir=xyz
that should work

I'm running php vendor/bin/phpunuhi scan:usage --dir=./src --scanner=mjml --set=General
but still get the following errors:
+-------+----------------------------------------------------+----------------------------------------------------------------------------------+
| #     | Keys (2)                                          | Error                                                                            |
+-------+----------------------------------------------------+----------------------------------------------------------------------------------+
| 1     | email.contact.intro                             | Key not found in any file                                                        |
| 2     | email.contact.subject                                | Key not found in any file                                                        |

This is my simplified configuration

Ok that's funny because of these unit tests with the exact same data haha
https://github.com/boxblinkracer/phpunuhi/blob/main/tests/phpunit/Bundles/Scanner/MJML/MjmlScannerTest.php
could only think of the file not being found somehow
ill check it out...maybe ill add a debug output which files have been scanned
would be good right? or a verbose mode for that maybe?

Great, I've tested the fix on the dev-main and it works 👍
The last step to make the CI validation work is the compatibility for mjml with the validate:all command. I suppose it's better to create another ticket for this?

Hi
amazing thank you
yeah i didnt really want to add the scanners to the official validation command
because the result is not 100% valid....please see the warnings in the text
if snippets are added thru PHP for instance...then its okay that they are not found in those files.
so scanners are more some working helpers.....
do you have any better idea on that?

released in 1.17.0


“Storage Scale” tag update

Is your feature request related to a problem? Please describe.
“Storage Scale” tag update
Describe the solution you'd like
“Storage Scale” tag update

Sorry, not sure I understand the specific request. What do you mean by "Storage Scale tag update"?


train_eval_metrics.zip not found for Cartpole-v1

Hello,
I encounter the following error when I run the hugging face demo line from the README.md.
python -m utils.load_from_hub --algo ppo --env CartPole-v1 -f logs/ -orga sb3

Downloading from https://huggingface.co/sb3/ppo-CartPole-v1
No normalization file
Traceback (most recent call last):
  File<EMAIL_ADDRESS>line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File<EMAIL_ADDRESS>line 87, in _run_code
    exec(code, run_globals)
  File "/Users/quentingallouedec/rl-baselines3-zoo/utils/load_from_hub.py", line 50, in <module>
    train_eval_metrics = load_from_hub(repo_id, "train_eval_metrics.zip")
  File "/Users/quentingallouedec/rl-baselines3-zoo/.env/lib/python3.9/site-packages/huggingface_sb3/load_from_hub.py", line 16, in load_from_hub
    downloaded_model_file = hf_hub_download(
  File "/Users/quentingallouedec/rl-baselines3-zoo/.env/lib/python3.9/site-packages/huggingface_hub/utils/_deprecation.py", line 32, in inner_f
    return f(*args, **kwargs)
  File "/Users/quentingallouedec/rl-baselines3-zoo/.env/lib/python3.9/site-packages/huggingface_hub/file_download.py", line 778, in hf_hub_download
    return cached_download(
  File "/Users/quentingallouedec/rl-baselines3-zoo/.env/lib/python3.9/site-packages/huggingface_hub/utils/_deprecation.py", line 32, in inner_f
    return f(*args, **kwargs)
  File "/Users/quentingallouedec/rl-baselines3-zoo/.env/lib/python3.9/site-packages/huggingface_hub/file_download.py", line 554, in cached_download
    r.raise_for_status()
  File "/Users/quentingallouedec/rl-baselines3-zoo/.env/lib/python3.9/site-packages/requests/models.py", line 960, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/sb3/ppo-CartPole-v1/resolve/main/train_eval_metrics.zip

I checked  the repo https://huggingface.co/sb3. Indeed, for some environments, including CartPole-v1, there isn't any train_eval_metrics.zip file.

Hello,
thanks for reporting the issue, the agent was uploaded with older version of push to hub, will update it soon ;)

Fixed https://huggingface.co/sb3/ppo-CartPole-v1 ;)


Need half-step option to support some encoders for GUI use

The KY-040 is a popular and low-cost rotary encoder which has a mechanical detent. Between each detent step there are only 2 transitions of the CLK and DT signals, not a full cycle (4 transitions). The existing library only signals a rotation increment or decrement after 4 transitions have been detected.
This is problematic if the encoder is used for a GUI application. For example, scrolling a menu requires a 2 detent rotation (instead of 1 detent). To the user this feels like the input is has missed a detent and is thus non-responsive.
A solution is to add an optional parameter, half_step, which causes the library to signal a rotation increment/decrement after seeing only 2 encoder state transitions.
This has been implemented in the following branch of the forked project.
https://github.com/urbite/micropython_rotary_encoder/tree/feature/half-step

Pull request accepted. Commit: Add half-step rotation increment/decrement parameter