ISSUE-38: Checking for SaslAuthenticated event when connecting.

Here is a suggestion of how to handle this issue. It adds new constructors to ZkClient to let the app specify whether it expects SASL authentication, and if positive, then it waits for the SaslAuthenticated event.
Let me add a couple of notes to this issue:

I could do a patch that does not add constructors and instead detects the presence of the jaas login file. For this first cut, I opted for giving the option to the app, but I could switch to the other way. In fact, detecting the presence of the jaas file is typically the preferred mechanism.
The changes here right now don't do anything about the auth failure event. If the app is waiting for the SaslAuthenticated event and the event doesn't come because of an auth failure, then it will fail on a timeout because the waitForKeepeState method only checks for the specific state. To deal with it in a better way, or at least have a better error message, we would need to change the state transition handling.

I have changed my mind and removed the new constructors, and now ssl authentication is configured through system properties like with the underlying zookeeper client.

Hey @fpj , so i'm fine with that changed.
I'm just wondering if that isZkSaslEnabled() should be invoked for every waitUntilConnected() or if it should determined when constructing the ZkClient and kept as a field! WDYT ?
Also is it possible to simulate that auth failure in a test ?
Maybe we can give the waitForKeeperState() an enumSet of states where it should fail on !?

Hey @jzillmann, Thanks for checking it out.

I'm just wondering if that isZkSaslEnabled() should be invoked for every waitUntilConnected() or if it should determined when constructing the ZkClient and kept as a field! WDYT ?

isZkSaslEnabled() isn't called frequently, but I'd say it is fine to do it once upon the creation of the object. I'll make the change.

Also is it possible to simulate that auth failure in a test ? Maybe we can give the waitForKeeperState() an enumSet of states where it should fail on !?

Simulating the the auth failure in a test is fairly simple, the issue is that right now it will fall with a timeout because waitForKeeperState will be waiting for the SaslAuthenticated. Passing a set of states to fail on works if you don't mind the change of signature of the method. waitForKeeperState is a public method, so I'll need to add a second call to preserve compatibility.

@jzillmann I've pushed some more changes based on your comments. I've added new test cases, but decided to ignore AuthFailed for the following reason. The client gets AuthFailed when auth fails and the  property zookeeper.allowSaslFailedClients is enabled. If the property is disabled, then the servers will keep disconnecting the client, and ZkClient will perceive it as a failure to connect, which is fine. Overall the property is there to give flexibility to client and server to keep going with an unauthenticated connection. Unauthenticated clients can still manipulate znodes with open acls.
I tried adding an EnumSet to pass failed states, but then found that I wanted to return true in waitForKeeperState on an AuthFailed event to allow the connection to proceed. It felt awkward to return positively upon an event that I'm calling failed though. I also tried to pass an EnumSet to wait for multiple events, but it turns out that it isn't clear cut because the state machine is complex and it would end up being specific for this case. The bottom line is that these changes turned out to add complexity, and making it work properly might require deeper changes.
I've opted to keep it simple for now. The current behavior is that the connection doesn't succeed in the case of authentication failure, independent of how the server deals with SASL auth failures. It is a reasonable behavior, but doesn't provide all the flexibility zk allows. If you think it is worth working on this change, then we could give it s try in a different branch.
Let me know if this makes sense, please.

@fpj: In Kafka, currently we have tools (kafka-topics) that may just need to read from ZK and ideally they can run without sasl configured. So, you are saying that those tools will fail if sasl is not configured? That may not be convenient. Could we add a new waitForKeeperState() that takes a set and wait on either  AuthFailed or SaslAuthenticated?

@junrao

So, you are saying that those tools will fail if sasl is not configured?

I'm not sure what I said that led you to that thought. The failure scenarios I described are for the cases in which the user is configuring sasl to be used. If the jaas config file isn't passed, then it is all good. Note that Kafka tests pass consistently for me with this fix, including all tests that don't use auth and the test cases in KAFKA-2732, which were failing for me and triggered the investigation of this issue.

Could we add a new waitForKeeperState() that takes a set and wait on either AuthFailed or SaslAuthenticated?

There are two points here:

I have tried to pass a set of states and came to the conclusion and generalizing this way could lead to broken behavior in the future if we start passing arbitrary sets of states.
We could succeed on both AuthFailed and SaslAuthenticated, but the issue is that it is going to be transparent to the app using it because we are just returning a boolean. At that point, I thought of having a isAuthenticated call, but thought that I was adding too much complexity and determined that it was better to fail the connection if auth fails, independent of whether the server allows the connection to move forward or not.

@fpj: Is the following true?

If client is configured with sasl and auth passes, client eventually will get SaslAuthenticated state.
If client is not configured with sasl and zk server has zookeeper.allowSaslFailedClients enabled, the connection will be established and client eventually will get AuthFailed state?

@junrao

If client is configured with sasl and auth passes, client eventually will get SaslAuthenticated state.

Yes, SyncConnected followed by SaslAuthenticated.

If client is not configured with sasl and zk server has zookeeper.allowSaslFailedClients enabled, the connection will be established and client eventually will get AuthFailed state?

No, if client doesn't try to authenticate, then the server will accept the connection. AuthFailed happens when the client tries to authenticate, but fails, e.g., user is configured at the server as fpj but the client passes fjp. Also, zookeeper.allowSaslFailedClients needs to be enabled for the client to receive AuthFailed. If it is disabled, then the server simply drops the connection on the authentication failure, and since all servers will be doing it, then zkclient will eventually give up.
The rationale for AuthFailed is this. ZK servers accept connections from unauthenticated clients, so why not accept connections that failed to authenticate (such clients in principle won't be allowed to perform actions that require authorization)? This is a valid behavior, but apps often prefer to drop the connection if auth fails because the intent of the app was to have an authenticated connection.

So ya, a bit complex with all those options. But if i get things right with the current solution every possible scenario could still be covered (correct me @fpj  if i'm wrong):

SASL protected server - SASL authenticated client (-Djava.security.auth.login.config=myJass.conf)
SASL protected server - unauthenticated client (-Dzookeeper.sasl.client=false)

@jzillmann

But if i get things right with the current solution every possible scenario could still be covered

yes, that's right

SASL protected server - unauthenticated client (-Dzookeeper.sasl.client=false)

you don't even need to do that, this option was actually created because of this issue: https://issues.apache.org/jira/browse/ZOOKEEPER-1657
are you ok otherwise with the pull request?

So i played with the tests...
As soon as there is a AuthFailed, the client will fail with an timeout exception, even if 'zookeeper.allowSaslFailedClients' is set to false, right ?
So is there any reason in ZkClient#waitForKeeperState() not to raise an authentication exception in case there is an AuthFailed ? I feel that would make it clearer for the users of the client !?

Ok, anyway pushed the change and triggered the release of ZkClient-0.7.
It should be online in the next 24h.
But we should talk about the Timeout- vs AuthenticationException topic to see if we can improve things!

@jzillmann I actually made the change and was about to push, if you want I can do it in the next few minutes

@fpj
So its to late for 0.7...
We could do a 0.7.1 or just fix this in master for upcoming versions...
So please base you work on current master and in case you wanna have that fast released as a 0.7.1 pease say so!


Split main.py functionalities into separate files

Fixes #121. Since massive changes like this will totally mess up diffs, I tried to make as few logic changes as possible when copying and pasting. This change has been tested.
The only real "logic change" I did was to remove the get_random_emoji() function and get a random emoji inline, since we only used it once and it was the only utility function which fit in neither song_utils nor discord_utils. I put command_form into a separate file voting.py for now, since eventually i'd like all of the forms logic to be there and have only thin wrapper code translating message --> command in either main.py or bust.py

Note that since we're moving essentially all code, the result is that any invocation of git blame will likely run into this commit, causing noise. We can remove this by creating a .git-blame-ignore-revs file at the root of the repository, containing something like the following:
# Commits in this file will be removed from GitHub blame results.
#
# To use this file locally, use:
#   git blame --ignore-revs-file="path/to/.git-blame-ignore-revs" <files>
#
# or configure the `blame.ignoreRevsFile` option in your git config.
#
# If ignoring a pull request that was not squash merged, only the merge
# commit needs to be put here. Child commits will be resolved from it.

# Split main.py functionalities into separate files (https://github.com/anoadragon453/busty/pull/125).
<merge commit>

we'll need to merge first before we know what commit to put though.


feat: add custom_api_max_price_age parameter

BEFORE REVIEW
Please note that this PR is pointing to be merged into master but this is only so that the CI runs on every new commit. We should instead merge this into the v1.17.0 hotfix branch instead (to not diverge master from upstream).
TL;DR: Do not merge to master!

Before submitting this PR, please make sure:

[x] Your code builds clean without any errors or warnings
[x] You are using approved title ("feat/", "fix/", "docs/", "refactor/")

A description of the changes proposed in the pull request:
This PR adds a new custom_api_max_price_age which allows users to set a threshold for how old the price returned by a custom api can be before invalidating it.
Before this change, the CustomAPIDataFeed would return a cached version of the price whenever there is any error when trying to fetch the price from the API. This is obviously not safe as the price could be outdated but still being used as the reference price for placing orders by the strategies (for example in the PMM). With the new parameter the price will essentially "expire" after certain time if it's not being updated successfully, and just return NaN. At the same time the PMM strategy will check that the price is not NaN before running the strategy steps (proposing/modifying orders, etc) and will just cancel any active orders for safety instead.
By default the new parameter will be disabled (-1) for backwards compatibility and to not introduce changes to any of the existing strategies.
Tests performed by the developer:

Wrote unit tests for the CustomAPIDataFeed which didn't exist before
Ran this locally using this simple python API that exposes methods for testing the behaviour of the bot in different circumstances:

GET /price get the latest price
GET /set?price=0.5set the latest price to 0.5
GET /off switch off /price for it to return 500 with a NaN response (to simulate failures)
GET /on switch on the /price endpoint to start returning 200 responses again and the price value
from flask import Flask, request

app = Flask(__name__)

PRICE = 1.0
PRICE_STATE = True

@app.route('/price', methods=['GET'])
def get_price():
    if PRICE_STATE:
        return str(PRICE), 200
    else:
        return "NaN", 500

@app.route('/set', methods=['GET'])
def set_price():
    global PRICE
    PRICE = float(request.args.get('price', '1.0'))
    return "Price set", 200

@app.route('/off', methods=['GET'])
def off_price():
    global PRICE_STATE
    PRICE_STATE = False
    return "Switched off", 200

@app.route('/on', methods=['GET'])
def on_price():
    global PRICE_STATE
    PRICE_STATE = True
    return "Switched on", 200

if __name__ == '__main__':
    app.run(debug=True)

Tips for QA testing:
Run the PMM strategy pointing to the local API and try different scenarios like modifying the set price and flipping the endpoint on and off.

I know this is non-standard but given we want to submit this PR upstream, I'm leaving some comments on the issue description as well. I think the first sentence after "A description of the changes proposed in the pull request:" is hard to read. I would rewrite those first few lines by starting with the status quo: there is this CustomAPIDataFeed which can feed a reference price into the strategies, if the API throws an error the class always returns the last valid response, this PR adds lifetime to the values collected from the API, and a setting to configure when a value should expire.
I think starting with custom_api_max_price_age is jumping the gun a bit.

I know this is non-standard but given we want to submit this PR upstream, I'm leaving some comments on the issue description as well. I think the first sentence after "A description of the changes proposed in the pull request:" is hard to read. I would rewrite those first few lines by starting with the status quo: there is this CustomAPIDataFeed which can feed a reference price into the strategies, if the API throws an error the class always returns the last valid response, this PR adds lifetime to the values collected from the API, and a setting to configure when a value should expire.
I think starting with custom_api_max_price_age is jumping the gun a bit.

Yeah this sounds reasonable to me. I was planning to rewrite the PR description before I submit it to upstream and ask for internal feedback. I agree that the way we frame this is important (since this seems like a bug rather than a new feature)


show download counts for the extensions

fixed in https://github.com/zaggino/brackets-npm-registry/commit/86e548065a952d899be4d85bc298945f41fb5eaf


1.0.0-rc1-final does not support .NetCoreApp 1.0

Is this normal, or a bug.  Or does the code currently not support the latest RC2 1.0.0 of .NetCoreApp?

Why are you using rc1-final? 1.0 has shipped.

I found it yesterday, what confused me is the name of the package changed.
This one is
Microsoft.AspNet.Session
And the one for 1.0.0 is
Microsoft.AspNetCore.Session

All the AspNet packages has been changed to AspNetCore before 1.0.0 is released. For more information please visit this link

Thank you for the link, exactly what I was looking for.

You 're welcome :smile:

Looks like there's nothing pending in this issue so closing for now. Please re-open if there are still unresolved issues.


Issue 4: Dataview support

Thanks so much! This nearly works perfectly; the only issue I've found is that Dataview queries may replace the text of a link to a note with some other text. For example:
LIST WITHOUT ID link(file.path, "test") FROM "Notes"

will create a list of links to every note in /Notes but where the text of each link is with the word "test", like the following:

The problem is that this version of the plugin replaces that text with the mathLink if it's defined, no matter what. So this displays as

This isn't a big problem for me, because the only reason I use the link(,) feature is to get the mathLink to display in my vault. As such, this bug breaks my workaround:

but also fixes the thing it's working around:

However, it could still break other people's workflow, so it should probably be fixed before a release or if that isn't possible, there should be an option to turn the feature on and off.
Thanks again!

Ahh, I see; I didn't know that dataview can replace the text of a link. If it leaves some trace that it does so, this should be an easy fix. I'll look into it.

Okay, I tried another method that hopefully should fix this. Could you please try downloading main.js (from the same 0.2.6 pre-release) again and see if it works?

Works perfectly in my vault, thanks so much!


ARROW-4847: [Python] Add pyarrow.table factory function

Rudimentary implementation for https://issues.apache.org/jira/browse/ARROW-4847.
For now I made the minimal choices:

Which formats to support? For now I only included DataFrame and dictionary, and not list of arrays or list of batches, because DataFrame/dictionary are most unambiguous and don't require additional keywords to be interpreted (eg list of arrays needs a list of names (or the schema)).
from_batches could probably easily be supported (eg by checking the first element of the list being a RecordBatch), but then it might also be nice to support a list of DataFrames to be interpreted as a list of batches?

Which keywords to expose? For now I only included the keywords that are common to the different constructors being dispatched to (in practice only schema), in the idea you can always use the specialized constructors if needing more control.

Codecov Report

Merging #4601 into master will decrease coverage by 23.46%.
The diff coverage is 91.3%.

@@             Coverage Diff             @@
##           master    #4601       +/-   ##
===========================================
- Coverage   88.58%   65.11%   -23.47%
===========================================
  Files         860      478      -382
  Lines      108214    61814    -46400
  Branches     1253        0     -1253
===========================================
- Hits        95862    40253    -55609
- Misses      12073    21561     +9488
+ Partials      279        0      -279

Impacted Files
Coverage Δ

python/pyarrow/__init__.py
40.54% <ø> (ø)
:arrow_up:

python/pyarrow/tests/test_table.py
99.63% <100%> (+0.01%)
:arrow_up:

python/pyarrow/table.pxi
87.23% <66.66%> (-0.27%)
:arrow_down:

cpp/src/arrow/util/memory.h
0% <0%> (-100%)
:arrow_down:

cpp/src/gandiva/date_utils.h
0% <0%> (-100%)
:arrow_down:

cpp/src/arrow/compute/kernels/compare.h
0% <0%> (-100%)
:arrow_down:

cpp/src/arrow/util/memory.cc
0% <0%> (-100%)
:arrow_down:

cpp/src/arrow/filesystem/util-internal.cc
0% <0%> (-100%)
:arrow_down:

cpp/src/arrow/util/sse-util.h
0% <0%> (-100%)
:arrow_down:

cpp/src/gandiva/decimal_type_util.h
0% <0%> (-100%)
:arrow_down:

... and 613 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0631357...69ab78a. Read the comment docs.

I would say to make pa.table([a0, a1, a2], columns=['foo', 'bar', 'baz']) (or names?) also work would be reasonable. Table.from_pydict does not have a names/columns argument (only schema), so this might be a good time to fix that

I would say to make pa.table([a0, a1, a2], columns=['foo', 'bar', 'baz']) (or names?) also work would be reasonable

pandas is treating such list of arrays as rows:
In [11]: pd.DataFrame([np.array([1, 2, 3]), np.array([2, 3, 4])], columns=['a', 'b', 'c'])
Out[11]:
   a  b  c
0  1  2  3
1  2  3  4

which is not to say that is better / we can't deviate from it, but IMO it certainly indicates that it might not be obvious what the behaviour will be.

Table.from_pydict does not have a names/columns argument (only schema), so this might be a good time to fix that

What would that argument do? (since the dict already has names in it) To select a subset from the dictionary, or to provide a specific order of the columns?
It might also be interpreted as overwriting the names from the dict (which I would not do, as that would rely on the order of the dict). Although you could say the same for the schema argument of course.

Although you could say the same for the schema argument of course.

Good that I check. This is actually buggy. It is "kind of" overwriting the names of the dictionary, but without properly doing type conversion:
In [15]: table = pa.Table.from_pydict(
    ...:     {'a': [1, 2, 3], 'b': [3, 4, 5]},
    ...:     schema=pa.schema([('a', pa.int64()), ('c', pa.int32())]))
In [16]: table
Out[16]:
pyarrow.Table
a: int64
c: int32

In [17]: table.to_pandas()
Out[17]:
   a  c
0  1  3
1  2  0
2  3  4

So I am passing a schema with different names, but also with a non-default type (int32 vs int64), and the values in the column are probably first converted as int64 but then viewed as int32 instead of casted (or rather, instead of being converted directly to int32 in the first place).
Will open a JIRA for this.

+1. @jorisvandenbossche if you would please use git rebase instead of git merge in this project, it eases maintenance for us since we are rebase-centric here.
I'll open a follow up JIRA about adding support for Table.from_arrays to this function


🤗 [Question]: Is there any updates about fiber v3 support for middlewares such a otelfiber?

Question Description
I was checking other issues about unsupporting middlewares for fiber v3
Is there any news about it cuz I can't find any other solution for otel tracing in fiber v3 and I so don't want to go back on v2 :(
Code Snippet (optional)
No response
Checklist:

[X] I agree to follow Fiber's Code of Conduct.
[X] I have checked for existing issues that describe my questions prior to opening this one.
[X] I understand that improperly formatted questions may be closed without explanation.

#1163

@defany v3 is still in beta, middlewares will be upgrade once we are closer to release.

love u, ty


Update pnpm-lock.yaml

Diff generated after running pnpm install with pnpm version 9.7.1

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Add scripts for automated development contribution

🛠 Note:
We are meticulous about merging code into the main branch, so please understand that pull requests not meeting the project's standards may be rejected. It's never personal!
🎮 Note for game-related scripts: These have a lower likelihood of being merged.

✍ Description
Please see everything inside the dev-scripts folder!
I have a readme file with the breakdown and steps needed for the scripts.
Essentially what these scripts do is to setup a branch for creating a new helper script and automate a couple of things that may be forgotten like changing the build.func, install.func etc to point to the new branch instead of the official repo!
@michelroegl-brunner there you go mate, they're rough around the edge since they were developed for personal use but the workflow works! Although change_lxc.sh and cleanup_lxc.sh need a bit of more testing, new_lxc.sh seems to work for my needs pretty good (tested with 3-4 new helper scripts that I created a PR for them!)

Please remove unneeded lines!

Related Issue: # (issue number, if applicable)
Related PR: #920 (if applicable)
Related Discussion: https://github.com/community-scripts/ProxmoxVE/pull/920#issuecomment-2563582115 and https://github.com/community-scripts/ProxmoxVE/pull/920#issuecomment-2563602712(if applicable)

🛠 Type of Change
Please check the relevant options:

[ ] Bug fix (non-breaking change that resolves an issue)
[ ] New feature (non-breaking change that adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change unexpectedly)
[ ] New script (a fully functional and thoroughly tested script or set of scripts)

✅ Prerequisites
The following steps must be completed for the pull request to be considered:

[ ] Self-review performed (I have reviewed my code to ensure it follows established patterns and conventions.)
[ ] Testing performed (I have thoroughly tested my changes and verified expected functionality.)
[x] Documentation updated (I have updated any relevant documentation)

📋 Additional Information (optional)

Thank you, ill have a look!

Thank you, ill have a look!

Not really used to writing bash scripts, I'm more of a PowerShell guy, so excuse my poor bash scripting skills 😁

All good. I merhe it in a sepereate branch for testing.


[BUG] [DARWIN] MTRSetupPayload doesn't parse the same thing

Reproduction steps
I want create MTRSetupPayload using both qrCode (MT:xxxxx) and using manual pairing key (11-digit key) using:
MTRSetupPaylad(onboardingPayload: theString)
However, upon further inspecting, the same payload doesn't produce the same MTRSetupPayload.
From the same payload:
Using QRCode :
JSONMap is {manualEntryCode:<PHONE_NUMBER>2, productId: 32768, discriminator: 3840}
Using manual pairing code:
JSONMap is {manualEntryCode:<PHONE_NUMBER>2, discriminator: 15, productId: 0}
I also tested with the deprecated API i.e. MTRManualSetupPayloadParser to create the MTRSetupPayload, but still getting the same result.
Bug prevalence
Everytime
GitHub hash of the SDK that was being used

Platform
darwin
Platform Version(s)
No response
Anything else?
Correct me if I am wrong,

QRPayload and its manual pairing key should produce the same MTRSetupPayload

A Matter QR code encodes a "long discriminator" which is 12 bits.  A Matter manual code only encodes a "short discriminator", which is the top 4 bits of the 12-bit code.  You can see that here: 3840 == 0xf00 and 15 = 0xf.  Given an MTRSetupPayload you can examine hasShortDiscriminator to see what its discriminator value means (which will depend on what sort of setup code it came from).
Further, a Matter QR code always has a vendor id and product id, but a Matter manual code only has those when it's a 21-digit code; 11-digit ones do not encode a vendor id or product id.  You also see that above, with productId: 0.
The only thing that is guaranteed to be the same across the two payload types is the setupPasscode, which is the one thing that you absolutely have to have to commission a Matter device.  All the other information in there is just for filtering which devices you might try to discover and commission.

qrcode: long discriminator (12bit)
11 digit code: short discriminator (first 4bit), no vendor and product id
12 digit code: short/long discriminator?, has vendor and product id
always the same both payload type: setupPasscode, also the most important thing that identifies the device, the vendor, product id is just to further filter it


Unable to change user's role

Describe the bug
As an admin of the default organization I want to change another user's role from member to manager or owner and I get an error in the red bottom bar that reads "You don't have permissions to update your organization role." I get the same error in the REST API (see attached screenshots). I don't have Debug logging enabled so all I see in logs is
web        | INFO:     <IP_ADDRESS>:52367 - "PUT /default/users/4 HTTP/1.1" 403 Forbidden.
Expected behavior
I can change user's role according to the Dispatch documentation here.
Screenshots

Thanks for the report. I was able to reproduce it. I'll put a fix for it. In the meantime, please use the following CLI command: dispatch user update -o <ORG_NAME> -r <ROLE> <EMAIL>

@mvilanova Thank you for your response. Yes, I can do it if I have execute option available but, in my case, I launched the app in AWS ECS with docker-compose using ECS context and don't have SSM access to the container so the API was the only non-UI option to do that.

Fixed it in #2242. Let me know if you still have issues with it.


Template  section in identity.xml

Fixed By: https://github.com/wso2/carbon-identity-framework/pull/2322


The minigame concept

Cross post from MinigameCore/MinigameCoreAPI#3
Currently, the Minigame system works on a class system. While this looks fine, it is not a good thing from an API point of view (After all the framework is made of an API "and" an implementation). The users of the API should not be aware of the implementation details, as much as possible.
Although developers just can extend the Minigame class and add content, it limits them as only one class can be extended per class. It also very much exposes the implementation. Since we have the blessing of Guice, an interface format is better.
Also currently there is no centralized way of handling minigames. MinigameCore provides the classes that should be used and the plugins implement them however they want. MinigameCore in itself isn't aware of what minigames are present. The implementation should keep track of what games are available, and be able to control if they should be active or not.
In addition, Events should become a major component of the framework. Will detail on this later
This is granted to be split over multiple issues/PRs.

Although developers just can extend the Minigame class and add content, it limits them as only one class can be extended per class.

This is a realy common issue. What you can do to solve it is to create a component based system. In game engines you have e.g. an entity. The entity itself is just a basic class, it knows its unique idea, position, velocity and some other stuff that is required to just have an basic entity, but to add real functions like moving, jumping, glowing you can add components to your entity. You could consider to adopt to concept.
So instead of allowing to extend various functions into a minigame you can provide components. This would also be really great for all contributors, because they just can write a single class (component) to add new stuff that can be used in all frameworks. :)

What I had in mind was something very similar... however I hadnt yet polished it and discussed it with @Flibio yet


Missing SPI SCK on STM32F103vctx

Describe the bug
A custom board using stm32F103vctx MCU uses SPI1 with remaped sck pin pb3 does not output any clock signal. The SPI1 peripheral was used and configured with the stm vendor dts / dtsi. However once I tried to sent out some SPI commands using the stm spi driver I noticed that no spi clock signal is emitted on pb3 (using an oscilloscope). The other spi signals (MOSI, MISO and nCS on pb4, pb5 and pb8) are correctly pushed by the stm spi driver.
A quick implementation using only STM HAL produces the same register contents of the affected register banks GPIOB, AFIO, and SPI1. However using STM HAL the clock signal is emitted correctly on pb3.
To Reproduce
Steps to reproduce the behavior:

Grab a plattform using a stm32F103 MCU (preferably stm32F103vctx)
configure SPI1 in the dts e.g. like this
pinctrl-0 = <&spi1_sck_master_pb3 &spi1_miso_master_pb4 &spi1_mosi_master_pb5>;  cs-gpios = <&gpiob 8 (GPIO_ACTIVE_LOW)>;      pinctrl-names = "default";  status = "ok";
try to send something on SPI using stm spi driver and spi_transceive_dt
measure spi clock on pb3 (e.g by using oscilloscope or logic analyzer)

Expected behavior
SPI communication with remapped Pins pb3, pb4 and pb5 is working
Impact
In that case no SPI communication can be established which is a show stopper on my board right now.
Environment (please complete the following information):

OS: Windows
Toolchain Zephyr SDK
Commit SHA 49d08c25

when remapping the SPI1 (REMAP_1), I suspect a pin conflict between SPI1 and DEBUG pins
PB3 and JTDO
PB4 and NJTRST
--> if SPI1 is remapped thenLL_GPIO_AF_DisableRemap_SWJ(); or or LL_GPIO_AF_Remap_SWJ_NOJTAG();

Thanks for your update. We had a further look into that and found the config option GPIO_STM32_SWJ_NOJTAG. After setting this option we were able to see the clock (so pin conflict seems the right idea). Unfortunately after that we were not able to debug with SWD - the program is getting stuck in the ARM reset (reset.S).

This is the same issue as #38354 and relates to the #38741 PR (regarding AFIO and SWJ).
Unfortunately, the issue doesn't seem to be resolved in the latest Zephyr.
Setting the CONFIG_GPIO_STM32_SWJ_NOJTAG  results in getting stuck within z_arm_reset. When SWJ_CFG is left unchanged, then PB3 can't be used as SPI 1 SCK pin.

Setting the CONFIG_GPIO_STM32_SWJ_NOJTAG results in getting stuck within z_arm_reset.

Do you get the same issue even when not using SPI1 at all (eg: hello_world)?

Setting the CONFIG_GPIO_STM32_SWJ_NOJTAG results in getting stuck within z_arm_reset.

Do you get the same issue even when not using SPI1 at all (eg: hello_world)?

yes

@GeorgeCGV Thanks for the investigation and finding the issue!
For the fix, I suggest to purely remove gpio_stm32_afio_init and move its logic directly in pinctrl_stm32.c (and pinmux_stm32.c while it's still there).

The PR is created. I can confirm that the SPI 1 SCK signal is present on PB3. Unfortunately, we still didn't manage to communicate with the other IC over the SPI 1 on F1VC (L5 manages to communicate with that IC, so it is not a custom driver issue), investigating further...
Some notes:
HAL and LL don't disable the LL_APB2_GRP1_PERIPH_AFIO clock.
There is a difference in the order of how SWJ and remap are being set between HAL and LL:

HAL

Enables the clock and disables JTAG in HAL_MspInit (HAL_Init)
Enables remap after GPIO initialization

LL

Enables the clock and disables JTAG
Reads MAPR and awaits AFIO_MAPR_SWJ_CFG_JTAGDISABLE to be set (SWJ_CFG part, it supposed to be write-only).
Enables remap before GPIO initialization

Everything works now.

What helped is the usage of  AFIO_MAPR_SWJ_CFG (111) during every remap setting and not disabling the AFIO clock after the SWJ initialization.
Unfortunately, it is unknown how setting the AFIO_MAPR_SWJ_CFG (111) helps. However, HAL and LL do the same.
For example:
__STATIC_INLINE void LL_GPIO_AF_EnableRemap_SPI3(void)
{
  SET_BIT(AFIO->MAPR, AFIO_MAPR_SPI3_REMAP | AFIO_MAPR_SWJ_CFG);
}

As for the initialization and the clock - now it is similar to the HAL. AFIO clock is enabled, then comes the SWJ_CFG application, then remap happens later on (if required).

purely remove gpio_stm32_afio_init

The SWJ initialization was moved into pinctrl and pinmux modules.
I don't think that we shall get rid of the SYS_INIT approach. There might be a case when no remap is used but someone needs some IO pin that is occupied by the JTAG or SWD in the project. Therefore, we have to provide a way to free the desired pin (e.g. PB4, PA15, etc...).
The PR also follows your suggestion regarding the DTS usage instead of Kconfig for the SWJ. That makes more sense, as it is a setting that is more or less F1 specific.


Bad interface on install

I'm trying to install codiad in the folder /codiad. I added the path to the /etc/apache2/apache2.conf file as below:
<Directory />
        Options FollowSymLinks
        AllowOverride None
        Require all denied
</Directory>

<Directory /usr/share>
        AllowOverride None
        Require all granted
</Directory>

<Directory /codiad/>
        Options Indexes FollowSymLinks MultiViews
        AllowOverride All
        Require all granted
</Directory>

<Directory /var/www/>
        Options Indexes FollowSymLinks
        AllowOverride None
        Require all granted
</Directory>

I tried to modify the absolute path in config.php to /codiad but it doesn't work. The only thing I get is this bad interface. Tried also with Firefox, same result but a couple of field appear (username and password I think). All permissions are ok, chown www-data:www-data and chmod 777, I really don't know what to do to solve this thing. Also, I restarted apache2 server everytime.
I modified /etc/apache2/mods-enabled/dir.conf file to:
<IfModule mod_dir.c>
        DirectoryIndex index.php index.html index.cgi index.pl index.xhtml index.htm
</IfModule>

(I put index.php before index.html)
And finally I also tried to modify DocumentRoot in /etc/apache2/sites-available/000-default.conf to:
DocumentRoot /codiad/
What's wrong? Thank you!

So are you saying that when you try to browse to /codiad that you just get the above screenshot of php code? If that's the case you should make sure that you have the Apache php module installed, you'll need that for the php code to actually be executed.
What OS are you on? If you're on Ubuntu, do this:
sudo apt install libapache2-mod-php
# Get PHP version:
php -v
sudo a2enmod php$PHP_VERSION
# e.g.
a2enmod php7.1
sudo service apache2 restart

You'll want to take great care to make sure that you're securing things properly while setting it up, chown-ing everything to 777 is bad security practice, I would advise that you make sure you're comfortable with a LAMP stack before you fully put it into production (or, at the very least the A and P part of it).
This app is pretty powerful and you want to make sure that no one besides the www-data user has access to the application files and that the app itself doesn't have more access than it needs to your projects. If you're deploying this to the open internet then enabling TLS with something like Let's Encrypt is a must.
These should help you:

https://www.rosehosting.com/blog/how-to-secure-your-lamp-server/
https://www.linode.com/docs/web-servers/apache-tips-and-tricks/apache-configuration-basics


New version: PolynomialTestSystems v0.1.3

Registering package: PolynomialTestSystems
Repository: https://github.com/JuliaHomotopyContinuation/PolynomialTestSystems.jl
Created by: @saschatimme
Version: v0.1.3
Commit: 870050a0388301fe74787d7eb0c5bbc335d9fc1e
Reviewed by: @saschatimme
Reference: https://github.com/JuliaHomotopyContinuation/PolynomialTestSystems.jl/commit/870050a0388301fe74787d7eb0c5bbc335d9fc1e#commitcomment-34039476

Can you also add compat for julia:
[compat]
julia = "1"

and retrigger the bot.

I added a compat statement for Julia

Why did you remove the other bounds?

I noticed that I also need to widen the DynamicPolynomials constraint and I was a little bit too lazy. Now I just widened that compact statement


Permutation Importances: Calculating multiple scores at once for better performance

Is it possible to calculate multiple scores without fitting PermutationImportance/calling get_score_importances multiple times? I am working a dataset with lots of features and want to try various scores. All the expensive computations should be shared.
I have already tried scoring=dict(score1=..., score2=...), but this does not seem to be supported. I am also not able to return a tuple of scores from get_score_importances.
A hack would be fine too. Any suggestions?

I have decided to implement my own permutation importance function. It include two optimizations:

It reduces the number of predict_proba(x) calls. This makes a big difference in my case (LightGBM RandomForest).
It allows to compute multiple scores at once (by passing a dict to scorers).

Maybe the code is useful for somebody else (no guarantees for correctness):
def fast_permutation_importance(model, x, y, scorers, n_iter=5, batch_thresh=100000):
  is_single = callable(scorers)
  if is_single:
    scorers = dict(single=scorers)

  x = np.array(x)
  n, n_cols = x.shape

  class Classifier:
    def __init__(self, proba):
      self.proba = proba
    def predict_proba(self, x):
      assert len(self.proba) == len(x)
      return self.proba

  base_results = {}
  proba = model.predict_proba(x)
  for scorer_name, scorer in scorers.items():
    base_results[scorer_name] = scorer(Classifier(proba), x, y)

  results = {scorer_name: np.zeros((n_cols, n_iter)) for scorer_name, scorer in scorers.items()}
  batch = []

  def run_batch():
    nonlocal batch
    x3 = np.concatenate([x2 for i, j, x2 in batch], axis=0)
    assert x3.shape[1] == n_cols
    proba = model.predict_proba(x3)

    start = 0
    for i, j, x2 in batch:
      for scorer_name, scorer in scorers.items():
        value = scorer(Classifier(proba[start:start + n]), x2, y)
        results[scorer_name][i][j] = value
        start += n
    batch = []

  for i in range(n_cols):
    for j in range(n_iter):
      x2 = np.copy(x)
      col = x2[:, i]
      np.random.shuffle(col)
      x2[:, i] = col
      batch.append((i, j, x2))
      if len(batch) * n >= batch_thresh:
        run_batch()
  run_batch()

  for scorer_name, result in results.items():
    results[scorer_name] = base_results[scorer_name] - np.mean(results[scorer_name], axis=1)
    assert len(results[scorer_name]) == n_cols

  if is_single:
    results = results["single"]
  return results


Deprecate containsExactly matcher

The generic array matcher with default configs does the same thing as the containsExactly matcher which should be deprecated.

@kapishmalik  please

@tommysitu raised PR for the same.


Pandas-like Apply() functionality for the db

Note, that unlike Pandas the apply() call does an in-place remplacement of the db-entries.
One could therefore consider to relabel the function to apply_inplace() instead.
Also the repair() and delete_if() calls are now powered by the apply() call, which improves their performance. The previous implementation copied the database twice, now data is only copied / rewritten once.

The test seems to contain an explicit call of the test function. Could you either remove it or guard it via __name__=="__main__" please?  Otherwise,  it may crashes the test environment in case in fails in the future.

 This is fixed now.

Thanks!

I believe that there could be a problem with 'apply' if the folder "old_db" exists by chance. Maybe it would be safer to use a unique folder name?

This is indeed a good idea! I've adapted the code, the directory name now includes both a timestamp (maybe useful for debugging) and a random number.

The current implementation of apply only takes care of the data part, but not of the fingerprints and the environment data. I added a new test case that shows the problematic behavior.

Thank you for pointing this out, I will work on a solution.

You will essentially have to move the old data base and then fill the now empty one anew. Maybe it could be useful to implement a 'move_and_clear' method for this.

The issue is fixed on this branch, however I realized there is a smarter way to implement this feature.
I will therefore close this request and create a new pull request.


Exporter writes a zero-length tangent vector

I'm attempting to update a few models in KhronosGroup/glTF-Sample-Models#130.  One of them, the DamagedHelmet model, exports from Blender and fails in the glTF Validator with a zero-length tangent vector.
Here's the Blender file:  DamagedHelmet.blend
You can export glTF or glb versions.  The validator fails both with this message:

#/accessors/3: Accessor element at index 11043 is not of unit length: 0.0.

Accessor 3 in this case is the TANGENT vector collection, and it appears that one of them has been exported with zero length.
/cc @UX3D-nopper @lexaknyazev

@emackey
I'm looking into this.

Error message should be adjusted a bit, but the exported model contains zero-length tangent vector
[0.0, 0.0, -0.0, 1.0].

I presume the official Khronos Blender exporter should be expected to not produce such vectors?  Or is this some problem with the model itself?  I tried re-generating the normal vectors to no avail, and Blender does not have a user-option to re-generate the tangent vectors.

Tangents come from Blender's engine:
https://github.com/KhronosGroup/glTF-Blender-Exporter/blob/5ffd6efa0bc1378fe681253ea397c3250b94f26c/scripts/addons/io_scene_gltf2/gltf2_extract.py#L372-L377
Exporter just adds the fourth component to them:
https://github.com/KhronosGroup/glTF-Blender-Exporter/blob/5ffd6efa0bc1378fe681253ea397c3250b94f26c/scripts/addons/io_scene_gltf2/gltf2_extract.py#L41-L45

I see.  What's the correct solution here?  Should the exporter be checking each one for zero length?  What should it do if it finds a zero-length tangent?

Ideally, I think we should identify what kind of geometry produces this issue.
For now, exporter could verify unit vectors for all-zeros and print something to console.

I dream of a day when VSCode shows you a mesh wireframe and points to the vertex that offended the validator...
But until then, I'll just hazard a guess.  It's possible that there are degenerate triangles in UV space that are not degenerate in XYZ space.  This isn't necessarily an error, but it results in some "stretched" imagery on the triangle in question and maybe makes the tangents impossible to calculate.  There's no perfect answer as to what to do with a normal map in this case, but it may well have been the artist's intention to stretch the base map across the triangle in this manner.

I dream of a day when VSCode shows you a mesh wireframe and points to the vertex that offended the validator...

What kind of validation output would be needed to implement this?

What kind of validation output would be needed to implement this?

Moving that topic to AnalyticalGraphicsInc/gltf-vscode#43.
Since this issue is in the Blender-Exporter repo, we'll keep it on the topic of how the exporter should react to Blender's calculation of zero-length tangents.

Added a warning output here:
https://github.com/KhronosGroup/glTF-Blender-Exporter/commit/0089f3198b74ffb9491a953873c1214c4b3cfe10
I close this ticket, as in this case a 3D artist has to fix the 3D model.


AttributeError:Can't pickle local object 'affine_augmentatio..augmentation'

How do you solve this problem: AttributeError:Can't pickle local object 'affine_augmentatio..augmentation' ?
I tried to modify the affine_augmentation in order to have 'augmentation' as a global function but it doesn't work anyway. Have you got the same problem? Thank you

Hello,
Could you please elaborate a bit what exactly would you like to achieve?
By the way - new version of WSI registration method is available at: https://github.com/MWod/DeeperHistReg <- it should be much easier to use than this one.
Bests,


include steps for tagging releases in the README

i'd consider incrementally bumping the changelog as each pull request lands using the [Unreleased] section ideal.
That said, there's nothing wrong with isolating the commit history since the last release and doing it all at once immediately before publishing either.
https://github.com/Esri/resource-proxy/compare/v1.1.2...HEAD

Looks clear.  I'll try to follow them for next release.


Running/Disabling test groups in should be supported in ballerina 1.0.0

Description:
--group and --disable-group flags have been removed from ballerina 1.0.0 which was available upto 0.991. These flags should be added back to make use of test groups.

Closing since its a duplicated issue. This would be tracked by #18772


Broken Builds Bonanza, Oct 2024

Mostly this is a POC, but here's the idea... many of our container builds are broken currently (https://github.com/elfhosted/containers/actions/runs/11496576681), partly because the servarr folks' API has changed / gone.
I'm trying to attach a fund to this issue, which (in theory) will let us offer a bounty to have 3rd party contributors apply a fix for us.

Upvote & Fund

We're using Polar.sh so you can upvote and help fund this issue.
We receive the funding once the issue is completed & confirmed by you.
Thank you in advance for helping prioritize & fund our backlog.

You can pledge behind and help support this effort using Polar.sh


add supports_webpack and run_js_devserver example (#125)

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Makes http::serve take a SockAddr instead of a starting

This is a pretty simple PR, but it was hurting my eyes seeing that much replication and weird variable naming with respect to the internal API.
In a nutshell, this simply makes http::serve take grpc_bind as SocketAddr instead of String, so we can get rid of a useless variable and rename things more accordingly.

Actually, we only needed to enclose once (i.e. we don't need to clone twice, the second, and last, time can be a move).

Removed the latest commit


Change phone number edits

Issue: Make sure the behavior of phone number inputs are consistent across the application and make it so that if a user has made changes to a phone number and the click off without saving the phone number returns to how it was originally.

Did:

Created two new functions in base.js: setupPhoneNumber and processPhoneSetup.
Changed the notification when the phone number is saved from a msgFlash to a msgToast.

Expected behavior when editing phone number:

On click of "Edit" or on focus of input: "Edit" will become "Save" and the phone input will be focused.
On click of "Save" with a VALID phone number: "Save" will become "Edit", the phone input will be un-focused, the new phone number will be saved, and a toast will appear in the bottom right confirming the new phone number has been saved.
On click of "Save" with an INVALID phone number: "Save" will stay "Save", the background of the phone input will turn red, and the phone input will stay focused.
On focus out of phone input without clicking "Save": "Save" will become "Edit" and the contents of the phone input will restore the phone number to its original state before the input was focused.

To test on User Profile:

Make sure that the edit button behaves as expected when you are focusing/un-focusing the phone input.
Delete part of the phone number then click out of the phone input without click the save button and make sure the phone number is restored to its origin state.
Delete part of the phone number  and attempt to save it. It should not be allowed to save and the phone input bg color should turn red while still being focused.
Change the phone number and save it. Make sure the new phone number is actually saved and the msgToast pops up confirming the save.

To test on SLC Instructor:

Edit an SLC proposal and repeat all testing actions that were done on the User Profile on the Instructors phone number.
Add a second SLC instructor to the instructor table and make sure the phone numbers still behave as expected.
Remove all instructors from the instructor table and then add one back and make sure the phone number still behaves as expected.

 Phone number 'Edit' link on User Profile not changing #689

Everything works as expected. However, the toast messages blend in with the page color, someone might not notice them right away especially since they are at the bottom corner. I think maybe changing the color of the pop-up to a light green might make them more visible.


cc_common.merge_cc_infos is missing

Relevant issue: https://github.com/bazelbuild/bazel/issues/20375


docs: Remove AUTHORS.txt (or make it up-to-date)

The intent to credit contributors is admirable but the file is outdated and does not really give us a lot of value over git shortlog -s.  I also do not think the file helps in tracking authorship since the file provides no information about what people authored.
Admittedly the file is better formatted than shortlog output... but if the file is not up to date then that does not help.

AUTHORS.txt (and the acknowledgement section in the main README) has a few names that aren't in git shortlog, because

their involvement predates the use of git for python-tuf development, or
they contributed to a TUF subproject other than python-tuf (e.g. notary, etc.)

I suggest to

update the acknowledgement section in README.md (or replace it with a dedicated acknowledgements document) that lists all significant and no longer active contributors and maintainers of python-tuf, which we should keep up-to-date (see #1793), optionally with their organization affiliation
maybe do something similar for the TUF organization (for the lack of a theupdateframework/community repo, it could be part of theupdateframework.io),
remove AUTHORS.txt in this repo

@JustinCappos, would you mind weighing in on what to do with the AUTHORS.txt document?

I am in favor of the approach you suggest @lukpueh .  Does anyone object?

No objections.
From an implementation perspective, reproducible-builds.org contributor page lists everyone who touched the repo plus explicitly named personnel. We might create a similar script for listing TUF contributors on theupdateframework.io? See https://salsa.debian.org/reproducible-builds/reproducible-website/-/blob/master/bin/contributors.sh


Fixes issue 5054

Fixes #5054
Added new with_revision parameter to both create_page and create_title.
By default this parameter is False.
Raises ImproperlyConfigured error if with_revision is True but reversion is not enabled.
@yakky @mkoistinen please review

LGTM
@evildmp I think we can safely add 'clickjacking' to dictionary (the only failing test)

LGTM


⚠ Jenkins Plugins Artifactory has degraded performance

In cf13cd0, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1084 ms

Resolved: Jenkins Plugins Artifactory performance has improved in 2804e5d after 25 minutes.


Make exercises and solutions for 03

Done.
See resources:

Exercise template notebook - https://github.com/mrdbourke/pytorch-deep-learning/blob/main/extras/exercises/03_pytorch_computer_vision_exercises.ipynb
Exercise solutions example - https://github.com/mrdbourke/pytorch-deep-learning/blob/main/extras/solutions/03_pytorch_computer_vision_exercise_solutions.ipynb
Video walkthrough live-coding the exercises/solutions - https://youtu.be/_PibmqpEyhA


implement AutoCloseable for actor apis

Task

Make the generated actor api classes implement AutoCloseable
Should be done by making the super class (ActorAPI) implement the interface

Motivation
This should improve safety, letting the user and compiler know the actor is a resource that should be closed once it's not needed anymore
Notes

should not be implemented before #29 is implemented

Not possible since the close function has the suspend modifier.


Migration to 1.4 from 1.3.x doesn't preserve the current root correctly.

Steps to duplicate:

fresh dcos cluster
install 1.3.6
launch task (example below)
after it is running, destroy the app
uninstall marathon 1.3.6
install 1.4.0-snap30
(The 3 tasks that were removed "destroyed" in marathon 1.3.6 will start running again)

{
      "id": "test3",
      "instances":  3,
      "cmd": "for (( ; ; )); do sleep 100000000; done",
      "cpus": 0.01,
      "mem": 1,
      "disk": 0
    }

Additional note:  The tasks will be started up and will be running when viewed from dcos or mesos... however they do NOT show up in the list of apps in v2/apps as they are under a different framework_id.

upon uninstallation of marathon 1.3.6, are the tasks actually destroyed? or do they keep running in the background? I would expect uninstall to effectively terminate (on mesos) any existing tasks associated with 1.3.6. if that's not happening, and these tasks just keep running, then mesos is probably waiting for some scheduler failover period before killing the executors running the task

I'll run through the scenario again... the tasks were KILLED from a mesos standpoint  from what I remember.

I see the issue, prepping a fix.

currently suspect a bug in migration:
(1) migration report indicates that only root+apps have been migrated (counts for other objects are all zero)
(2) mesos and marathon logs indicate that marathon-user (MoM) tasks are KILLED and removed prior to framework shutdown
@kensipe has mentioned that some tasks show as "orphaned" via mesos /state. Not sure what's happening there. Possibly a bug in reconciliation, but needs further investigation.

xref #4581

Should be fixed with fix for #4824. Will test this now


[TableCell] No "scope" attribute added for type "head"

Fixes #10269.

@z-ax I had fixed the TypeScript build. Let me know if the pull-request still address your need.

@z-ax It's a great first pull request 👌🏻. Thank you for giving it a shot!


Develop new SWPC use case using_gen_vx_mask

Describe the New Feature
We need to develop a new use case for the SWPC project that illustrates the new gen_vx_mask wrapper capability that was developed in #387 and #424. The goal will be to have a use case that masks by solar azimuthal angle and/or time of day.
Acceptance Testing
Input: GloTEC netCDF files assimilated with and without COSMIC-1 Radio Occultation data
Test: Generate a time-varying mask for each model time level
Time Estimate
2-3 days
Relevant Deadlines
No hard deadline. A good goal would be by April 21.
Funding Source
2770471 - SWPC
Define the Metadata
Assignee

[ ] Select engineer(s) or no engineer required
[x] Select scientist(s) or no scientist required

Labels

[x] Select component(s)
[x] Select priority
[x] Select requestor(s)

Projects and Milestone

[x] Review projects and select relevant Repository and Organization ones
[x] Select milestone

Define Related Issue(s)
Consider the impact to the other METplus components.

[x] METplus, MET, METdb, METviewer, METexpress, METcalcpy, METplotpy

New Feature Checklist
See the METplus Workflow for details.

[x] Complete the issue definition above.
[x] Fork this repository or create a branch of develop.
Branch name: feature_<Issue Number>_<Description>
[ ] Complete the development and test your changes.
[ ] Add/update unit tests.
[ ] Add/update documentation.
[ ] Push local changes to GitHub.
[ ] Submit a pull request to merge into develop, listing the <Issue Number> in the title.
[ ] Iterate until the reviewer(s) accept and merge your changes.
[ ] Delete your fork or branch.
[ ] Close this issue.

I made some good progress on this use case (METplus #427).

I have prototyped the use case -- I am now debugging it. Currently, it doesn’t doesn’t seem to be reading the input file, even though I think the path is now correct. I’m guessing this might be because the GloTEC data are not fully CF-compliant.
The next step is to probably schedule a one-on-one with George.

I've completed development. Am awaiting George's review.

Awesome. Thanks
On Tue, May 26, 2020, 6:17 PM Jonathan Vigh<EMAIL_ADDRESS>wrote:

I've completed development. Am awaiting George's review.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/NCAR/METplus/issues/427#issuecomment-634347808, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AFQ7BGYVYQH34EFSWDUL3ILRTRLYFANCNFSM4MHPAW4A
.

Pull request approved, new use case is now being run in automated tests.


Incompatibility with nbsphinx.

Hi there, lovely theme!
Unfortunately it doesn't seem to work nicely with nbsphinx, a tool to include Jupyter Notebooks in Sphinx generated documentation. Specifically, <pre> elements (products of Python warnings) don't overflow nicely.

Fixed by adding the following CSS:
pre {
white-space: pre-wrap;
}

I'll try to make a PR for this.

This was addressed with #10 - but i'll leave this open until I create a new release containing the fix. :slightly_smiling_face:

This has been released with 1.2.0.


[BUG] Gears with high pressure angle self-intersect

Describe the bug
Gears with high pressure angles produce self-intersecting geometry. The implementation code appears to assume the pressure angle is low enough that the involute curves do not intersect each other before reaching the root radius.
An example of gears with these high pressure angles is the popular bearing at https://www.thingiverse.com/thing:53451 from 2013 by emmett.
Code To Reproduce Bug
include <BOSL2/std.scad>
include <BOSL2/gears.scad>

spur_gear(teeth=4,pressure_angle=60);

Expected behavior
Self intersections can produce problems with rendering and use. It is much better to have a final shape that is clearly closed.
Screenshots

Additional context
Add any other context about the problem here.

OpenSCAD Version: This is a build of git commit 3241bf88b from 2023-01-04 . I'll be pulling a new version today, but it will take me a while to build and use it.
Other libraries used: BOSL2 0f09d1fac976099f824e4fce705362ebe240b161 .

This problem has been solved a handful of ways elsewhere. I handwrote some gear code in C++ once that used the inverse of the involute function to calculate the intersection point here. Because this inverse is hard to calculate I attach this code:
involute_inverse.cpp . I am still new to OpenSCAD.

Note, the OpenSCAD version won't make a difference.    (You can download prebuilt nightly builds unless you have special needs.)
We are absolutely NOT experts on gears, so insight you have on how to improve the gear implementation is welcome.    One of our main open issues is that worm gears self-intersect.   I didn't even know anybody ever used pressure angles above 20 degrees.
The OpenSCAD language is quirky, to say the least, but we can likely assist you if you want to make specific changes, for example.  Revar wrote the core of the gear code and he's not very available these days.   It looks like it's got a bunch of linear interpolation inverse lookups.   Perhaps it can/should be completely rewritten?   (This is the function _gear_tooth_profile() I'm referring to.)
Testing suggests that pressure angles above about 33 fail.  Does it suffice to terminate the involute curve at a higher point, or are there other complications at the high pressure angles?
That inverse involute seems rather complicated indeed.  It looks like a many term series approximation to the inverse function.  I suspect that if we only need a single inverse we may find it adequate to use root_find(), our generic root finder, but I suppose the explicit series could be more robust.

f=function (angle) tan(angle) - angle*PI/180;

function inv_f(val) = root_find(function(x) f(x)-val, 0, 89.99);

for(u=[5:5:89])
{
  fu = f(u);
  rt=inv_f(fu);
  echo(u, fu, rt, rt-u);
}

Above runs the same test as your C++ code.  Error is higher mostly because we don't have long doubles, I think.  Translation of the other code to OpenSCAD if I didn't botch it:
  function involute_to_pressure(angle) =
    angle==0 ? 0
  :  let(
        inv = abs(angle)<=1.8 ?
                  let(
                        angle1_3 = angle^(1/3),
                        angvec = [angle1_3, angle, angle1_3^5, angle1_3^7, angle1_3^9, angle1_3^11, angle1_3^13, angle1_3^15],
                        coef = [3^(1/3), -2/5, 9*9^(1/3)/175, -2*3^(1/3)/175, -144/67375, 3258*9^(1/3)/3128125, -49711*3^(1/3)/153278125, -1130112/9306171875]
                  )
                  coef*angvec
            : abs(angle)<5 ?
                  let(
                       zeta = angle - (tan(5*PI/12*180/PI) - 5*PI/12),
                       zetavec = [1, zeta, zeta^2, zeta^3, zeta^4, zeta^5, zeta^6],
                       coef = [5*PI/12, 7 - 4*sqrt(3), - (388 - 224*sqrt(3)), 41784 - 24124*sqrt(3),-(5623484 - 3246720*sqrt(3)), 847344640 - 489214656*sqrt(3),
                                -<PHONE_NUMBER>16/3 -<PHONE_NUMBER>44*sqrt(3)/3)]
                  )
                  coef*zetavec
            : let(
                   angvec = [1, 1/angle, 1/angle^2, 1/angle^3, 1/angle^4, 1/angle^5],
                   coef = [PI/2, -1, PI/2, -(PI^2/4+2/3), PI^3/8+PI, -(PI^4/16+PI^2+13/15)]
              )
              coef*angvec,
        f = function (inv) let(tan_inv=tan(inv*180/PI)) inv+(angle - (tan_inv - inv))/(tan_inv * tan_inv)
    )
    f(f(f(f(inv))));

Error slightly better than root_find().


Continuation indents in KDoc

Continuation indents in KDoc block tags are currently removed.  Is this a specific design decision?
/**
 * Summary fragment.
 *
 * @param blockTag This is the first
 * block tag.
 */

If the block tag wraps onto a new line, ktfmt will not use any indentation, and remove 1- or 2-space indentation that already exists.  Indents of 3 or more spaces will be automatically maintained with ``` marks.
The Google Style Guide for Java recommends at least 4 spaces indent (consistent with the continuation indent).  I would propose that ktfmt likewise use the continuation indent here.

It's probably a bug. Thanks for reporting! we'll take a look.

I believe this issue could be closed. Ktfmt currently inserts correct continuation indents in KDoc @property and @param tags, at least in my current testing using the IntelliJ plugin.

Thanks for catching that, @koljakube. Closing the issue


Broken pipe error in Dataloader

@lordmartian That change resolved the error, however, I had set numworkers in the config file to 0 in order to avoid a mutliprocess error that was masking the original error. Setting the numworkers to anything other than 0 still causes this error.
File "", line 1, in
File "D:\Users\arunm\anaconda3\lib\multiprocessing\spawn.py", line 105, in spawn_main
exitcode = _main(fd)
File "D:\Users\arunm\anaconda3\lib\multiprocessing\spawn.py", line 114, in _main
prepare(preparation_data)
File "D:\Users\arunm\anaconda3\lib\multiprocessing\spawn.py", line 225, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "D:\Users\arunm\anaconda3\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
run_name="mp_main")
File "D:\Users\arunm\anaconda3\lib\runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "D:\Users\arunm\anaconda3\lib\runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "D:\Users\arunm\anaconda3\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "D:\Users\arunm\PycharmProjects\AV_Speech_Recognition\audio_visual\pretrain.py", line 106, in
trainingLoss, trainingCER, trainingWER = train(model, pretrainLoader, optimizer, loss_function, device, trainParams)
File "D:\Users\arunm\PycharmProjects\AV_Speech_Recognition\audio_visual\utils\general.py", line 39, in train
for batch, (inputBatch, targetBatch, inputLenBatch, targetLenBatch) in enumerate(tqdm(trainLoader, leave=False, desc="Train", ncols=75)):
File "D:\Users\arunm\anaconda3\lib\site-packages\tqdm\std.py", line 1107, in iter
for obj in iterable:
File "D:\Users\arunm\anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 278, in iter
return _MultiProcessingDataLoaderIter(self)
File "D:\Users\arunm\anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 682, in init
w.start()
File "D:\Users\arunm\anaconda3\lib\multiprocessing\process.py", line 112, in start
self._popen = self._Popen(self)
File "D:\Users\arunm\anaconda3\lib\multiprocessing\context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "D:\Users\arunm\anaconda3\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "D:\Users\arunm\anaconda3\lib\multiprocessing\popen_spawn_win32.py", line 46, in init
prep_data = spawn.get_preparation_data(process_obj._name)
File "D:\Users\arunm\anaconda3\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
Traceback (most recent call last):
File "D:/Users/arunm/PycharmProjects/AV_Speech_Recognition/audio_visual/pretrain.py", line 106, in
_check_not_importing_main()
File "D:\Users\arunm\anaconda3\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
trainingLoss, trainingCER, trainingWER = train(model, pretrainLoader, optimizer, loss_function, device, trainParams)
File "D:\Users\arunm\PycharmProjects\AV_Speech_Recognition\audio_visual\utils\general.py", line 39, in train
for batch, (inputBatch, targetBatch, inputLenBatch, targetLenBatch) in enumerate(tqdm(trainLoader, leave=False, desc="Train", ncols=75)):
File "D:\Users\arunm\anaconda3\lib\site-packages\tqdm\std.py", line 1107, in iter
is not going to be frozen to produce an executable.''')
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.
for obj in iterable:

File "D:\Users\arunm\anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 278, in iter
return _MultiProcessingDataLoaderIter(self)
File "D:\Users\arunm\anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 682, in init
w.start()
File "D:\Users\arunm\anaconda3\lib\multiprocessing\process.py", line 112, in start
self._popen = self._Popen(self)
File "D:\Users\arunm\anaconda3\lib\multiprocessing\context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "D:\Users\arunm\anaconda3\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "D:\Users\arunm\anaconda3\lib\multiprocessing\popen_spawn_win32.py", line 89, in init
reduction.dump(process_obj, to_child)
File "D:\Users\arunm\anaconda3\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
BrokenPipeError: [Errno 32] Broken pipe
Originally posted by @arunm95 in https://github.com/lordmartian/deep_avsr/issues/9#issuecomment-691497097

If you executed the code with num_workers=0 and if you didn't get any error, then that means the code is correct. If you search up this error on the internet, there are many posts related to it. According to these posts, this issue appears when running on Windows. One solution that seems to work for the majority of the people is to use if __name__ == "main" in the code. I do not have access to any Windows machine with GPU. Thus even if I make the suggested changes, I will not be able to try them out. I am open for any PRs which resolve this issue and are tested on Windows machines.

Confirming that this workaround does work. Wrapping the entire code of the pretrain.py under if name == 'main' resolves the issue.

Thanks. I'll make these changes in the next commit.


Scrolling with hidden columns

reported by @kakatoh

First I tried: columns: { hidden: [0,1,2,3,4,5,,,,,90] } as in
http://mafft.cbrc.jp/alignment/server/spool/question11.html
With this setting, I partly got an expected result (columns 0-90 are indeed hidden), but the width of alignment screen became strangely narrow and the residue indices do not move with scrolling.

well that one is more complicated, if someone wants to have a try at this: the MSAViewer already calculates the number of hidden columns to its right, s.t. it can correctly render the sequences, but there seems something wrong.
A good place to start would be this line:
https://github.com/wilzbach/msa/blob/master/src/views/canvas/CanvasSeqBlock.js#L349
(with Chrome you can directly browse all bundled files and set breakpoints)


Windows install broken

Wheels for the latest version, 0.7.10, are not available on PyPI, and local setup.py install fails (at least for Python 3.4) while trying to compile the roof duality extension.
Error trace available on a dwave-hybrid AppVeyor build.

Yes, I had to revert back to 0.7.9 because of that.


fixed effects in formula?

Thank you for this great package!
Is there a way to include a fixed effect term in utility?
In my application, I have neighborhood fixed effects, and there are many possibilities (>50) so I would prefer to not add them in the formula by hand.
Is there a way to construct the utility formula programmatically, i.e. in FixedEffectModels it's possible to do sum([term(Symbol("somevar" * string(i) for i=1:N]).

A note that I can't figure out how to pass an expression to the @utility macro, and I think that doesn't work with macros in Julia (as they only have access to the "text" not the contents).
I am currently "hacking" this by

redefining function utility(ex::Expr) instead of a macro
defining some utility formula myu programmatically
using eval(utility(myu)) as an argument of multinomial_logit
I think this is probably terrible, but it works for me... for now.

There's currently no way to do this, and I think you're correct about the macros.

Thinking about this more, I think there are two ways to go about this:

As you say, the macro doesn't have access to the data. But what the macro is generating is actually code that is executed at runtime, so that could theoretically have access to the data and create the fixed effects that way.
You could change the multinomial logit code to handle fixed effects directly.

@Gkreindler I ran into this in a project I'm working on and kinda-sorta worked around it by using == in the utility function, e.g.
        "carpool" ~
            αcarpool_1990 * (year == 1990) +
            αcarpool_2000 * (year == 2000) +
            αcarpool_2009 * (year == 2009) +
            αcarpool_2014 * (year == 2014) +
            αcarpool_2019 * (year == 2019) +
            αcarpool_2021 * (year == 2021)

It's still rather awkward, and may have performance implications for things that aren't numbers. One option would be to add a macro @fe that basically just generates this code right inside the utility function—so you could write
"carpool" ~ @fe(αcarpool, year, [1990, 2000, 2009, 2014, 2019, 2021])

and have it expanded out like this. You would still have to specify the levels manually, but that's a lot less painful than the current situation.

Thanks! For my application, I ended up coding by hand the MLE and bootstrap SEs for the simple case I had (binary logit) + analytic gradients.


Fix issue with multiple software types being registered but not all are referenced

In the first iteration of software types, there was a defect where multiple registered software type plugins could be applied to a project even if they were not all referenced in the declarative dsl.  This fixes the issue so that unreferenced software types are not applied to the project.
Reviewing cheatsheet
Before merging the PR, comments starting with

❌ ❓must be fixed
🤔 💅 should be fixed
💭 may be fixed
🎉 celebrate happy things

@bot-gradle test this

I've triggered the following builds for you. Click here to see all build failures.

PullRequestFeedback build

@bot-gradle test this

I've triggered the following builds for you. Click here to see all build failures.

PullRequestFeedback build

Good idea.  Thanks for the quick review @tresat!


Bootstrap detects wrong version number

Platform
Which platform does the bug apply to?

WebExtension (Google Chrome, Mozilla Firefox)

Describe the bug
Bootstrap shows the wrong version-number
To Reproduce
Steps to reproduce the behavior:

Go to 'https://github.com/'
Check the Wappalyzer predictions for Bootstrap detection
See error

Expected behavior
Should show no version number
Additional context
Also on this site 'https://arastta.org/get-involved''

Looking at the rule, it's kind of finicky but there isn't really any other option. You're more likely to get an actual version of Bootstrap because most people likely use CDN-linked assets (like BootstrapCDN), and the main rare case would be if the assets were custom compiled with an arbitrary hash, which then shows up as the version number (see screenshot below).
So it would either be, accept the small risk of a wrong version number, or remove the version completely - and in this case, I think it would be acceptable to leave it as-is.

If you look at https://www.booker.senate.gov, you'll see Wappalyzer identify Bootstrap 4.7.0, which doesn't exist. Investigating the source, it looks like it's using 4.3.1.
It's possible that it's picking up the version from the Bootstrap CDN link to Font Awesome: https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css
(Apologies if this is hijacking this issue, but I thought it may be related. I can submit this as a new bug following the format if that's helpful.)

(Apologies if this is hijacking this issue, but I thought it may be related. I can submit this as a new bug following the format if that's helpful.)

Not at all, this is exactly the issue. In this case, the regex is looking for a CSS file that contains the word Bootstrap, then is preceded with a multi-digit number of some kind. (https://regexr.com/5m4pn)
"<link[^>]* href=[^>]*?bootstrap(?:[^>]*?([0-9a-fA-F]{7,40}|[\\d]+(?:.[\\d]+(?:.[\\d]+)?)?)|)[^>]*?(?:\\.min)?\\.css\\;version:\\1"

It might be best to isolate CDN provided Bootstrap libraries to reduce any errors like this (BootstrapCDN, cdnjs, jsdelivery, etc), but would also be open to other suggestions.

That sounds like a good approach to me. An alternative could be: set the regex to ignore http/https protocol (if there, because that can be optional) and domain name up until the first '/', then check for matches in the rest of the file path. In other words, ignore any part of the URL that's between "//" and the first "/". That might complicate the regex a bit more, but it would get around specifying CDNs domain names.


Unable to crop while downloading raw VODs.

Why not tell Twitch Leecher to move the .ts file from the temp folder to the destination?

RTFM. The tooltip in the preferences explains exactly that. cropping is not available when downloading ts files. ffmpeg is used to crop. when downloading raw... no ffmpeg... no cropping.

FFMpeg isn't even open until converting the combined .ts files. I even deleted FFMpeg and it still merges when "cropped" (which only tells it which .ts files to download, barring cut bookends, if any.) though deletes the merged file right after.  BTW, it has trouble with a streamer's dropped frames. Might want to check that out.

LUL... you smartass really understood how TL works... NOT. Internet smartasses are welcomed to the blocked users list... so.... WELCOME 😄


fix: backoffice localized fields show correct validation errors

Fixing bugs with localized attributes at backoffice. These attributes now properly show:

if it is required
validation error if such error was raised during update

Testing instructions
Backoffice...
Tracking
BugFix ;)

Ahh, yes I remember now, why I did not use I18n.with_locale when fetching object and just showing inputs for normal attributes. Because of default fallback, is that right?

Yes, that is why I still need to use helper at views

Ahh, yes I remember now, why I did not use I18n.with_locale when fetching object and just showing inputs for normal attributes. Because of default fallback, is that right?

Yes, that is why I still need to use helper at views

Other problem is that content language uses language from db as default one so you cannot actually fetch object for some language when you need its language to decide its content language :). But workaround with that new helper method should work ;)


WinGet manifest Keyboard Shortcuts extension

Summary of the Pull Request
Work for Shortcut Guide v2

PR Checklist

[ ] Closes: #xxx
[ ] Communication: I've discussed this with core contributors already. If work hasn't been agreed, this work might be rejected
[ ] Tests: Added/updated and all pass
[ ] Localization: All end user facing strings can be localized
[ ] Dev docs: Added/updated
[ ] New binaries: Added on the required places

[ ] JSON for signing for new binaries
[ ] WXS for installer for new binaries and localization folder
[ ] YML for CI pipeline for new test projects
[ ] YML for signed pipeline

[ ] Documentation updated: If checked, please file a pull request on our docs repo and link it here: #xxx

Detailed Description of the Pull Request / Additional comments

Validation Steps Performed

This is ready for review now


Make the indentation in comprehensions configurable

Some people prefer this:
void testDestructureComprehensions() {
    value v = {
        for ([a, b, c] in [[1, 2.0, '3']])
        for ([Integer x, Float y, Character z] in [[a, b, c]])
        for ([init, *others] in [[0, *(1..runtime.maxIntegerValue)]])
        for ([String first, String *rest] in [["executableName", *process.arguments]])
        for (m->n in [true->"true"])
        for (Boolean o -> String p in [m->n])
        if (exists [ai, bi, ci] = { [1, 2.0, '3'] }.cycled.rest.first)
        if (exists [Integer xi, Float yi, Character zi] = { [ai, bi, ci] }.cycled.rest.first)
        if (exists [initi, *othersi] = [0, *(1..runtime.maxIntegerValue)])
        if (exists [String firsti, String *resti] = ["executableName", *process.arguments])
        if (exists mi->ni = { true->"true" }.cycled.rest.first)
        if (exists Boolean oi -> String pi = { mi->ni }.cycled.rest.first)
        nothing
    };
}

over this:
void testDestructureComprehensions() {
    value v = {
        for ([a, b, c] in [[1, 2.0, '3']])
            for ([Integer x, Float y, Character z] in [[a, b, c]])
                for ([init, *others] in [[0, *(1..runtime.maxIntegerValue)]])
                    for ([String first, String *rest] in [["executableName", *process.arguments]])
                        for (m->n in [true->"true"])
                            for (Boolean o -> String p in [m->n])
                                if (exists [ai, bi, ci] = { [1, 2.0, '3'] }.cycled.rest.first)
                                    if (exists [Integer xi, Float yi, Character zi] = { [ai, bi, ci] }.cycled.rest.first)
                                        if (exists [initi, *othersi] = [0, *(1..runtime.maxIntegerValue)])
                                            if (exists [String firsti, String *resti] = ["executableName", *process.arguments])
                                                if (exists mi->ni = { true->"true" }.cycled.rest.first)
                                                    if (exists Boolean oi -> String pi = { mi->ni }.cycled.rest.first)
                                                        nothing
    };
}

It would be nice if the indentation was configurable.

Should this be an Integer comprehensionIndentationLevels (default 1, @bjansen 0) or a Boolean indentComprehensions (default true, @bjansen false)? Integer is more general, but I assume almost nobody would use any values but 0 and 1, and a Boolean is friendlier to the IDE settings dialog (checkbox instead of number entry field).

I vote in favor of a Boolean, people are not crazy enough to want double indents for this ;)

Boolean | Integer indentationEnabledOrSpaces :)
I've seen 1-space indenting on gitter.

🙄

They should add that emoji to the standard list

I got it from macOS' emoji keyboard! http://emojipedia.org/face-with-rolling-eyes/

I really don’t want to support indentation independent of indentMode… let’s keep it to levels, at least.


Add tag based sensor filter

We have had support for tagging activities with sensors for a while, but we never connected it to the search.  This PR finally connects it. Previously the sensor search was based on the outdated probe models, that was removed and never replaced with the tag based approach.
This also replaces all references of 'probe' with 'sensor' in this part of the code. This will make it easier to remove all of the older probe code in a future PR.
The database schema was not changed in this PR. Two probe fields saved into Solr were removed. The sensors tag field was added a while ago, so it will not be necessary to reset Solr after this is released.
PT story: https://www.pivotaltracker.com/story/show/158574697

There is at least one failing test here, so it isn't mergeable yet...

OK the test are passing now. The jasmine tests are failing randomly as usual.


docker: build and push image

[x] Bug fix #3118
[x] New feature?
[ ] BC breaks?
[ ] Docs added?

I target 6.x so that 6.x and 7.x (later) could have Docker releases.
I tested it in another project: I created a 0.0.3 tag here https://github.com/alexislefebvre/docker-images/tree/0.0.3 GitHub Actions built an image and pushed it there: https://github.com/alexislefebvre/docker-images/pkgs/container/php-8.1-jakzal-phpqa-gd
Once this will work, people will be able to run something like  docker run ghcr.io/deployphp/deployer:v6.9.1 dep, use the image on their CI system, etc.
:warning: I don't know if it may allow a contributor to push bad code to a Docker image, sorry.
If you merge it and create a tag, it may fail if the registry is not active yet, IIRC we had the issue too and we had to change the visibility of our GitHub registry to private to public before we could push images.
To test the image: checkout this branch and run docker build:
docker build . --tag deployer --build-arg VERSION=v6.9.1
docker run deployer

Real test on my fork:

workflow: https://github.com/alexislefebvre/deployer/actions/runs/4179168257/jobs/7238799360 (it will expire at some point)
resulting image: https://github.com/alexislefebvre/deployer/pkgs/container/deployer
it works!

$ docker run -t --rm ghcr.io/alexislefebvre/deployer:v6.9.3 help
Description:
  Display help for a command

Usage:
  help [options] [--] [<command_name>]

[…]

Awesome cool. Will review and if not merge definitely take an inspiration from it)

FYI, this will build multiarch images for different php/deployer version combinations:

https://github.com/ochorocho/deployer/blob/main/Dockerfile
https://github.com/ochorocho/deployer/blob/main/.github/workflows/build.yml

@antonmedv Could you please check this PR?
@ochorocho Thanks for sharing it, I think we can merge this PR as is then add support of other architectures in another PR.

Please let me know if you intend to merge this PR at some point, otherwise we can close it.

I'm going to adapt this PR to implement docker image

It is done! https://hub.docker.com/r/deployphp/deployer


Link errors on OS X

I get following output when compiling the code on OS X, it seems that some link errors occurred:
➜  rebel git:(master) ✗ make
mkdir -p build && cd build && cmake ../csrc/liars_dice -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=../cfvpy && make -j
-- Found PythonInterp: /usr/local/miniconda3/bin/python (found version "3.7.2")
-- Found PythonInterp: /usr/local/miniconda3/bin/python (found suitable version "3.7.2", minimum required is "3.7")
-- pybind11 v2.3.dev1
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/user/rebel/build
make[1]: Entering directory '/Users/user/rebel/build'
make[2]: Entering directory '/Users/user/rebel/build'
make[3]: Entering directory '/Users/user/rebel/build'
make[3]: Entering directory '/Users/user/rebel/build'
make[3]: Entering directory '/Users/user/rebel/build'
make[3]: Leaving directory '/Users/user/rebel/build'
make[3]: Leaving directory '/Users/user/rebel/build'
make[3]: Leaving directory '/Users/user/rebel/build'
[  7%] Built target gtest
make[3]: Entering directory '/Users/user/rebel/build'
[ 15%] Built target _rela
[ 38%] Built target liars_dice_lib
make[3]: Entering directory '/Users/user/rebel/build'
make[3]: Entering directory '/Users/user/rebel/build'
make[3]: Entering directory '/Users/user/rebel/build'
make[3]: Leaving directory '/Users/user/rebel/build'
make[3]: Leaving directory '/Users/user/rebel/build'
make[3]: Leaving directory '/Users/user/rebel/build'
[ 46%] Built target gtest_main
make[3]: Entering directory '/Users/user/rebel/build'
make[3]: Entering directory '/Users/user/rebel/build'
make[3]: Entering directory '/Users/user/rebel/build'
make[3]: Entering directory '/Users/user/rebel/build'
[ 50%] Linking CXX executable recursive_eval
make[3]: Entering directory '/Users/user/rebel/build'
make[3]: Leaving directory '/Users/user/rebel/build'
make[3]: Leaving directory '/Users/user/rebel/build'
make[3]: Leaving directory '/Users/user/rebel/build'
make[3]: Entering directory '/Users/user/rebel/build'
make[3]: Leaving directory '/Users/user/rebel/build'
[ 53%] Linking CXX shared module /Users/user/rebel/cfvpy/rela.cpython-37m-darwin.so
make[3]: Leaving directory '/Users/user/rebel/build'
make[3]: Entering directory '/Users/user/rebel/build'
make[3]: Entering directory '/Users/user/rebel/build'
make[3]: Entering directory '/Users/user/rebel/build'
make[3]: Entering directory '/Users/user/rebel/build'
[ 57%] Linking CXX executable liar_tree_test
[ 61%] Linking CXX executable liar_recursive_solving_test
make[3]: Entering directory '/Users/user/rebel/build'
[ 65%] Linking CXX executable liar_game_test
[ 69%] Linking CXX executable liar_subgame_solving_test
[ 73%] Linking CXX executable gen_benchmark
ld: in section __TEXT,__text reloc 11: length != 2 and X86_64_RELOC_GOT not supported file 'CMakeFiles/liar_game_test.dir/liars_dice_test.cc.o' for architecture x86_64
ld: in section __TEXT,__text reloc 3: length != 2 and X86_64_RELOC_GOT not supported file 'CMakeFiles/liar_recursive_solving_test.dir/recursive_solving_test.cc.o' for architecture x86_64
ld: in section __TEXT,__text reloc 7: length != 2 and X86_64_RELOC_GOT not supported file 'CMakeFiles/liar_subgame_solving_test.dir/subgame_solving_test.cc.o' for architecture x86_64
ld: in section __TEXT,__text reloc 11: length != 2 and X86_64_RELOC_GOT not supported file 'CMakeFiles/liar_tree_test.dir/liars_dice_test.cc.o' for architecture x86_64
ld: in section __TEXT,__text reloc 24: length != 2 and X86_64_RELOC_GOT not supported file 'CMakeFiles/gen_benchmark.dir/gen_benchmark.cc.o' for architecture x86_64
ld: in section __TEXT,__text reloc 119: length != 2 and X86_64_RELOC_GOT not supported file 'CMakeFiles/recursive_eval.dir/recursive_eval.cc.o' for architecture x86_64
clang: clang: fatal error: linker command failed with exit code 1 (use -v to see invocation)fatal error: linker command failed with exit code 1 (use -v to see invocation)
clang: fatal error: linker command failed with exit code 1 (use -v to see invocation)
clang: fatal error: linker command failed with exit code 1 (use -v to see invocation)
clang: fatal error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [CMakeFiles/liar_game_test.dir/build.make:89: liar_game_test] Error 1
make[3]: Leaving directory '/Users/user/rebel/build'
make[3]: *** [CMakeFiles/gen_benchmark.dir/build.make:88: gen_benchmark] Error 1
make[3]: Leaving directory '/Users/user/rebel/build'
make[3]: *** [CMakeFiles/liar_tree_test.dir/build.make:89: liar_tree_test] Error 1
make[3]: Leaving directory '/Users/user/rebel/build'
make[2]: *** [CMakeFiles/Makefile2:228: CMakeFiles/gen_benchmark.dir/all] Error 2
make[2]: *** Waiting for unfinished jobs....
make[2]: *** [CMakeFiles/Makefile2:151: CMakeFiles/liar_game_test.dir/all] Error 2
make[2]: *** [CMakeFiles/Makefile2:190: CMakeFiles/liar_tree_test.dir/all] Error 2
make[3]: *** [CMakeFiles/liar_recursive_solving_test.dir/build.make:89: liar_recursive_solving_test] Error 1
make[3]: Leaving directory '/Users/user/rebel/build'
make[3]: *** [CMakeFiles/liar_subgame_solving_test.dir/build.make:89: liar_subgame_solving_test] Error 1
make[3]: Leaving directory '/Users/user/rebel/build'
make[2]: *** [CMakeFiles/Makefile2:379: CMakeFiles/liar_recursive_solving_test.dir/all] Error 2
make[2]: *** [CMakeFiles/Makefile2:112: CMakeFiles/liar_subgame_solving_test.dir/all] Error 2
clang: fatal error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [CMakeFiles/recursive_eval.dir/build.make:87: recursive_eval] Error 1
make[3]: Leaving directory '/Users/user/rebel/build'
make[2]: *** [CMakeFiles/Makefile2:73: CMakeFiles/recursive_eval.dir/all] Error 2
ld: in libliars_dice_lib.a(subgame_solving.cc.o), in section __TEXT,__text reloc 7: length != 2 and X86_64_RELOC_GOT not supported for architecture x86_64
clang: fatal error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [CMakeFiles/rela.dir/build.make:88: /Users/user/rebel/cfvpy/rela.cpython-37m-darwin.so] Error 1
make[3]: Leaving directory '/Users/user/rebel/build'
make[2]: *** [CMakeFiles/Makefile2:340: CMakeFiles/rela.dir/all] Error 2
make[2]: Leaving directory '/Users/user/rebel/build'
make[1]: *** [Makefile:141: all] Error 2
make[1]: Leaving directory '/Users/user/rebel/build'
make: *** [compile] Error 2

Sorry, we only support linux systems.

@akhti
The C codes also work on Windows. I have tested  it with Visual Studio 2017.


GNOME 43 "The extension is incompatible with the current GNOME version" fix.

When updating to gnome 43, gnome's extension manager displays The extension is incompatible with the current GNOME version. To fix this, navigate to<EMAIL_ADDRESS>and add 43 in the list of shell versions.
"shell-version": [
    "43"
  ],

This fixes the compatibility issue and the extension works as intended.

For me it was<EMAIL_ADDRESS>

Yeah, we just need an update where the extension is marked as 43-compatible

Merged #78


Pull request

Minor layout changes in the fieldset template.

Uhm it breaks the layout of the test project, screenshot here:

I've already picked changes of d90a0b3.

Closing this then.


Fix bucket_name bucketname consistency

Separating out into smaller PRs  :smile_cat:
Refactor for consistency see: https://github.com/coralnet/pyspacer/blob/cb0b427b03f60bb92e2e9bdf1ccbb64e5b2230c3/spacer/storage.py#L187
https://github.com/coralnet/pyspacer/blob/cb0b427b03f60bb92e2e9bdf1ccbb64e5b2230c3/spacer/storage.py#L205
https://github.com/coralnet/pyspacer/blob/cb0b427b03f60bb92e2e9bdf1ccbb64e5b2230c3/spacer/messages.py#L25

Additionally a new change from #85 ircc will also need to be changed  https://github.com/coralnet/pyspacer/blob/478a8f6f01d7bdc1cb7b40866d36d89b53dda314/spacer/messages.py#L71

Thanks Lauren! This'll make reviewing easier.
I'm definitely onboard with making this param name consistent, but:

It looks like there are still a decent amount of bucket_name usages remaining: for example in messages.py, storage.py, tests/test_storage.py, tests/utils.py, and README.md.
Pardon my pickiness, but unless there's another reason I've overlooked (e.g. consistency with boto), I think I'd actually favor bucket_name over bucketname. The style goes better with the storage_type param in DataLocation.__init__() and storage_factory(); and originally there were more usages of bucket_name than bucketname (around 50 vs. 10).

@StephenChan I agree that makes more sense - the latest push should address those changes !