Disallow adding new class attributes unless using the existing built-ins

The idea is to block the direct assignment of new class attributes in the test class. The user should always go through the parameter and variable built-ins to add new class attributes. Though, nothing changes at the instance level, where injecting attributes there should still be allowed.
class MyTest(rfm.RegressionTest):
    p = parameter(range(10))
    v = variable(int, value=10)
    a = 10 # ERROR: Use variable or parameter to declare a new class attribute.

    @rfm.run_after('setup')
    def add_attr(self):
        # Nothing changes at the instance level.
        # Adding a new attribute in the standard Python way is allowed.
        self.a = 10

It turns out that this would overly complicate the implementation and it would also clash with the way Python assigns the class-bound functions and other internals.
Not worth the trouble.


Errors after upgrading jar from 3.0.8 to 3.2.3

Getting the following errors after upgrading .jar from 3.0.8 to 3.2.3.  The DB was upgraded to 3.2.0 Postgres.  The UI /jobs is populated but not /clusters or /applications.
Below is entry from the error logs
" WARN 45466 --- [nio-8080-exec-8] o.h.engine.jdbc.spi.SqlExceptionHelper   : SQL Error: 0, SQLState: 22003
2017-11-02 19:22:56.199 ERROR 45466 --- [nio-8080-exec-8] o.h.engine.jdbc.spi.SqlExceptionHelper   : Bad value for type long : Analytics Runtime Framework SIT/Andromeda Submit Command
2017-11-02 19:22:56.206 ERROR 45466 --- [nio-8080-exec-8] c.n.g.w.c.GenieExceptionMapper           : Failed to execute data service method
"
looks like some data type mismatch between existing data and Spring Hibernate.
Any advice would be appreciated!

Hey @kirubel To verify your upgrade process you were on the postgres schema from 3.0.1?
And then you ran the following files to upgrade:
3.0.1 to 3.1.0
3.1.x to 3.2.0
Is this true?

can you run pg_dump on your database with the following flags:
pg_dump --no-tablespaces --no-owner --no-privileges --schema-only and diff it to the expected schema and post any results?
I believe @mprimi tested all this as he was doing the 3.2.x changes but we may have missed something as Postgresql isn't our current production DB

Here are some difference

Dumped from database version 9.6.2

-- Dumped by pg_dump version 9.6.5	6.	-- Dumped by pg_dump version 9.6.4
(output from my env was 9.6.2 for server and 9.6.4 for client node

The following lines were present in my env but missing from the expected schema.
1st diff**************

"--
283.	-- Name: schema_version; Type: TABLE; Schema: public; Owner: -
284.	--
285.
286.	CREATE TABLE schema_version (
287.	    version_rank integer NOT NULL,
288.	    installed_rank integer NOT NULL,
289.	    version character varying(50) NOT NULL,
290.	    description character varying(200) NOT NULL,
291.	    type character varying(20) NOT NULL,
292.	    script character varying(1000) NOT NULL,
293.	    checksum integer,
294.	    installed_by character varying(100) NOT NULL,
295.	    installed_on timestamp without time zone DEFAULT now() NOT NULL,
296.	    execution_time integer NOT NULL,
297.	    success boolean NOT NULL"
2nd************************
-- Name: schema_version schema_version_pk; Type: CONSTRAINT; Schema: public; Owner: -
431.	--
432.
433.	ALTER TABLE ONLY schema_version
434.	    ADD CONSTRAINT schema_version_pk PRIMARY KEY (version);
435.
436.
437.	--
3rd diff*****
-- Name: schema_version_ir_idx; Type: INDEX; Schema: public; Owner: -
684.	--
685.
686.	CREATE INDEX schema_version_ir_idx ON schema_version USING btree (installed_rank);
687.
688.
689.	--
690.	-- Name: schema_version_s_idx; Type: INDEX; Schema: public; Owner: -
691.	--
692.
693.	CREATE INDEX schema_version_s_idx ON schema_version USING btree (success);
694.
695.
696.	--
697.	-- Name: schema_version_vr_idx; Type: INDEX; Schema: public; Owner: -
698.	--
699.
700.	CREATE INDEX schema_version_vr_idx ON schema_version USING btree (version_rank);
701.
702.
703.	--
5th diff*********
These lines ",
240.	    configs text DEFAULT ''::character varying NOT NULL"
came after the line "memory integer,

   applications character varying(2048) DEFAULT '[]'::character varying NOT NULL,	238.	    applications character varying(2048) DEFAULT '[]'::character varying NOT NULL,"

but in the schema you provided the line "configs text DEFAULT ''::character varying NOT NULL," comes after the line "dependencies text DEFAULT ''::character varying NOT NULL,"

Hello @kirubel, let me try to:

Run all integration tests against PG with a fresh 3.2.0 database
Retry the upgrade from 3.0.x to 3.2.x schema.

I'll get back to you in a bit.

@mprimi Thank you!

@kirubel I tried to perform all upgrades from 3.0.x to 3.2.x and compare against fresh created schemas. (e.g. compare 3.1.x upgraded to 3.2.x against fresh 3.2.x).
Dumps attached: schema-dumps.zip
The only difference I'm seeing is order the configs column in the job_requests table in 3.1.x (fresh vs upgraded from 3.0.x) which you mentioned as difference number 5.
But the order of that column should not create any issue.
Can you post any more info you have, such as the full exception trace, raw database schema dumps, etc?

@kirubel Still trying to reproduce your problem, but actually bumped into something interesting.
The schema migration from 3.0.1 -> 3.1.0 -> 3.2.0 seems to work if the database is empty (as per previous comment).
However if the database is not empty it seems to fail. I can reproduce as follows:

Create a genie database with 3.0.1 schema using PostgreSQL 9.6
Start an instance of Genie server configure to use the above database
use the API to create some clusters/apps/commands, schedule a couple of jobs to check everything is ok
Shut down the server
Perform 3.0.1 -> 3.1.0 schema migration
Perform 3.1.0 -> 3.2.0 schema migration

The last step fails:
ERROR:  multiple primary keys for table "application_configs" are not allowed
SQL state: 42P16

Is it possible this step failed for you too, and the errors you're seeing now are result of this botched migration?

After manually fixing the upgrade script (not 100% sure this is the proper way), I can perform the 3.1.0 -> 3.2.0 upgrade.
However once I start the 3.2.x server, there are migration error from Flyway, and the data in the database is wiped.
Not great, will have to see about this with @tgianos.

The database was showing blank when using v3.2.3 but all data was present again when we reverted back the jar back to v3.0.8.  Maybe the DB never upgraded due to error you mentioned.  I will double check if the dB got upgraded or not and update you.  Thanks !
Sent from my iPhone

On Nov 9, 2017, at 6:53 PM, Marco Primi<EMAIL_ADDRESS>wrote:
After manually fixing the upgrade script (not 100% sure this is the proper way), I can perform the 3.1.0 -> 3.2.0 upgrade.
However once I start the 3.2.x server, there are migration error from Flyway, and the data in the database is wiped.
Not great, will have to see about this with @tgianos.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

I think the problem is the following:
In 3.1 we moved some fields from varchar to text (https://github.com/Netflix/genie/pull/558).
Some field entities were marked @Lob to reflect this.
With HSQL and MySQL this seems to work ok.
With PostgreSQL/Hibernate however, the connector expects a long (a pointer to a blob containing the data) and it finds some text instead, so it errors out.
In more detail: https://shred.zone/cilla/page/299/string-lobs-on-postgresql-with-hibernate-36.html
We're going to patch this.

Oh okay! Thank you!
Sent from my iPhone

On Nov 10, 2017, at 6:57 PM, Marco Primi<EMAIL_ADDRESS>wrote:
I think the problem is the following:
In 3.1 we moved some fields from varchar to text (#558).
Some field entities were marked @Lob to reflect this.
With HSQL and MySQL this seems to work ok.
With PostgreSQL/Hibernate however, the connector expects a long (a pointer to a blob containing the data) and it finds some text instead, so it errors out.
In more detail: https://shred.zone/cilla/page/299/string-lobs-on-postgresql-with-hibernate-36.html
We're going to patch this.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@kirubel I just released 3.1.3 which should fix this problem, could you confirm by doing the following:

Start with your 3.0.1 schema with data
Run the database upgrade to 3.1.0
Run server version 3.1.3

The App/Cluster/Commands page should no longer show the error you initially reported.
Before you can upgrade to 3.2.x, we need to release 3.2.5 which I hope to do soon. However if in the meantime you could test the fix is working for 3.1.x, that would be great.
These are the relevant changes:
3.1.x: https://github.com/Netflix/genie/pull/641
3.2.x: https://github.com/Netflix/genie/pull/642
More testing: https://github.com/Netflix/genie/pull/639

@kirubel after confirming 3.0.x to 3.1.x is working, you can now update to 3.2.x (3.2.5 or newer).
Notice that the 3.1 to 3.2 PostgreSQL upgrade script is different from the one you previously applied, make sure to use the latest.
Let us know if you run into any trouble.

Thank you very much.  I will work on applying the new updates and will let
you know of the results!
On Wed, Nov 15, 2017 at 7:14 PM, Marco Primi<EMAIL_ADDRESS>wrote:

@kirubel https://github.com/kirubel after confirming 3.0.x to 3.1.x is
working, you can now update to 3.2.x (3.2.5 or newer).
Notice that the 3.1 to 3.2 PostgreSQL upgrade script
https://github.com/Netflix/genie/blob/3.2.x/genie-ddl/src/main/sql/postgresql/upgrade-3.1.x-to-3.2.0.postgresql.sql
is different from the one you previously applied, make sure to use the
latest.
Let us know if you run into any trouble.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Netflix/genie/issues/636#issuecomment-344772600, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AGTgB-JKzLji-A3OTQDQdglBD30yMzpdks5s235mgaJpZM4QTy3B
.

@kirubel have you have any problems? We'd like to close this issue if it is resolved

hi Tom, apologies, we haven't been able to retest the issue in the
environment due to scheduled releases that have been going on the past few
weeks.  But I will update you as soon as we test it.  If you need to close
it now, maybe we can reopen it later if the problem persists.
On Mon, Dec 11, 2017 at 3:35 PM, Tom Gianos<EMAIL_ADDRESS>wrote:

@kirubel https://github.com/kirubel have you have any problems? We'd
like to close this issue if it is resolved
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Netflix/genie/issues/636#issuecomment-350850069, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AGTgByk2gIeLeHchvvfUeY1hiASMfV6hks5s_ZIlgaJpZM4QTy3B
.


UnsupportedOperationException("use the array(String name) form");

Hi,
Found that using the method .array() on a PactDslJsonArray object resulted in the above error code. I have an array
[{"id":1057,"name":"Spiderman","number":1,"secret_identity":"Peter Parker"},{"id":1058,"name":"Silk","number":2,"secret_identity":"Cindy Moon"},{"id":1059,"name":"DeadPool","number":3,"secret_identity":"Wade Wilson"},{"id":1060,"name":"Black Cat","number":4,"secret_identity":"Felecia Hardy"}]
that has no name, yet this is unsupported. This is also valid JSON (as far as i can tell).
Instead, it looks like i need to use .array(""), so a little counter intuitive.
Just wondering how i should proceed?

This is solved by this ticket:
https://github.com/DiUS/pact-jvm/issues/240
I'll update the docs for this.

Sorry, but I subsequently found the Documentation for the Array at root level. I'll close this.


Add Achievements (SUGGESTION)

Is there a way down the road to make logic/frontend so one can create/edit achievements on the fly which could be integrated into a existing admin panel or something. That way staff of site could add new Achievements from site and not have to use terminal and edit code in like sublime text?
I know this is long shot and probably not easy todo with how the Achievements system works now but thought i would throw the idea out there as it would make it more user friendly.

There's not actually a problem with adding achievements by the staff via an admin panel, but you'd need to have a developer change the source code anyway in order to create the unlock condition. Without changing the source, you'd only have an achievement which nobody can actually progress on, or unlock.

makes sense. Didn't think about the unlock condition part! I hope you don't mind all the questions and suggestions some valid and some not. Just like this package and have been trying to get better and better at laravel for awhile now

Keep making them! Questions and suggestions will only make the package better. :D

Aside from the fact that you would have to implement it in the code, I'm curious how you would go about creating achievements from an admin panel? Would you just execute the artisan command from your controller to generate the new files?

Yes, I would create Achievements from an admin panel by running the artisan command from the controller.
In case you need to edit them (for example, changing the name, description or points required to unlock the Achievement) all you need to do is alter the values in the achievement_details table, since they take precedence over the values declared on the Achievement class.


Is there any typescript definition file?

Hi,
We're using this SDK in a typescript project,
is there any type files available? Or do we need to create one ourselves?
Thanks!

Hello @sallar
there is no typescript definition for this project.


Example to add

Hello,
First, thanks for your work on this library.
I spent sometime trying to figure out how to get responses from multi_curl to use for later processing. Figured it out and thought other couple benefit from this example:
use Curl\MultiCurl;

// urls array
$urls = ["https://www.example1.com",
        "https://www.example1.com",
        "https://www.example1.com"];

// array to hold responses
$responses = [];

$multi_curl = new MultiCurl();
$multi_curl->setProxies(PRIVATE_PROXIES["ips"]);
$multi_curl->setProxyAuth(PRIVATE_PROXIES["creds"]);
$multi_curl->setProxyTunnel();
foreach ($urls as $url) {
    $multi_curl->addGet($url);
}
$multi_curl->complete(function ($instance) use (&$responses) {
    $responses[] = $instance->response;
});
$multi_curl->start();

$responses; // this is an array of response from multi_curl to use

Thanks!

Is there any guarantee for the order of the $responses?
On a related note, addDownload() takes a callback. Why doesn't addGet() also take one?

Is there any guarantee for the order of the $responses?

No. There's $instance->id available to help identify.

On a related note, addDownload() takes a callback. Why doesn't addGet() also take one?

MultiCurl::addDownload is different from all the other request types (MultiCurl::addGet, MultiCurl::addPost, etc.).


Ats2 releasenotes donotpublishbefore19/10/2017

Email Arjan when reviewed.

The release of ATS 2.0 is postponed. There is no definitief new date yet.

Hi @Adam-Dupaski,
It is the "ats-and-ci-cd-2" how-to which is currently in PR 795. Can I just add the links? (Since it will break the build.)

@zwartd Please make a note to add links to that how-to as well as to Reference Guide 2 once those are published. I cannot merge PR 795 yet, as it needs a thorough review (based on a quick look, I see several types of unnecessary changes). I will merge this PR now.


How to determin my pos mining is working in command line

hello ,
The generatepos just print []
thanks.

The command ,’generatepos‘, is mining one block.when it return null ,it's represent for mining failed.


ClusterMembership listen method should return observable of cluster update events

Subscribe to change notifications of the cluster membership The events to track the life-cycle of members are:
ADDED- A new member has joined the cluster and its status has been changed to Up.
LEAVING - A member is leaving the cluster and its status has been changed to Exiting Note that the node might already have been shutdown when this event is published on another node.
LEFT - Member completely removed from the cluster.
UNREACHABLE- A member is considered as unreachable, detected by the failure detector of at least one other node.
REACHABLE- A member is considered as reachable again, after having been unreachable. All nodes that previously detected it as unreachable has detected it as reachable again.
Improve ClusterMembership API. Listen updates should return Observable of events and event should include information about event type: ADDED, LEFT, UNREACHABLE, REACHABLE, and
ClusterMember.
Observable<ClusterMemberEvent> listenUpdates();

Where
class ClusterMemberEvent{
   private EventType type;
   private ClusterMember member;
}

Done at PR: https://github.com/scalecube/scalecube/pull/88


Add support for word-level audio transcription timestamp granularity

Describe the change
Add support for word-level audio transcription timestamp granularity.
Provide OpenAI documentation link

timestamp_granularities[]
words

Describe your solution
Added AudioRequest.TimestampGranularities and AudioResponse.Words fields.
Tests
Filled AudioRequest.TimestampGranularities field in the existing tests' audio requests.
Additional context

Related to two-months-abandoned #673, self-closed #680, and self-closed #674.
Also removed some obsolete comments on AudioRequest fields.

I would have proposes separating transcription and translation request/response types, but that would have been a breaking change.

Thank you for working on this!

@agcom the PR looks good to me in its current state!

I would have proposed separating transcription and translation request/response types, but that would have been a breaking change.

We can discuss it in a separate PR, also totally open to look at a sketch of how this might look like 🙌🏻

@sashabaranov alright then, ready to review. I just wanted to test it out in our development server before marking it for review (it works fine).

We can discuss it in a separate PR, also totally open to look at a sketch of how this might look like 🙌🏻

May god bless me with more code refactoring tasks so I would work on this :smile:.


mullvad 2021.1 cannot connect to server

Describe the bug
After upgrading mullvad to 2021.1 it can't connect to the servers properly.
To Reproduce
Configure mullvad service:
  networking.wireguard.enable = true; # also doesn't work with this disabled
  services.mullvad-vpn.enable = true;

Run mullvad-vpn GUI and try to connect to a WireGuard or OpenVPN server.
For WireGuard it'll get stuck in the connecting state.
For OpenVPN it will show that it's connected in the GUI, but there's no actual internet connection.
Expected behavior
The connection should be established successfully.
Notify maintainers
@Br1ght0ne @ymarkus @xfix
Metadata
 - system: `"x86_64-linux"`
 - host os: `Linux 5.10.11, NixOS, 21.05.20210217.6b1057b (Okapi)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.4pre20201205_a5d85d0`
 - channels(root): `"nixos-21.03pre246062.420f89ceb26"`
 - channels(felschr): `"nixpkgs-unstable-20.09pre239318.c59ea8b8a0e"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Maintainer information:
# a list of nixpkgs attributes affected by the problem
attribute: mullvad-vpn
# a list of nixos modules affected by the problem
module: services.mullvad-vpn

mullvad-daemon log
[2021-02-18 13:45:43.956][mullvad_daemon::version][INFO] Starting mullvad-daemon - 2021.1 2021-02-10
[2021-02-18 13:45:43.957][mullvad_daemon][INFO] Logging to /var/log/mullvad-vpn
[2021-02-18 13:45:43.957][mullvad_daemon][INFO] Management interface listening on /var/run/mullvad-vpn
[2021-02-18 13:45:44.015][mullvad_daemon::relays][INFO] Initialized with 764 cached relays from 2021-02-18 13:08:45.363
[2021-02-18 13:45:44.015][mullvad_daemon::settings][INFO] Loading settings from /etc/mullvad-vpn/settings.json
[2021-02-18 13:45:44.015][mullvad_daemon::account_history][INFO] Opening account history file in /etc/mullvad-vpn/account-history.json
[2021-02-18 13:45:44.033][talpid_core::firewall][INFO] Resetting firewall policy
[2021-02-18 13:45:44.033][mullvad_daemon][INFO] Account already has wireguard key, starting key rotation.
[2021-02-18 13:45:44.315][mullvad_daemon::relays][INFO] Downloaded relay inventory has 764 relays
[2021-02-18 13:45:54.475][mullvad_daemon::relays][INFO] Selected relay de9-wireguard at <IP_ADDRESS>
[2021-02-18 13:45:54.476][talpid_core::firewall][INFO] Applying firewall policy: Connecting to <IP_ADDRESS>:31328 over UDP with gateways <IP_ADDRESS>,fc00:bbbb:bbbb:bb01::1, Allowing LAN
[2021-02-18 13:46:09.560][talpid_core::tunnel::wireguard][WARN] Timeout while checking tunnel connection
[2021-02-18 13:46:09.636][talpid_core::tunnel_state_machine::connecting_state][INFO] Tunnel closed. Reconnecting, attempt 1.
[2021-02-18 13:46:09.637][talpid_core::routing::imp::imp][ERROR] Failed to remove route - ::/0 via dev wg-mullvad table<PHONE_NUMBER> - Netlink error
[2021-02-18 13:46:09.637][mullvad_daemon::relays][INFO] Selected relay de17-wireguard at <IP_ADDRESS>
[2021-02-18 13:46:09.637][talpid_core::firewall][INFO] Applying firewall policy: Connecting to <IP_ADDRESS>:53 over UDP with gateways <IP_ADDRESS>,fc00:bbbb:bbbb:bb01::1, Allowing LAN

Is this still the case for 2021.2? Then I'll have a look at it.

I believe this is a bug with the daemon itself. I was wondering if @felschr could share more of their config to see how to trigger the issue. A minimal config is not able to reproduce the issue.

Seems like NixOS's firewall is interfering with our netfilter rules. And even if I force the firewall to use a netfilter comptabile iptables, it's still broken (but OpenVPN connects). We changed the way tunnel traffic is routed to rely on some netfilter rules to allow for split tunelling. The long and short of it is, I'll try and see if there's anything we can do daemon side to circumvent this issue, possibly adding iptables if iptables exist. Otherwise, you'll have to change the mullvad-vpn service to add permissive firewall rules to NixOS's firewall, and maybe even apply the nftables compatible iptable package.

I'm having a similar issue with 2021.2 2021-02-18, my full config is available here: https://github.com/shazow/nixfiles/blob/1fa955ea1fcb8510561c85feda2993b6647e1056/common/desktop.nix#L98
Some networking-related configs live here: https://github.com/shazow/nixfiles/blob/1fa955ea1fcb8510561c85feda2993b6647e1056/shazowic-corvus.nix
I thought it was related to having tailscale enabled (both being wireguard) but removing tailscale didn't seem to help. I also tried switching from networking.firewall to networking.nftables but that didn't seem to help.
journalctl _SYSTEMD_UNIT=mullvad-daemon.service
Mar 05 10:56:23 [mullvad_daemon::version][INFO] Starting mullvad-daemon - 2021.2 2021-02-18
Mar 05 10:56:23 [mullvad_daemon][INFO] Logging to /var/log/mullvad-vpn
Mar 05 10:56:23 [mullvad_daemon::rpc_uniqueness_check][DEBUG] Failed to locate/connect to another daemon instance, assuming there isn't one
Mar 05 10:56:23 [mullvad_daemon][INFO] Management interface listening on /var/run/mullvad-vpn
Mar 05 10:56:23 [mullvad_rpc::address_cache][DEBUG] Loading API addresses from "/var/cache/mullvad-vpn/api-ip-address.txt"
Mar 05 10:56:23 [mullvad_rpc::address_cache][DEBUG] Using API address: <IP_ADDRESS>:443
Mar 05 10:56:23 [mullvad_daemon::relays][DEBUG] Reading relays from /var/cache/mullvad-vpn/relays.json
Mar 05 10:56:23 [mullvad_daemon::relays][DEBUG] Reading relays from /nix/store/fxbhjwsdhgn4cz0a26i2p86zl1dqh9fs-mullvad-vpn-2021.2/share/mullvad/resources/r>
Mar 05 10:56:23 [mullvad_daemon::relays][INFO] Initialized with 761 cached relays from 2021-03-05 09:53:44.716
Mar 05 10:56:23 [mullvad_daemon::settings][INFO] Loading settings from /etc/mullvad-vpn/settings.json
Mar 05 10:56:23 [mullvad_daemon::version_check][DEBUG] Loading version check cache from /var/cache/mullvad-vpn/version-info.json
Mar 05 10:56:23 [mullvad_daemon::account_history][INFO] Opening account history file in /etc/mullvad-vpn/account-history.json
Mar 05 10:56:23 [mullvad_daemon][DEBUG] No cached target state to load
Mar 05 10:56:23 [talpid_core::firewall][INFO] Resetting firewall policy
Mar 05 10:56:23 [talpid_core::firewall::imp][DEBUG] Removing table and chain from netfilter
Mar 05 10:56:23 [mullvad_daemon][INFO] Account already has wireguard key, starting key rotation.
Mar 05 10:56:23 [mullvad_daemon::wireguard][DEBUG] Starting automatic key rotation job
Mar 05 10:56:23 [talpid_core::firewall][INFO] Resetting firewall policy
Mar 05 10:56:23 [talpid_core::firewall::imp][DEBUG] Removing table and chain from netfilter
Mar 05 10:56:23 [mullvad_rpc::rest][ERROR] HTTP request failed: Hyper error, using address <IP_ADDRESS>:443. Trying next API address: <IP_ADDRESS>:444
Mar 05 10:56:23 [mullvad_daemon::relays][ERROR] Failed to fetch new relay list: Hyper error. Will retry in 59 minutes
Mar 05 11:01:24 [mullvad_daemon::version_check][DEBUG] Writing version check cache to /var/cache/mullvad-vpn/version-info.json
Mar 05 11:01:24 [mullvad_daemon::management_interface][DEBUG] Broadcasting new app version info
Mar 05 11:11:24 [mullvad_rpc::rest][DEBUG] Fetched new API addresses [<IP_ADDRESS>:443, <IP_ADDRESS>:444], will fetch again in 24 hours
Mar 05 11:56:25 [mullvad_daemon::relays][DEBUG] Writing relays cache to /var/cache/mullvad-vpn/relays.json
Mar 05 11:56:25 [mullvad_daemon::relays][INFO] Downloaded relay inventory has 761 relays
Mar 05 11:56:25 [mullvad_daemon::management_interface][DEBUG] Broadcasting new relay list
Mar 05 12:44:05 [mullvad_daemon::management_interface][DEBUG] get_account_history
Mar 05 12:44:05 [mullvad_daemon::management_interface][DEBUG] get_tunnel_state
Mar 05 12:44:05 [mullvad_daemon::management_interface][DEBUG] get_settings
Mar 05 12:44:05 [mullvad_daemon::management_interface][DEBUG] get_current_location
Mar 05 12:44:05 [mullvad_daemon::management_interface][DEBUG] get_relay_locations
Mar 05 12:44:05 [mullvad_daemon::management_interface][DEBUG] get_wireguard_key
Mar 05 12:44:05 [mullvad_daemon::management_interface][DEBUG] get_account_data
Mar 05 12:44:05 [mullvad_daemon::management_interface][DEBUG] get_account_history
Mar 05 12:44:05 [mullvad_daemon::management_interface][DEBUG] get_current_version
Mar 05 12:44:05 [mullvad_daemon::management_interface][DEBUG] get_version_info
Mar 05 12:44:05 [mullvad_daemon::geoip][DEBUG] Error: Unable to fetch IPv6 GeoIP location
Mar 05 12:44:05 Caused by: Hyper error
Mar 05 12:44:05 Caused by: error trying to connect: tcp connect error: Network is unreachable (os error 101)
Mar 05 12:44:05 Caused by: tcp connect error: Network is unreachable (os error 101)
Mar 05 12:44:05 Caused by: Network is unreachable (os error 101)
Mar 05 12:44:11 [mullvad_daemon::management_interface][DEBUG] connect_tunnel
Mar 05 12:44:11 [mullvad_daemon][DEBUG] Target state Unsecured => Secured
Mar 05 12:44:11 [mullvad_daemon::relays][DEBUG] Selecting among 53 relays with combined weight 53000
Mar 05 12:44:11 [mullvad_daemon::relays][INFO] Selected relay us105-wireguard at <IP_ADDRESS>
Mar 05 12:44:11 [mullvad_daemon::relays][DEBUG] Relay matched on highest preference for retry attempt 0
Mar 05 12:44:11 [talpid_core::firewall][INFO] Applying firewall policy: Connecting to <IP_ADDRESS>:39140 over UDP with gateways <IP_ADDRESS>,fc00:bbbb:bbbb:b>
Mar 05 12:44:11 [talpid_dbus::network_manager][DEBUG] /etc/resolv.conf differs from reference resolv.conf, therefore NM is not managing DNS
Mar 05 12:44:11 [talpid_core::tunnel::wireguard][DEBUG] Using kernel WireGuard implementation
Mar 05 12:44:11 [talpid_core::routing::imp::imp][DEBUG] Adding routes: {RequiredRoute { prefix: V4(Ipv4Network { addr: <IP_ADDRESS>, prefix: 0 }), node: RealNode>
Mar 05 12:44:11 [mullvad_daemon][DEBUG] New tunnel state: Connecting { endpoint: TunnelEndpoint { endpoint: Endpoint { address: <IP_ADDRESS>:39140, protoc>
Mar 05 12:44:26 [talpid_core::tunnel::wireguard][WARN] Timeout while checking tunnel connection
Mar 05 12:44:26 [talpid_core::tunnel_state_machine::connecting_state][DEBUG] WireGuard tunnel timed out
Mar 05 12:44:26 [talpid_core::tunnel_state_machine::connecting_state][DEBUG] The tunnel disconnected unexpectedly
Mar 05 12:44:26 [talpid_core::tunnel_state_machine::connecting_state][DEBUG] Tunnel monitor exited with block reason: None
Mar 05 12:44:26 [talpid_core::routing::imp::imp][DEBUG] Clearing routes
Mar 05 12:44:26 [talpid_core::routing::imp::imp][ERROR] Failed to remove route - ::/0 via dev wg-mullvad table<PHONE_NUMBER> - Netlink error
Mar 05 12:44:26 [mullvad_daemon][DEBUG] New tunnel state: Disconnecting(Reconnect)
Mar 05 12:44:26 [mullvad_daemon::relays][DEBUG] Selecting among 53 relays with combined weight 53000
Mar 05 12:44:26 [mullvad_daemon::relays][INFO] Selected relay us88-wireguard at <IP_ADDRESS>
Mar 05 12:44:26 [mullvad_daemon::relays][DEBUG] Relay matched on highest preference for retry attempt 1
Mar 05 12:44:26 [talpid_core::firewall][INFO] Applying firewall policy: Connecting to <IP_ADDRESS>:53 over UDP with gateways <IP_ADDRESS>,fc00:bbbb:bbbb:bb01:>
Mar 05 12:44:26 [talpid_dbus::network_manager][DEBUG] /etc/resolv.conf differs from reference resolv.conf, therefore NM is not managing DNS
Mar 05 12:44:26 [talpid_core::tunnel::wireguard][DEBUG] Using kernel WireGuard implementation
Mar 05 12:44:26 [talpid_core::routing::imp::imp][DEBUG] Adding routes: {RequiredRoute { prefix: V4(Ipv4Network { addr: <IP_ADDRESS>, prefix: 0 }), node: RealNode>
Mar 05 12:44:26 [mullvad_daemon][DEBUG] New tunnel state: Connecting { endpoint: TunnelEndpoint { endpoint: Endpoint { address: <IP_ADDRESS>:53, protocol: >
Mar 05 12:44:41 [talpid_core::tunnel::wireguard][WARN] Timeout while checking tunnel connection
Mar 05 12:44:42 [talpid_core::tunnel_state_machine::connecting_state][DEBUG] WireGuard tunnel timed out
Mar 05 12:44:42 [talpid_core::tunnel_state_machine::connecting_state][DEBUG] Tunnel monitor exited with block reason: None
Mar 05 12:44:42 [talpid_core::tunnel_state_machine::connecting_state][DEBUG] The tunnel disconnected unexpectedly
Mar 05 12:44:42 [talpid_core::routing::imp::imp][DEBUG] Clearing routes
Mar 05 12:44:42 [talpid_core::routing::imp::imp][ERROR] Failed to remove route - ::/0 via dev wg-mullvad table<PHONE_NUMBER> - Netlink error
Mar 05 12:44:42 [mullvad_daemon][DEBUG] New tunnel state: Disconnecting(Reconnect)
Mar 05 12:44:42 [mullvad_daemon::relays][DEBUG] Selecting among 20 relays with combined weight 18200
Mar 05 12:44:42 [mullvad_daemon::relays][INFO] Selected relay us-nyc-219 at <IP_ADDRESS>
Mar 05 12:44:42 [mullvad_daemon::relays][DEBUG] Relay matched on highest preference for retry attempt 2
Mar 05 12:44:42 [talpid_core::firewall][INFO] Applying firewall policy: Connecting to <IP_ADDRESS>:1196 over UDP with gateways , Allowing LAN
Mar 05 12:44:42 [talpid_core::tunnel::openvpn][DEBUG] Writing credentials to /tmp/ad14b204-7cbd-4184-8a6d-34d5520df83f
Mar 05 12:44:42 [talpid_core::process::openvpn][DEBUG] Building expression: /nix/store/fxbhjwsdhgn4cz0a26i2p86zl1dqh9fs-mullvad-vpn-2021.2/share/mullvad/res>
Mar 05 12:44:42 [mullvad_daemon][DEBUG] New tunnel state: Connecting { endpoint: TunnelEndpoint { endpoint: Endpoint { address: <IP_ADDRESS>:1196, protoco>
Mar 05 12:44:44 [talpid_core::tunnel::openvpn::event_server][INFO] OpenVPN event Request { metadata: MetadataMap { headers: {"te": "trailers", "content-type>
Mar 05 12:44:44 [talpid_core::routing::imp::imp][DEBUG] Adding routes: {RequiredRoute { prefix: V4(Ipv4Network { addr: <IP_ADDRESS>, prefix: 0 }), node: RealNode>
Mar 05 12:44:44 [talpid_core::tunnel::openvpn::event_server][INFO] OpenVPN event Request { metadata: MetadataMap { headers: {"te": "trailers", "content-type>
Mar 05 12:44:44 [talpid_core::firewall][INFO] Applying firewall policy: Connected to <IP_ADDRESS>:1196 over UDP over "tun0" (ip: <IP_ADDRESS>,fdda:d0d0:cafe:>
Mar 05 12:44:44 [talpid_core::dns][INFO] Setting DNS servers to <IP_ADDRESS>, fdda:d0d0:cafe:1196::
Mar 05 12:44:44 [talpid_dbus::network_manager][DEBUG] /etc/resolv.conf differs from reference resolv.conf, therefore NM is not managing DNS
Mar 05 12:44:44 [talpid_core::dns::imp::static_resolv_conf][DEBUG] No DNS state backup to restore
Mar 05 12:44:44 [talpid_core::dns::imp][DEBUG] Managing DNS via /etc/resolv.conf
Mar 05 12:44:44 [mullvad_daemon][DEBUG] New tunnel state: Connected { endpoint: TunnelEndpoint { endpoint: Endpoint { address: <IP_ADDRESS>:1196, protocol>
Mar 05 12:44:44 [mullvad_daemon::management_interface][DEBUG] get_current_location
Mar 05 12:44:54 [mullvad_daemon][WARN] Unable to fetch GeoIP location: Error: Request timed out
Mar 05 12:44:54 Caused by: deadline has elapsed
...

@pinkisemils thanks for having a look! Which firewall rules would you need? Or shall we wait for you to try and fix it on your end first?

I've merged a fix for this already in our code. The issue was reverse path filtering. It seems that the default firewall does have a module to do that, so we added a chain to mark traffic from the relay so that it passes the RPF. Users who have enabled strict reverse path filtering in kernel should also configure  net.*.conf.*.src_valid_mark=1 appropriately, otherwise the kernel won't take the firewall mark into account when evaluating the reverse route, thus filtering our traffic.
The issues were stemming from the fact that we use policy based routing now, which is inherently at odds with reverse path filters unless our rules accommodate them. And now they do. We are still discussing if we should set the src_valid_mark sysctl at runtime. On one hand, it's just a very bad thing to do. On the other, we don't know under what circumstances would a user want to have strict reverse path filtering in the kernel and not want a way to have policy based routing to work.

I've now tried the beta. It times out when connecting via wireguard. When it switches to OpenVPN, it shows connected, but all connections time out. When disabling the firewall, it works perfectly again.
Here's some info:
❯ sudo sysctl -a | grep src_valid
sysctl: Schlüssel »kernel.spl.hostid« wird gelesen
net.ipv4.conf.all.src_valid_mark = 1
net.ipv4.conf.default.src_valid_mark = 1
net.ipv4.conf.docker0.src_valid_mark = 1
net.ipv4.conf.enp0s31f6.src_valid_mark = 1
net.ipv4.conf.enp59s0u1u3.src_valid_mark = 1
net.ipv4.conf.lo.src_valid_mark = 1
net.ipv4.conf.vboxnet0.src_valid_mark = 1
net.ipv4.conf.wlp2s0.src_valid_mark = 1

logs while trying to connect
❯ sudo ./result/bin/mullvad-daemon
[2021-04-13 14:39:47.606][mullvad_daemon::version][INFO] Starting mullvad-daemon - 2021.3-beta1 2021-04-13
[2021-04-13 14:39:47.606][mullvad_daemon][INFO] Logging to /var/log/mullvad-vpn
[2021-04-13 14:39:47.607][mullvad_daemon][INFO] Management interface listening on /var/run/mullvad-vpn
[2021-04-13 14:39:47.674][mullvad_daemon::relays][INFO] Initialized with 754 cached relays from 2021-04-13 14:25:09.868
[2021-04-13 14:39:47.674][mullvad_daemon::settings][INFO] Loading settings from /etc/mullvad-vpn/settings.json
[2021-04-13 14:39:47.674][mullvad_daemon::account_history][INFO] Opening account history file in /etc/mullvad-vpn/account-history.json
[2021-04-13 14:39:47.685][talpid_core::firewall][INFO] Resetting firewall policy
[2021-04-13 14:39:47.686][mullvad_daemon][INFO] Account already has WireGuard key
[2021-04-13 14:39:51.372][mullvad_daemon::relays][INFO] Selected relay de11-wireguard at <IP_ADDRESS>
[2021-04-13 14:39:51.376][talpid_core::firewall][INFO] Applying firewall policy: Connecting to <IP_ADDRESS>:46490 over UDP with gateways <IP_ADDRESS>,fc00:bbbb:bbbb:bb01::1, Blocking LAN
[2021-04-13 14:40:06.446][talpid_core::tunnel::wireguard][WARN] Timeout while checking tunnel connection
[2021-04-13 14:40:06.512][talpid_core::tunnel_state_machine::connecting_state][INFO] Tunnel closed. Reconnecting, attempt 1.
[2021-04-13 14:40:06.513][mullvad_daemon::relays][INFO] Selected relay de21-wireguard at <IP_ADDRESS>
[2021-04-13 14:40:06.513][talpid_core::firewall][INFO] Applying firewall policy: Connecting to <IP_ADDRESS>:53 over UDP with gateways <IP_ADDRESS>,fc00:bbbb:bbbb:bb01::1, Blocking LAN
[2021-04-13 14:40:21.563][talpid_core::tunnel::wireguard][WARN] Timeout while checking tunnel connection
[2021-04-13 14:40:21.625][talpid_core::tunnel_state_machine::connecting_state][INFO] Tunnel closed. Reconnecting, attempt 2.
[2021-04-13 14:40:21.626][mullvad_daemon::relays][INFO] Selected relay de-fra-002 at <IP_ADDRESS>
[2021-04-13 14:40:21.626][talpid_core::firewall][INFO] Applying firewall policy: Connecting to <IP_ADDRESS>:1302 over UDP with gateways , Blocking LAN
[2021-04-13 14:40:24.004][talpid_core::firewall][INFO] Applying firewall policy: Connected to <IP_ADDRESS>:1302 over UDP over "tun0" (ip: <IP_ADDRESS>, v4 gw: <IP_ADDRESS>, v6 gw: None), Blocking LAN
[2021-04-13 14:40:24.006][talpid_core::dns][INFO] Setting DNS servers to <IP_ADDRESS>
[2021-04-13 14:40:34.127][mullvad_daemon][WARN] Unable to fetch GeoIP location: Error: Request timed out
Caused by: deadline has elapsed
[2021-04-13 14:40:43.407][talpid_core::dns][INFO] Resetting DNS
[2021-04-13 14:40:44.325][talpid_core::firewall][INFO] Resetting firewall policy
^C[2021-04-13 14:40:47.052][mullvad_daemon::relays][ERROR] Relay list updater shutting down
[2021-04-13 14:40:47.052][mullvad_daemon][INFO] Tunnel state machine shut down
[2021-04-13 14:40:47.053][mullvad_daemon][INFO] Mullvad daemon is quitting
[2021-04-13 14:40:47.053][mullvad_daemon::management_interface][INFO] Management interface shut down

I peeked at mullvad's code and it seems like it uses nftables which I assume causes issues in some way with iptables. Replacing iptables with just nftables as a workaround seems to have fixed wireguard with the newest beta for me.
For reference here is the config I wrote to test with nftables:
{
    networking.firewall.enable = false;
    networking.nftables = {
      enable = true;
      ruleset = ''
        table inet firewall {
          chain inbound {
            type filter hook input priority 0;
            policy drop;

            iifname lo accept;

            tcp dport { ${builtins.concatStringsSep ", " ((map (port: toString port) config.networking.firewall.allowedTCPPorts) ++ (map (range: "${toString range.from}-${toString range.to}") config.networking.firewall.allowedTCPPortRanges)) } } accept
            udp dport { ${builtins.concatStringsSep ", " ((map (port: toString port) config.networking.firewall.allowedUDPPorts) ++ (map (range: "${toString range.from}-${toString range.to}") config.networking.firewall.allowedUDPPortRanges)) } } accept

            ct state {established, related} accept;
          }

          chain forward {
            type filter hook forward priority 0; policy drop;
          }

          chain outbound {
            type filter hook output priority 0; policy accept;
          }
        }
      '';
    };
}

No idea what the best way to fix this is, or if it even is possible without disabling iptables and moving to nftables which is a bit out of scope for this issue.

So, I've found a fix. Setting networking.firewall.checkReversePath = "loose" will allow connection.
@thefloweringash As you were the last one to edit the firewall module, I'm pinging you: Is there a security implication if this module sets reverse path filtering to loose?

So, I've found a fix. Setting networking.firewall.checkReversePath = "loose" will allow connection.
@thefloweringash As you were the last one to edit the firewall module, I'm pinging you: Is there a security implication if this module sets reverse path filtering to loose?

There's precedent for disabling this when it breaks vpn modules, see wg-quick. I don't want to make strong claims about security here. The reverse path filter is a security feature so disabling it necessarily affects security. If what wg-quick does is ok, then it should be ok here too.

Thanks for the feedback. I've opened a PR which acknowledges the potential security issue.

Have you tried our latest release?
Regardless, I don't believe RPF done via iptables is any better than RPF done via the kernel, but I must assume that iptables allows for a more flexible configuration.  Ideally, iptables RP filter would consider the packet mark during evaluation.

+1 to workarounds in this issue.
To summarize for anyone else wanting to setup mullvad VPN on NixOS (as of 2021-08-04)...

Add the following to /etc/nixos/configuration.nix:

  networking.firewall.checkReversePath = "loose";
  networking.wireguard.enable = true;
  services.mullvad-vpn.enable = true;

nixos-rebuild switch and reboot.

nix-shell -i mullvad-vpn

Follow https://mullvad.net/en/help/how-use-mullvad-cli/

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/mullvad-module-installed-but-missing-files-or-components/15953/2

I had a lot of headache making a WireGuard setup work with systemd-networkd, where all traffic is sent through wg tunnel. After a long amount of fiddling, I could make it work only with networking.firewall.checkReversePath = "loose";. Here is the config that worked for me, so that future me (and others) may suffer less:
let
  address = "<IP_ADDRESS>,dead::beef";
  dns = [ "<IP_ADDRESS>" ];
  peerPk = "...";
  allowedIPs = "<IP_ADDRESS>/0,::0/0";
  endpoint = "my-vpn-provider.net:5300";
  firewallMark = 13;
  routeTable = 1000;
  # utils
  inherit (config.networking) hostName;
  skAttr = "wg-private-key-${hostName}";
  split = lib.strings.splitString ",";
in {
  age.secrets.${skAttr} = {
    file = ../secrets/${skAttr};
    owner = "systemd-network";
  };

  systemd.network =
    {
      netdevs."90-wg" = {
        netdevConfig.Kind = "wireguard";
        netdevConfig.Name = "wg-extern";
        wireguardConfig.PrivateKeyFile = config.age.secrets.${skAttr}.path;
        wireguardConfig.FirewallMark = firewallMark;
        wireguardConfig.RouteTable = routeTable;
        wireguardPeers = [
          {
            wireguardPeerConfig.AllowedIPs = split allowedIPs;
            wireguardPeerConfig.Endpoint = endpoint;
            wireguardPeerConfig.PublicKey = peerPk;
          }
        ];
      };
      networks."90-wg" = {
        inherit dns;
        address = split address;
        linkConfig.ActivationPolicy = "manual";
        matchConfig.Name = "wg-extern";
        networkConfig.LinkLocalAddressing = "no";
        networkConfig.DNSDefaultRoute = true;
        networkConfig.Domains = "~";
        routingPolicyRules = [
          {
            routingPolicyRuleConfig = {
              FirewallMark = firewallMark;
              InvertRule = true;
              Table = routeTable;
              Priority = 10;
            };
          }
        ];
      };
    };
  # otherwise the firwallmark stuff doesn't work
  networking.firewall.checkReversePath = "loose";
}


Allow standardjs to custom extension file

What version of this package are you using?
16.0.4
What problem do you want to solve?
I am using a npm module called streamline that do have a custom extension file (._js).
And I would like to include them inside my standard call. But sadly there are ignored.
To check them i've to go one by one with standard ../path/to/file._js
What do you think is the correct solution to this problem?
The same way that you there is the possibility to add an ignore filter in the package.json file, add a key to include specific file extension.
Are you willing to submit a pull request to implement this change?
I don't think I'm capable of such things.

Found the option --ext in CLI
Managed a workaround :)


[CELEBORN-1178] Destroy fail reserved slots in LifecycleManager#reserveSlotsWithRetry

What changes were proposed in this pull request?
I'm testing main branch and encountered the following scenario.
I run sbin/stop-worker.sh near simultaneously on 3 out of 6 workers, and I'm expecting the 3 workers
will soon shutdown because I enabled graceful shutdown. However, only the first worker I stopped
shutdown in 15s as expected, the other two won't shutdown until shutdown timeout.
After digging into it, I found LifecycleManager#reserveSlotsWithRetry will reserve for the same location
twice:

At T1, only worker1 shutdown, pushes receive HARD_SPLIT and goes to revive
At T2, LifecycleManager handles revive requests in batch, and try to reallocate the locs to other workers
At T3, reserve to worker3 succeeds because it's not shutdown yet, but reserve to worker2 fails because it's shutdown
At T4, LifecycleManager will re-allocate the failed slots to other workers except worker1 and worker2. However, at this time Worker3 is also shutdown, so it fails to reserve on worker3
At T5, it re-allocates slots that failed to worker3. However, getFailedPartitionLocations will return slots allocated to worker3 in step 3, and increment the epoch to 2. At this time, worker3 has slots of epoch 1, but they will never to pushed to because newer epoch 3 is generated at the same time
Since the epoch 2 locs in worker3 will never be pushed to, it will never get a chance to return HARD_SPLIT, as a result it can't fast shutdown untile timeout.

This PR fixes this by destroying failed to be reserved slots in the process of reserveSlotsWithRetry
Why are the changes needed?
ditto
Does this PR introduce any user-facing change?
No
How was this patch tested?
Manual test.
Before:

After:

Thanks, merging to main(v0.4.0)


Make status checks take consecutive failures in time into account to get rid of temporary failures

When there is a temporary network failure, we should have the ability to wait for several consecutive checks to ensure that the failure is legit.
Each deployment graph vertex should have possibility to specify this behaviour.
Implementation suggestion - put this information in resource definition k8s annotations.
Proposal of specification of this behaviour:
"failLegitimacy": {
   "failsInTime": { # this will cause the status of vertex to be failed if it fails 3 times in 10 second time interval
      "failCount": 3,
     "time": 10,
   }
}

We should also take them into account "just after triggering previous event", so there should be some grace period.

IMO this should be in sync with forgiveness topic ( https://github.com/kubernetes/kubernetes/issues/1574 ).

@jellonek as a second phase for sure ;) For now we need internal mechanism to handle described situation ;-) Then we will need to extend it to more cases (forgiveness, node maintenance etc) and add more control (temporary disable etc etc).

Right, step after step.

Solved by https://github.com/Mirantis/k8s-AppController/pull/174


Remove Sk-wrapper versions of the DlAttribute objects

DisplayList classes like DlColorFilter, DlImageFilter and DlMaskFilter have a variant that wraps a similar Skia filter for cases where the engine receives a Skia object from a third party and needs to store it in a DisplayList structure.
Now that we have eliminated all code that uses SkCanvas for rendering from the engine, we no longer need these wrappers and they can be eliminated pursuant to a code search for remaining usages (mainly in the tests which should be rewritten to use only Dl classes).

Fixed in https://github.com/flutter/engine/pull/40005


[Bug]: Unique violation: 7 ERROR: duplicate key value violates unique constraint DETAIL: key already exist

Description
When adding a FQDN to a new service, however this domain name has already been added to a service before and deleted already.
The domain key however is still saved in the database of Coolify even after service has been deleted.
Minimal Reproduction (if possible, example repository)
Create a service with a FQDN, delete the service, and then re-create a new package with the same FQDN.
Exception or Error
No response
Version
v4.0.0-beta.62

Can you pleas try to delete the stuck service with this? Does it show you that service?

Hi,
Thank you so much. After running docker exec -ti coolify bash -c "php artisan resources:delete" and remove the service resouce, it works now.


Automate regenerating vendored licenses?

Just some random thoughts, can we automate regenerating vendored licenses? Could pre-commit.ci invoke this once every X days; does it need to be a hook for that? How should version numbers change when licenses are updated? etc...

yeah it would have to be a hook and with a version some how, I don't think abusing pre-commit.ci to do this is a good idea
I'm also not really sure that we need to update it often, nobody has asked about it in a while and the only consumer I know of is setup-cfg-fmt which has an even-more-restrictive set of licenses it supports (due to pypi)
probably not worth over-engineering this

Makes sense, thanks for the context


Add commutator support

We want to add two operators that make using the Open Ephys commutator easy:

A sink node that takes sequence of doubles and turns the commutator accordly via a specified serial port.  It can also expose commutator options such as enable/disable, LED status etc.
A transform node that converts Quaternions to rotation values about an axis.

This is being handled in open-ephys/bonsai-commutator#1


feat: Parse column types

Signed-off-by: Tamika Tannis<EMAIL_ADDRESS>Summary of Changes
This PR:

Passes database property from TableDetail through the column list component to ColumnType.
Creates parser.ts which owns the logic for parsing complex column types for supported databases. It creates an object of the NestedType interface.
Adds logic to ColumnType to render the information in NestedType.

Tests
Added some tests for parser.ts.
TODO on feature branch before merge:

Tests for parseNestedTypeHelper after collecting use cases.
Still need tests for ColumnType interactions

Documentation
N/A for this feature
CheckList
Make sure you have checked all steps below to ensure a timely review.

[x] PR title addresses the issue accurately and concisely. Example: "Updates the version of Flask to v1.0.2"

In case you are adding a dependency, check if the license complies with the ASF 3rd Party License Policy.

[x] PR includes a summary of changes, including screenshots of any UI changes.
[x] PR adds unit tests, updates existing unit tests, OR documents why no test additions or modifications are needed.
[x] In case of new functionality, my PR adds documentation that describes how to use it.

All the public python functions and the classes in the PR contain docstrings that explain what it does

[x] PR passes all tests documented in the developer guide

@Golodhros I'm not understanding why the betterer check doesn't pass here, as I expected it to only block on errors with new code not existing code. Am I missing something?

@Golodhros I'm not understanding why the betterer check doesn't pass here, as I expected it to only block on errors with new code not existing code. Am I missing something?

Sometimes it happens that the line of the error changes, so the new error entry is different. You will need to update the results with npm run betterer:update if that's the case.

@Golodhros I'm not understanding why the betterer check doesn't pass here, as I expected it to only block on errors with new code not existing code. Am I missing something?

Sometimes it happens that the line of the error changes, so the new error entry is different. You will need to update the results with npm run betterer:update if that's the case.

Oh that makes sense. I'll see how many I can fix in a chore PR, especially for files I think we are about to touch quite often for the new project`.

would like to take a look as well once the pr is merged back to master.


Update to loginPanel help page

About
This Automated Data Extraction PlaTform is a set of webtools to facilitate access to the xDD library of full-text documents (https://xdd.wisc.edu). xDD is the largest collection of open and controlled-access scientific articles available for bulk text data-mining in the world.
Machine-learning researchers can use ADEPT to:

SEARCH xDD metadata records
Create a SAVED SET of one or more search results
Request a TEST SET of 200 random documents from a saved set for local development purposes.
Submit APPLICATIONS for high-throughput deployment on the entire xDD library
Share saved sets, test-sets, and applications with GROUPS of collaborators

SAVED SET
A user may save the results of one or more metadata searches as a SAVED SET. Saved Sets are created in the saved sets tab, then populated by saving searches and citations on the Search page. The search parameters in a saved set can be also be used to access the xDD API directly to access the same data directly from the source.
TEST SET
xDD must control access to its resources to ensure that they are used for legitimate scientific research in bulk text data-mining and that xDD is not used by bad-actors to circumnavigate normal licensing restrictions (i.e., to download PDFs that they would not normally have legal access to for purposes other than text data-mining). xDD fulfils this responsibility by ensuring that its full-text documents do not leave its servers.
An exception to this rule are test-sets that ADEPT users may use for local development of xDD APPLICATIONS. A test-set is a random sample of 200 documents from a SAVED SET that a user may download onto their local machines for development purposes. New test sets can be requested from the SAVED SETS or DICTIONARIES tab. Once approved by the xDD system, users can find a unique API-key and URL to to their requested test-set in the TEST SETS tab.
DICTIONARIES
Dictionaries are a collection of search terms, and can be either xDD identified Dictionaries, or user constructed dictionaries. Dictionaries can also be used as the source for the creation of TEST SETS, which are a data product identified in the GeoDeepDive system used in ML processing.
APPLICATIONS
ADEPT defines an xDD application as a dockerized set of machine-learning scripts for the purpose of bulk text data-mining. xDD administrators need to monitor whether a submitted text data-mining application is respectful of its Terms of Service. Users can use the applications tab to submit an application for review and approval by an xDD administrator or to request that an already approved application be deployed on the xDD high-throughput computing architecture.
GROUPS
A user may create a GROUP of collaborators to share items with (i.e., SETS, TEST SETS, and APPLICATIONS). Group members have full edit privileges for shared items and may add, edit, or delete items within a group. Note that deleting a shared item will delete that item for all users.

In addition to updating this help page, would it be possible to hide or disable the "Status" and "Upload Document" buttons?


Added WebIdentityTokenCredentialsProvider in AWS Credentials Provider…

Added WebIdentityTokenCredentialsProvider in AWS Credentials Provider Chain. This is required for supporting pod level IAM roles for service accounts (IRSA).

Support for AWS IRSA for druid pods in a kubernetes cluster

I have added WebIdentityTokenCredentialsProvider credentials provider in the provider chain.

This PR has:

[x] been self-reviewed.
[ ] added documentation for new or modified features or behaviors.
[ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
[ ] added or updated version, license, or notice information in licenses.yaml
[ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
[ ] added unit tests or modified existing tests to cover new code paths.
[ ] added integration tests.
[x] been tested in a test Druid cluster.

Key changed/added classes in this PR

org.apache.druid.common.aws.AWSCredentialsUtils

It appears this support was added in #10541


Question: Math functions

Hi,
is it possible to integrate math functions like sqrt, sum, pow etc.. ?
There is an extension that include this functions: https://www.sqlite.org/contrib/download/extension-functions.c?get=25
Thanks for the nice work

This would be a huge improvement to include. I use SQLite as a client side cache of a remote PostgreSQL database, so there are several queries that I can't run (even with workarounds/rewrites) because of the lack of math functions.
I would love to contribute to this project and help out with this, iOS seems straightforward because the SQLite source is right there, but I'm lost when it comes to Android since I don't see the SQLite source files there. If someone could point me in the right direction, I'd happily send a pull request.

Yes I agree this would be nice to have. I am not entirely sure how to get started on this either. The objective is to keep the functionality on par for both platforms.

@andpor Is the difficulty that only the built-in SQLite is used on Android? Is there any way to use a custom version?

Both platforms have built in SQLite...

Ha, that's what I get for not looking through the source files and just assuming. Would using a custom SQLite build be beyond the scope of this project?
I would think using a custom build would be the only way to really ensure feature parity since there's no chance that iOS and Android would ever both stay on the latest SQLite releases and agree on the same extensions.

@andpor I would like to propose a small correction: I think there is also src/android-native that uses the custom build of SQLite that I did for Android. It is entirely possible to rebuild that part from https://github.com/liteglue/Android-sqlite-connector and https://github.com/liteglue/Android-sqlite-native-driver. For iOS I think it should be relatively straightforward to include a more recent sqlite amalgamation which is what I do in Cordova-sqlite-storage. Note that I include these sqlite libraries as external dependencies so they don't pollute the project history.
In theory it should be possible for people to simply include these math functions by rebuilding the Android-sqlite-native-driver libs and building the iOS version with a custom version of sqlite. Unfortunately this process can be a major challenge for many application developers. Guidance for such custom builds would be great, also for the Cordova-sqlite-storage project that I maintain.

@andpor,
I've made a plugin model for our storage implementation for react-native in order to quickly switch between different implementations on android platform.
I could contribute that part back.
this would allow hooking of any implementation of sqlite without changing the main code base.
But the change would lead to deletion of android-native project.
I didn't implement the support for that implementation as we don't need it.
On the bright side - it's going to remove most of the code duplication you have for android now :)
It should be easy to make an implementation of this database for the community after my changes though.

@andpor Is it possible to use these math functions now?


add database-global compact operation

REST API documentation: https://github.com/arangodb/arangodb/pull/12666


List of currency chemicals

From the 2024-01-29 ontology editors call:
Reactome has created a list of 'currency' chemicals that may help GO define those entities for use in the ontology and in annotation.
@deustp01 has asked Reactome curators/developers for help in getting this list.  Thx.

Here is an e-mail thread that discusses the Reactome list of trivial / housekeeping / currency chemicals. The link to a Google doc in the thread points to a site restricted to Reactome personnel, so here is the table as an Excel spreadsheet:
Trivial chemicals.xlsx
Here's the e-mail thread:

Subject:
Re: List of trivial chemicals

Fri, 27 Feb 2015 09:21:46 +0000

Phani Garapati<EMAIL_ADDRESS>

Henning Hermjakob<EMAIL_ADDRESS>

Bijay Jassal<EMAIL_ADDRESS>Steve Jupe<EMAIL_ADDRESS>Maximilian Koch<EMAIL_ADDRESS>Antonio Fabregat<EMAIL_ADDRESS>

Hi Henning,I removed amino acids and ammonia. Not sure about oxygen and carbon dioxide either but we can try with this list and see how it effects the score.I Will inform Max.Thanks,PhaniOn 27/02/2015 06:00, Henning Hermjakob wrote:Phani, could you please finalise and close the list this morning, and then notify Max?On 26/02/2015 10:10, Bijay Jassal wrote:Hi HenningAh, I didn’t realise that. In that case no, they are not specific inthat respect to distinguish the pathway.CheersBijayOn 26 Feb 2015, at 10:08, Henning Hermjakob <hhe<EMAIL_ADDRESS>wrote:Hi Bijay,the point (for this application) is not if the chemicals are importantfor the reaction, but if they are SPECIFIC and help distinguished thispathway from other pathways. I guess even in the examples you list,that would not be the case?On 26/02/2015 10:00, Bijay Jassal wrote:Hi PhaniGenerally, the list so far would be OK to exclude but as ever, I thinkthere are exceptions where we can’t so it all depends on the context ofthe reaction we’re looking at.Take H+ for example.Most of the time, I would be happy to exclude it but in some cases, H+is a valid input/output of a reaction. Take the electron transport chainexample herehttp://reactomecurator.oicr.on.ca/PathwayBrowser/#DIAGRAM=163200&ID=163217&PATH=1430728,1428517You need the electrochemical gradient generated by H+ pumping tocreate ATP.Also, ADP/ATP and GDP/GTP are important in nucleotide exchange reactionsto activate/deactivate enzymes.Another issue crops up in transport reactions where some of these commonmolecules are transported across membranes, so validly take part in thereaction.I guess there was always going to be exceptions and I wouldn’t know knowhow to globally distinguish a H+ as common or valid.However, I would still add these molecules to the exclusion list and seewhat the output looks like after exclusion.CheersBijayOn 26 Feb 2015, at 09:44, Phani Garapati <phani<EMAIL_ADDRESS>wrote:Hi Bijay,I added few members to the doc looking through the reference moleculelist in curator tool. Can you check from that if any molecule is nottrivial.Thanks,PhaniOn 26/02/2015 09:07, Bijay Jassal wrote:Hi guysIf anyone can find/retrieve the old pathfinder tool, that contained alist of exclusion molecules.CheersBijayOn 26 Feb 2015, at 06:45, Henning Hermjakob <hhe@ebi.ac.uk<mailto:hhe<EMAIL_ADDRESS>wrote:Hi Phani,could you please complete the list of "trivial chemicals" at [1]?These are chemicals which are used frequently in Reactome, but arevery unspecific and don't carry much information, standard exampleis water. I came across these because they are messing up thepathway analysis. We have a good match between a set of Biomodelsidentifiers and a Reactome pathway, but it's based on ATP, ADP, andNADPH; not great.These are also candidates to exclude when we try to presentsimplified pathway diagrams in the future.[1]https://docs.google.com/spreadsheets/d/1F_lQY_4cOL9BxhThOMHX-nhU4B0VwsW--rSCUYe_Wb0/edit?usp=sharing -- Best regards,HenningSubject:	Re: List of trivial chemicals
Date:	Fri, 27 Feb 2015 09:21:46 +0000
From:	Phani Garapati<EMAIL_ADDRESS>To:	Henning Hermjakob<EMAIL_ADDRESS>CC:	Bijay Jassal<EMAIL_ADDRESS>Steve Jupe<EMAIL_ADDRESS>Maximilian Koch<EMAIL_ADDRESS>Antonio Fabregat Mundo<EMAIL_ADDRESS>Hi Henning,
I removed amino acids and ammonia. Not sure about oxygen and carbon dioxide either but we can try with this list and see how it effects the score.
I Will inform Max.
Thanks,
Phani
On 27/02/2015 06:00, Henning Hermjakob wrote:
Phani, could you please finalise and close the list this morning, and then notify Max?
On 26/02/2015 10:10, Bijay Jassal wrote:
Hi Henning
Ah, I didn’t realise that. In that case no, they are not specific in
that respect to distinguish the pathway.
Cheers
Bijay
On 26 Feb 2015, at 10:08, Henning Hermjakob<EMAIL_ADDRESS><EMAIL_ADDRESS>wrote:
Hi Bijay,
the point (for this application) is not if the chemicals are important
for the reaction, but if they are SPECIFIC and help distinguished this
pathway from other pathways. I guess even in the examples you list,
that would not be the case?
On 26/02/2015 10:00, Bijay Jassal wrote:
Hi Phani
Generally, the list so far would be OK to exclude but as ever, I think
there are exceptions where we can’t so it all depends on the context of
the reaction we’re looking at.
Take H+ for example.
Most of the time, I would be happy to exclude it but in some cases, H+
is a valid input/output of a reaction. Take the electron transport chain
example here
http://reactomecurator.oicr.on.ca/PathwayBrowser/#DIAGRAM=163200&ID=163217&PATH=1430728,1428517
You need the electrochemical gradient generated by H+ pumping to
create ATP.
Also, ADP/ATP and GDP/GTP are important in nucleotide exchange reactions
to activate/deactivate enzymes.
Another issue crops up in transport reactions where some of these common
molecules are transported across membranes, so validly take part in the
reaction.
I guess there was always going to be exceptions and I wouldn’t know know
how to globally distinguish a H+ as common or valid.
However, I would still add these molecules to the exclusion list and see
what the output looks like after exclusion.
Cheers
Bijay
On 26 Feb 2015, at 09:44, Phani Garapati<EMAIL_ADDRESS><EMAIL_ADDRESS>wrote:
Hi Bijay,
I added few members to the doc looking through the reference molecule
list in curator tool. Can you check from that if any molecule is not
trivial.
Thanks,
Phani
On 26/02/2015 09:07, Bijay Jassal wrote:
Hi guys
If anyone can find/retrieve the old pathfinder tool, that contained a
list of exclusion molecules.
Cheers
Bijay
On 26 Feb 2015, at 06:45, Henning Hermjakob<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>wrote:
Hi Phani,
could you please complete the list of "trivial chemicals" at [1]?
These are chemicals which are used frequently in Reactome, but are
very unspecific and don't carry much information, standard example
is water. I came across these because they are messing up the
pathway analysis. We have a good match between a set of Biomodels
identifiers and a Reactome pathway, but it's based on ATP, ADP, and
NADPH; not great.
These are also candidates to exclude when we try to present
simplified pathway diagrams in the future.
[1]
https://docs.google.com/spreadsheets/d/1F_lQY_4cOL9BxhThOMHX-nhU4B0VwsW--rSCUYe_Wb0/edit?usp=sharing
--
Best regards,
Henning

Thanks @deustp01
I moved this to the wiki
https://wiki.geneontology.org/Currency_chemicals
(Note that in your list, UTP and NADP+ are present twice)
Thanks, Pascale

@deustp01 FYI

I have changed the ChEBI Ids to the pH 7.3 (UniProt set) in the wiki table.  Most chemicals were changed, except for H2O, H+, CO2  and O2
I removed OH-; I dont see any usage of this in GO or in RHEA. I think H2O is used instead.

Thanks, Pascale

Hi Peter,
I added this to next Monday's ontology call for discussion.
The key point, IMHO, is who is this list for? I thought it was internal to GO, to help define inputs and primary inputs in GO-CAMs. In this case we probably dont need the non-pH7.3 terms (and OH-).
But I may not be thinking of all use cases of this list.
Thanks, Pascale


config.yaml new loaction

Hey there,
config.yaml file is no longer under $HOME/.hueristiq/xurlfind3r/config.yaml , it's currently under /$HOME/.config/xurlfind3r/config.yaml , please update it in the readme file

Seems like there is another issue with intel x
[ERR] intelx: unexpected status code 400 received from https://2.intelx.io/phonebook/search/result?k=key&id=00000000-0000-0000-0000-000000000000&limit=10000
in the ID value it doesn't generate an actual search id , instead, it uses 00000000-0000-0000-0000-000000000000 which causes the call to crash

Hey there,
config.yaml file is no longer under $HOME/.hueristiq/xurlfind3r/config.yaml , it's currently under /$HOME/.config/xurlfind3r/config.yaml , please update it in the readme file

Ooh! yeah!

Let me check on the issues


BUG fix civisml 1.0 prediction template id

I noticed that both CivisML 1.0 and 1.1 had the same prediction id. This PR corrects the 1.0 template id.

I did some git blame archaeology here, and it looks like this was an intentional change to ensure that the most up-to-date prediction template within the major version. This isn't the process we've been following in 2.0; we've been keeping training and prediction matches at the minor version.
I personally lean toward keeping templates matched up at the minor version level, for the sake of reproducibility. Either way, I'd rather be consistent in how we're doing this between versions 1 and 2. What are your thoughts @stephen-hoover?

Yes, we'd originally planned to always make predictions with the most recent minor version, but discovered that wouldn't always work.
I agree, for all major versions we should use the prediction job minor version corresponding to the training job. Hopefully no one is still using v1, but the consistency would be a good idea anyway.


Update app icon for macOS according to Big Sur specifications

Summary of the new feature/enhancement
Please update the app icon for macOS according to Big Sur specifications - https://developer.apple.com/design/human-interface-guidelines/foundations/app-icons#platform-considerations
Proposed technical implementation details (optional)

/cc @TravisEz13 and @adityapatwardhan

ping

ping


Fix broken count

Resolves #49.
It appears that the ActiveRecord API has changed or maybe was interpreted incorrectly (?) causing this error. I went on a small adventure trying to figure this one out and came to the conclusion that all of the other piping is fine and that a small tweak to the API of the nulldb EmptyResult object gives us a solution.
ActiveRecord uses the type metadata in this context, in active_record/relations/calculations.rb inside #execute_simple_calculation:
type   = result.column_types.fetch(column_alias) do
  type_for(column_name)
end

Well, it looks like we need to see what the Result API should implement. What better place than the spec?
    test "cast_values uses identity type for unknown types" do
      values = [["1.1", "2.2"], ["3.3", "4.4"]]
      columns = ["col1", "col2"]
      types = { "col1" => Type::Integer.new }
      result = Result.new(columns, values, types)

      assert_equal [[1, "2.2"], [3, "4.4"]], result.cast_values
    end

Cool. We need a Hash of column names to reified ActiveRecord::Type objects! Problem solved. I implemented this interface, but there was another issue:
[1] pry(#<ActiveRecord::ConnectionAdapters::NullDBAdapter>)> ActiveRecord::Type.lookup(:primary_key)
ArgumentError: Unknown type :primary_key
from /home/mach/.rbenv/versions/2.4.0/lib/ruby/gems/2.4.0/gems/activemodel-5.2.0/lib/active_model/type/registry.rb:22:in `lookup'

We also have to register a :primary_key type with the ActiveRecord::Type registry, as this is a fairly common configuration that is exposed via schema.rb, so this functionality was also added to the project.
I added a spec to check that count works, which previously did not exist and would fail without these changes as below:
Failures:

  1) NullDB should handle count queries
     Failure/Error: expect(Employee.count).to eql(0)

     TypeError:
       no implicit conversion of String into Integer
     # ./spec/nulldb_spec.rb:292:in `block (2 levels) in <top (required)>'

Finished in 0.05611 seconds (files took 0.45248 seconds to load)
38 examples, 1 failure

Failed examples:

rspec ./spec/nulldb_spec.rb:291 # NullDB should handle count queries

Coverage report generated for RSpec to /home/mach/Workspace/nulldb/coverage. 250 / 311 LOC (80.39%) covered.

There is also a spec around the type registry change. I hope that this solves a lot of workarounds!

Looks like I broke everything < ActiveRecord 5.0. I think I found the API equivalent for 4.2x.
I didn't look yet how far back that convention goes, however we could also just not register the type and add a note in the README. I await your feedback.

@mach-kernel Technically nulldb still passes specs with rails 2, but I don't expect people to support rails 2 or even 3, but I think 4 should be supported (but I'm open for discussion).
Basically try to get the specs passing, but I'm happy to drop rails 2/3 support if needed (those needing legacy support can stick to old gem versions)

@mnoack I fixed it!

@mnoack :wave:, let me know if there's anything left to do for getting this out 😸

@mach-kernel - One comment, but otherwise I'm quite happy. It passes the tests and you've added a new test which is all good.

@mnoack looks like it breaks with master ActiveRecord. I can investigate this if needed, but I don't think it would be a good idea to build against new APIs or even API changes until it is officially decided that they wish to release.

@mach-kernel - Sorry finally got around to this. Yes master doesn't matter (I should work out how to exclude that test from the build). It's just so we can ensure we stay compatible with the latest rails as there are changes.
I've released a gem version with your changes and some other recent PRs since the last release.


created a MATLAB GUI using App Developer to take user input

Created a MATLAB app which takes user input and runs molto_it using that input. The aesthetics can be changed as required, this is just a sample prototype.
To run this just run app called 'run.mlapp'.
I can also edit the README and other stuff as required.

I changed the pull request from master to development branch


Auto size detection for mods

Usually when downloading with a web browser, it knows the size of the file it is downloading so can do resume correctly and a progress bar.
I think there needs to be a change within 7h to do the same.
Currently we have to manually put the size of the mod in the catalog, this has to be manually updated if a mod updated changes the size.
If the size is wrong on the catalog it can break resuming downloads or make downloads hang.  Generating 416 errors on the web server.
I did disable range detection on the web server but this ended up breaking "all" resumes so its now back on and the catalog sizes have been updated for every mod, however I think the way forward is to implement a new size detection process, whether this is hints sent from the server on downloads or another auto detection method.
Once this is done we can repurpose the size shown on the catalog page to be a cosmetic install size for the end user to know how much space they need instead of using that value for the download code.

Some additional info.
7thheaven when downloading mods seems to treat every file as a resume even when the starting point is 0 bytes, the response code is 206 when succeeding but any downloads from a web browser are code 200.

So it should be possible to use the server headers on connection to know how big a file is if the server sends that, however, because the download manager uses multiple methods it might not be so simple GDrive and Mega are Via API's verses raw HTTP address downloads.

There is a header that is returned for every request so yep should be possible to use the headers.
Should be the "content-length" header.
On a normal download (web browsing etc.) the client doesnt request a range, instead it just requests the file, and that header is sent to client so it knows the size.  7th heaven is "always" requesting a range of bytes for every download, which seems to be causing problems for some people.
I dont know if this change would even affect google drive or megadrive as the api's are different, but of course we should test to see if those are impacted.
Originally 7h 1.5 didnt even support resuming downloads, and I think this was the same for 2.0 as well, so that code if you can find it probably does the same as a web browser and I guess the resume support was added for every download instead of just when someone clicks the resume button.

yeah it should be but it can also be missing if it's vai a download script that does not send it and not sure what we can do about that, other than have a file hash verification

Ok, im going to kill this off for 7th heaven, and move it to prelude i think, adding this sort of fundamental rewrite to the downloader is going to be a big task due to the way the downloader is coded, so should work better with a new downloader which will be implemented into prelude.


Add missing Jetty hints

Based on my tests, the following Jetty hint is missing with Jetty 11.0.5 provided with Spring Boot 3:
[
  {
    "name": "org.eclipse.jetty.server.ForwardedRequestCustomizer$Forwarded",
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true
  }
]

cc @mhalbritter.

I'll take a look at this.

I wondered why our smoke tests were green. This issue only pops up when setting server.forward-headers-strategy=native

Fixed in https://github.com/oracle/graalvm-reachability-metadata/pull/285

Thanks!


[Brick] Add editable label widget

We should have an editable label widget which can switch between label and editor, for example on double click.

Editable label can be created like this:
BrEditableLabel new
	look: BrGlamorousEditableLabelLook new;
	text: 'Double click me'

When pressing enter or clicking outside the BrEditorAcceptWish event is send by the label and text changes:

When pressing escape the BrEditorCancelWish is sent by the label and text rolls back to the original:


npm ERR! missing script: build

I'm getting the following error when I perform npm run build:
$ npm run build
npm ERR! Darwin 16.0.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "build"
npm ERR! node v6.2.2
npm ERR! npm  v3.9.5

npm ERR! missing script: build

Good call. I recently removed the build script so all you have to do isnpm install. Will fix the readme.


Upgrade from 3.0 to 3.1 fails

Upgrading from 3.0 to 3.1 results in application server/container just hanging on startup. I stopped the server after approximately 30min. My setup is a powerful GNU/Linux box with Tomcat7 and OpenJDK7-JRE. I tried to redeploy the app restarting tomcat.
All previous upgrades from ~2.2 till 3.0 worked flawlessly within the same setup. Unfortunately nothing is in the logs except "[...] deploying gitbucket.war [...]". Going back to 3.0 again works fine.
Had somebody with similar setup managed to upgrade or having same issues?
Thx in Advance.

It might be H2 version compartibility issue. Some users are saying it worked by replacing h2-1.4.186.jar with h2-1.4.180.jar in gitbucket.war.
We will release 3.1.1 with h2-1.4.180 to avoid H2 version compatibility issue as soon as possible.

That was it. Thx.


Add options to theme notification buttons and badges

Addresses feature-request #28280

@hasan2040,
Thanks for your contribution.
To ensure that the project team has proper rights to use your work, please complete the Contribution License Agreement at https://cla.microsoft.com.
It will cover your contributions to all Microsoft-managed open source projects.
Thanks,
Microsoft Pull Request Bot

@hasan2040, thanks for signing the contribution license agreement. We will now validate the agreement and then the pull request.
Thanks, Microsoft Pull Request Bot

LGTM 👍


Non partial Data.Text.head

Is there a non partial Data.Text.head function? Since Text is the default 'String' type to use with relude, it would be nice to have. Also, safe functions is one of the main reasons I chose to use relude (fantastic work on this library, by the way). Does this function already exist somewhere that I haven't seen? If not, I can have a go at writing it and making a PR if you'd like.

Hi @JoshuaHall! Thanks for your kind words about Relude 😊
We reexport all Text-related functionality from the text package, and it doesn't contain the analogue of NonEmpty for lists or a safe head function. I imagine that the NonEmptyText data type will look like this:
data NonEmptyText = NonEmptyText Char Text

and indeed, there are a few packages in the Haskell ecosystem that provide such type and the corresponding API:

https://hackage.haskell.org/package/non-empty-text
https://hackage.haskell.org/package/text1

These packages don't seem common enough, and relude goals are to be minimal in terms of dependencies and introducing custom things, so I don't think that adding one of those package to relude dependencies or reimplementing the NonEmptyText data type is an option. But you can use them in your project, should play nice together with Relude 🙂
If you want to safely extract first element of the Text, I can suggest using the uncons function from text. But uncons is a very common name, and reexporting it only for Text will cause a lot of conflicts for Relude users. I wish GHC supported qualified reexports, it would allow making Relude even more convenient and friendly!
Ideally, it would be nice if the text package provided the NonEmptyText data type, but that's a proposal for text ✍

That makes sense, I'll use another package. Thanks.


Some problems aboult this code

Infact the "max_stepsize" in this code should't be 64.The "max_stepsize" is equal to 12,which is shrunk from original "image_width"(180)  to 180/2/2/2/2 = 12.Remenber the core idea in CRNN+CTC is that we split the image vertically to many slices,and we predict each slice's classes,finally using CTC to decode the predicted sequence to the respectd result.For example "aaa_bb_c_"and "a__b_ccc" both respect to the same label "abc",you can also read the paper  for more details.
But when I run the wrong code in author's dataset,and I got 98% accuracy while I got a bad result in  VGGWord dataset.Finally I got a good result after changing the code.
So, why this code work in your situation,I am very courious about this.Thank you.

@980044579 , thanks for sharing your observations and experience.

With the great source codes in this project and the data provided, I was able to reproduce the author's result, getting 0.997 at 50th epoch.
I agree with you on the max_stepsize. it should be in the direction of "image_width", 12 in this project. I also plan to correct this and see how it might impact the final result., If it's okay, can you share your code changes in this area?

Just change the code between CNN -> RNN in cnn_lstm_otc_ocr.py, make sure the shape of the input of RNN is [batch_size, max_stepsize, num_features].

Hi @980044579 , thanks a lot for your kind reply. I did the code changes too in yesterday and found the model can achieve 0.999 accuracy at 12th epoch. so the model is able to converge faster and achieve better performance after fixing this bug.
For those who are interested, here is my code changes.

Good job~

@anubhavrohatgi   @980044579 ,hello, i run into the same question,but i inspect the label and find the max length of label is not greater than maxT in[maxT,batch_size,num_char],have you solve it? i don't konw how to do it

@anubhavrohatgi @kstys   make sure you understand how the  framework "CNN + RNN + CTC" work and there are some bugs in this code.You should not only change the  "maxsteps" in utils.py but also the code between CNN ——> RNN in  cnn_lstm_otc_ocr.py

I have a question.  in the file of cnn_letm_otc_ocr.oy , after cnn, the x.set_shape([FLAGS.batch_size, filters[3], 24]) is right?  the time sequence should be the width which will be feed to the LSTM, but the code is the length of channels.

I changed the code as   @LevinJ  ,but i got a error  "tensorflow/core/util/ctc/ctc_loss_calculator.cc:144] No valid path found."
I set the max_step as 128  and my input image is 32*192


Bump base dependency

Could you update the Cabal file to allow building with base 4.11, which is what GHC 8.4 uses?

Done


audio support

hi im loving your addon!
is there a way to enable audio support in the firefox browser?
i access it via vnc on my old ipad
best regards

maybe anyone knows something to get this to work?

Hi, having the sound on the host where the add-on is running is something that should be possible.
That's mean if you have HA running on HA Yellow or Blue for instance, you could be able to hear the sound using the jack port.
However, if you want to be able to have get the sound on a remote PC that is used to access Firefox through ingress, it requires specific versions of VNC and / or the usage of Pulse Audio client.
A few issues related to audio are open in the upstream project:
https://github.com/jlesage/docker-firefox/issues?q=is%3Aissue+is%3Aopen+audio

thx for your answer!
so i connect to the firefox docker via an old ipad via vnc, is there something i must configure via the variables?
i saw in the docker config file of your addon that the audio part is uncommented?

I don't think that's possible because you won't be able to run a Pulse Audio client on the iPad.

Audio support is now available through the browser in version 1.3.2.
It's not a fix for your original request but as I explained, I don't think I can do much more for your specific setup.


Add build instructions on linux

I'll add installation instructions when everything is done.

Do we need this? @ImUrX

It seems correct, can't test it here.


Creating dom event with readonly props doesn't work on inherited events

Subject of the issue
Fail to trigger dragstart event on wrapper with clientX/Y props
Steps to reproduce
wrapper.trigger('dragstart', {clientX: 999})

Expected behaviour
Events are triggered.
Actual behaviour
Got error:
TypeError: Cannot set property clientX of #<MouseEvent> which has only a getter
	    at eval (webpack-internal:///./node_modules/@vue/test-utils/dist/vue-test-utils.js:3141:18)

Possible Solution

Since DragEvent inherits properties from MouseEvent, we shall get prototype recursively until find the field.

Like the message says, TypeError: Cannot set property clientX of #<MouseEvent> which has only a getter. This is a Web APIs limitation and not something we can do anything about.
You could consider something like https://github.com/cypress-io/cypress/tree/master/npm/vue. Cypress has support for drag n drop (here is the recipe: https://github.com/cypress-io/cypress-example-recipes/tree/master/examples/testing-dom__drag-drop)

Like the message says, TypeError: Cannot set property clientX of #<MouseEvent> which has only a getter. This is a Web APIs limitation and not something we can do anything about.
You could consider something like https://github.com/cypress-io/cypress/tree/master/npm/vue. Cypress has support for drag n drop (here is the recipe: https://github.com/cypress-io/cypress-example-recipes/tree/master/examples/testing-dom__drag-drop)

I think you don't get the point. I know clientX is the readonly property of MouseEvent as that event.clientX should not be assigned but as I mentioned above createDOMEvent  of vue-test-utils do it even though there is a readonly check. So what you shall care about is the incomplete implementation of canSetProperty which ignores inherited events.
In this case, clientX has already be assigned to the MouseEvent when initailizing the event. There is no need to assign it again later.
https://github.com/vuejs/vue-test-utils/blob/c4133d6a78cd0ad7efafdfadb4f1ae28e31079b4/packages/test-utils/src/create-dom-event.js#L51-L56

I think you don't get the point. I know clientX is the readonly property of MouseEvent as that event.clientX should not be assigned but as I mentioned above createDOMEvent  of vue-test-utils do it even though there is a readonly check. So what you shall care about is the incomplete implementation of canSetProperty which ignores inherited events.
In this case, clientX has already be assigned to the MouseEvent when initailizing the event. There is no need to assign it again later.
https://github.com/vuejs/vue-test-utils/blob/c4133d6a78cd0ad7efafdfadb4f1ae28e31079b4/packages/test-utils/src/create-dom-event.js#L51-L56

Maybe I misunderstood. Are you suggesting we should update the code to not assign the value?

Maybe I misunderstood. Are you suggesting we should update the code to not assign the value?

In the current implementation, there are two steps to assign values to event.
Step 1: The returned object of getOptions contains all options of createDOMEvent and the object is passed to Event constructor as second parameter. For MouseEvent, the object is recognized as mouseEventInit by which you can set clientX and other properties to MouseEvent at the stage of initializing it. clientX is setted at this step.
Step 2: Once event has been created, an iteration of options of createDOMEvent is triggered to assign key-values to the event. During the iteration, setter of event[key] is checked to ensure assignment is only applied to wrtitable field.
At the first step fields which are valid for event constructor are accepted and the second step is to ensure rest fields are attached to the event as well. It works well for MouseEvent but not for DragEvent. As I mentioned above, the setter check algorithm is not suitable for DragEvent because DragEvent inherits properties from MouseEvent. clientX is readonly property of MouseEvent as well as DragEvent but Object.getOwnPropertyDescriptor(dragEvent, 'clientX') could not handle it. getOwnPropertyDescriptorRecursively is a more proper for this situation.

In the current implementation, there are two steps to assign values to event.
Step 1: The returned object of getOptions contains all options of createDOMEvent and the object is passed to Event constructor as second parameter. For MouseEvent, the object is recognized as mouseEventInit by which you can set clientX and other properties to MouseEvent at the stage of initializing it. clientX is setted at this step.
Step 2: Once event has been created, an iteration of options of createDOMEvent is triggered to assign key-values to the event. During the iteration, setter of event[key] is checked to ensure assignment is only applied to wrtitable field.
At the first step fields which are valid for event constructor are accepted and the second step is to ensure rest fields are attached to the event as well. It works well for MouseEvent but not for DragEvent. As I mentioned above, the setter check algorithm is not suitable for DragEvent because DragEvent inherits properties from MouseEvent. clientX is readonly property of MouseEvent as well as DragEvent but Object.getOwnPropertyDescriptor(dragEvent, 'clientX') could not handle it. getOwnPropertyDescriptorRecursively is a more proper for this situation.

Interesting, you have researched this really well (I don't know a ton about event properties).
If you want to make a PR with this change, that'd be great. We likely will need to have something similar in Vue Test Utils v2. I could port over anything you bug fix here.

Interesting, you have researched this really well (I don't know a ton about event properties).
If you want to make a PR with this change, that'd be great. We likely will need to have something similar in Vue Test Utils v2. I could port over anything you bug fix here.


expose only webhooks from funnel

Proposed Changes
This allows exposing only webhooks through the funnel and keeping the same external URL for the devices with Tailscale.
This is my first change to any hassio addon, so please bear with me :) I'm opening the PR to have a conversation and see if this is something you'd like in or I should maintain separately.
Related Issues
Tailscale itself does not support ACL for funnel yet, so this is a workaround:

https://github.com/tailscale/tailscale/issues/13109

Summary by CodeRabbit

New Features

Introduced a funnel_protection configuration option for enhanced security control.
Added Nginx configuration for secure access to the Home Assistant service, including IP whitelisting and webhook access.

Bug Fixes

Adjusted service port configuration to ensure proper communication with Home Assistant.

Documentation

Updated configuration templates to reflect new options and settings.

I'm just another contributor.
Isn't it enough, to use a different line for "webhook only access"? Without running it through nginx? Something like (not tested):
exec /opt/tailscale ${tailscale_command} --https=$(bashio::config "proxy_and_funnel_port" "443") --set-path=/api/webhook "http://<IP_ADDRESS>:$(bashio::core.port)/api/webhook"

And the config switch means "full access to / or just /api/webhook"?
But the big question is: when (if ever) will TS provide a UI for serve/funnel config???

But that would expose only the webhooks, right?
I want to have mobile devices and laptops to be able to connect to Tailscale and use Home Assistant. Ideally that should be automatic with the VPN on demand and using the MagicDNS.
Exposing only webhooks would not do that and even when connecting to tailscale they'd not be able to use it automatically for a few reasons:

Home Assistant has 1 external URL shared for webhooks and devices - it needs to be the same for both
Devices would try to access https://something.ts12345.ts.net, because it is the external URL
Devices would need to access http://something.ts12345.ts.net:81234 when connected to Tailnet only

Right now you are outside, open HA, it either connects VPN automagically or in the worst case you get 403, connect Tailscale and it works.

OK, I clearly miss something.
Currently you can access https://something.ts12345.ts.net/ and  https://something.ts12345.ts.net/api/webhook through funnel when your device/laptop is not connected to VPN (you get an external IP resolved), and you can use the same urls (yes, httpS:), when it is connected to VPN (you get a 100.x.x.x address, funnel always includes a serve in the background).
So, what is the bottleneck you want to eliminate? Or add, in case of funnel_protection?

Exposing Home Assistant through funnel poses a security risk.
There is no WAF, no rate limiting, no layer to protect you from bots. If there would be Home Assiatant 0-day it could be a significant problem.
For example I see in the logs scans by bots the second I expose the funnel.
Probably the naming is not right and it should be something like funnel only webhooks - because that is what it does in the end result.

I think I understand, expose HA+webhooks with serve within the tailnet and only webhooks with funnel?
I've made some experiments, and TS serve/funnel config API is completely broken (nothing new), when we try to expose only webhooks with funnel, everything will be exposed. So we need an nginx proxy to filter traffic from funnel. In this case I think it is better to use an NGINX add-on behind funnel (in place of HA), and not replicate functionality here.
The question in this case is how to redirect this TS addon to another port, not directly to HA. Can't be done currently with config options. Maybe we should merge (breaking change) the current (a bit ugly) proxy and funnel config options with this use-case, a select/list for "serve where":

disabled
tailnet (proxy)
tailnet and internet (funnel)
And add "serve what" port config option, and when it is empty, HA will be served.

But I'm not the owner of this add-on, if we receive a green light for this direction, I can make a PR.
Meanwhile you can disable the proxy/funnel options in the config, and execute the tailscale funnel command with --bg toward the NGINX add-on port, it will be part of the TS state and will work after a restart or even add-on restore.

The nginx addon primarily focuses on TLS from the documentation - https://github.com/home-assistant/addons/blob/master/nginx_proxy/DOCS.md
Bending it to do what is done here seems a bit against the point of the addon. And the config needs you to set up TLS.
Alternative is Nginx Proxy Manager - but that is a pulling a full product just for a small thing.
Given the alternatives I'd rather see the support here then elsewhere. But I agree that being able to configure what you want to expose could be a very useful feature (not the localhost:8123 but for example localhost:80) to make it compatible with the Nginx Proxy Manager (in case someone wants a full control over the config).
I'll think a bit about how to expose this webhook only configuration flag better. Having a dropdown could make a lot of sense.
But configuration wise that would be a breaking change.

none
proxy
proxy / funnel (webhooks)
funnel

I agree, this 4 bullet point option was my first suggestion, before finally suggested the 3 bullet point + configurable port (external nginx).
I'm trying to merge this 2, but couldn't come up with anything elegant. Any idea appreciated. My problem: when we specify a port, we should not see the "proxy (UI, API) + funnel (webhooks)" option and vice versa.

If we break the current proxy and funnel binary config options, add it a future-proof way (at least try). I assume a lot of people use external nginx, they would need a port option here or exporting certificates (but cert export is not future proof, see #280), so port for now.


Error fixes

Fixes two errors that prevented correct usage in many cases:

enable numeric overrides in the measurement table (a test had screwed this up by setting everything to nan)
create parameter scales table correctly, for each condition

Codecov Report

Merging #51 into master will increase coverage by 0.26%.
The diff coverage is 85.71%.

@@            Coverage Diff             @@
##           master      #51      +/-   ##
==========================================
+ Coverage    65.8%   66.07%   +0.26%
==========================================
  Files           4        4
  Lines         389      392       +3
  Branches       75       75
==========================================
+ Hits          256      259       +3
  Misses        124      124
  Partials        9        9

Impacted Files
Coverage Δ

petab/lint.py
47.41% <100%> (+0.92%)
:arrow_up:

petab/version.py
100% <100%> (ø)
:arrow_up:

petab/core.py
73.52% <75%> (+0.09%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c3b9840...e4bd6b8. Read the comment docs.


Convert system.properties to set

Currently the data is stored as an object of booleans, which is a much more verbose data structure than a set but conveys the exact same information. Converting this data to a set would reduce the amount of data stored for every single item in the system and make accessing the property data a bit more logical.
Since we are already performing some breaking changes to properties with #2636, this would be a good time to sneak in this change.

I suppose they're not strictly equivalent. If I think about generating a set of checkboxes then I can know to even create a checkbox based on the existence of an entry in the object, and then I know the checkbox's value based on the entry's value. With a set we have to get the first piece of information externally. Are we already doing that maybe? In which case I suppose they're pretty equivalent.

We are effectively doing that by using the initialKeys option on MappingField to ensure the properties array always has the properties defined in config. This would just move that process from the data model to the sheet.


make-based/include: remove bashisms

make-based scripts (except docker functions) tested with dash
For fun and no particular profit :)

@manufactory, please solve the conflict.

Hackathon points: 30

Please update the commit message to use full sentences:
make-based/include: Remove bashisms

Update make-based scripts (except docker functions) to use `sh` / `dash` instead of `bash`. This improves portability.