SQLCipher v4 incompatibility

Hi !
While troubleshooting an issue  (Willena/sqlite-jdbc-crypt#25) on my repo, I came to the conclusion that there is something wrong when using SQLCipher and SQLiteMC together.  SQLiteMC cannot read databases created with SQLCipher and SQLCipher cannot read database created by SQLiteMC.
I did some tests myself with a fresh build of SQLCipher and using 1.0.1 binaries of SQLiteMC.
Here is the scenario I used:

I created a new db named fromCipher.db using SQLCipher:

SQLite version 3.33.0 2020-08-14 13:23:32 (SQLCipher 4.4.2 community)
sqlite> .open fromCipher.db
sqlite> pragma cipher_default_settings;
PRAGMA cipher_default_kdf_iter = 256000;
PRAGMA cipher_default_page_size = 4096;
PRAGMA cipher_default_use_hmac = 1;
PRAGMA cipher_default_plaintext_header_size = 0;
PRAGMA cipher_default_hmac_algorithm = HMAC_SHA512;
PRAGMA cipher_default_kdf_algorithm = PBKDF2_HMAC_SHA512;
sqlite> pragma key='abc';
ok
sqlite> CREATE TABLE "T1" (
	"Field1"	INTEGER,
	"Field2"	INTEGER,
	"Field3"	INTEGER,
	"Field4"	INTEGER
);

I created a "SQLCipher" database named fromMC.db  with SQLiteMC

SQLite version 3.33.0 2020-08-14 13:23:32
sqlite> .open fromMC.db
sqlite> pragma cipher = 'sqlcipher';
sqlcipher
sqlite> pragma kdf_iter = 256000;
256000
sqlite> pragma fast_kdf_iter = 2;
2
sqlite> pragma hmac_use = 1;
1
sqlite> pragma hmac_pgno = 1;
1
sqlite> pragma hmac_salt_mask = 0x3a;
58
sqlite> pragma legacy = 4;
4
sqlite> pragma legacy_page_size = 4096;
4096
sqlite> pragma kdf_algorithm = 2;
2
sqlite> pragma hmac_algorithm = 2;
2
sqlite> pragma plaintext_header_size = 0;
0
sqlite> pragma key = 'abc';
ok
sqlite> CREATE TABLE "T1" (
   ...> "Field1"INTEGER,
   ...> "Field2"INTEGER,
   ...> "Field3"INTEGER,
   ...> "Field4"INTEGER
   ...> );

Try to open fromMC.db with SQLCipher (it fails)

SQLite version 3.33.0 2020-08-14 13:23:32 (SQLCipher 4.4.2 community)
sqlite> .open fromMC.db
sqlite> pragma cipher_default_settings;
PRAGMA cipher_default_kdf_iter = 256000;
PRAGMA cipher_default_page_size = 4096;
PRAGMA cipher_default_use_hmac = 1;
PRAGMA cipher_default_plaintext_header_size = 0;
PRAGMA cipher_default_hmac_algorithm = HMAC_SHA512;
PRAGMA cipher_default_kdf_algorithm = PBKDF2_HMAC_SHA512;
sqlite> pragma key='abc';
ok
sqlite> select 1 from sqlite_master;
Error: file is not a database

Try to open fromCipher.db with SQLiteMC (it fails)

SQLite version 3.33.0 2020-08-14 13:23:32 (SQLCipher 4.4.2 community)
sqlite> .open fromCipher.db
sqlite> pragma cipher = 'sqlcipher';
sqlcipher
sqlite> pragma kdf_iter = 256000;
256000
sqlite> pragma fast_kdf_iter = 2;
2
sqlite> pragma hmac_use = 1;
1
sqlite> pragma hmac_pgno = 1;
1
sqlite> pragma hmac_salt_mask = 0x3a;
58
sqlite> pragma legacy = 4;
4
sqlite> pragma legacy_page_size = 4096;
4096
sqlite> pragma kdf_algorithm = 2;
2
sqlite> pragma hmac_algorithm = 2;
2
sqlite> pragma plaintext_header_size = 0;
0
sqlite> pragma key = 'abc';
ok
sqlite> select 1 from sqlite_master;
Error: file is not a database

Both test databases files are available here test_databases.zip
Any thoughts or ideas ?
Note : SQLite Version is 3.33.0, SQLiteMC version is 1.0.1, SQLCipher is 4.4.2
Note : SQLiteMC can write and read to it own created database as well as SQLCipher.
Note : Bug is also present in latest version (SQLiteMC 1.1.0)
This issue prevent users coming from SQLite tools like DB Browser (which support SQLCipher) to use their database with their java code (in the use case of my JDBC connector).

Thanks for reporting the issue.

Note : SQLite Version is 3.33.0, SQLiteMC version is 1.0.1, SQLCipher is 4.4.2

The default configuration parameters for SQLCipher 4 did not change, only the underlying SQLite version was updated.

Note : SQLiteMC can write and read to it own created database as well as SQLCipher.

The big question is, which cipher configuration parameters SQLite3MC effectively used.

Edit : Bug is also present in latest version (SQLiteMC 1.1.0)

Obviously all versions based on the new implementation of the encryption extension are affected, unfortunately. Well, there is a reason I call this project still work in progress ... 😞

This issue prevent users coming from SQLite tools like DB Browser (which support SQLCipher) to use their database with their java code.

Yes, I fully understand that. Be assured that we will find a solution soon.

Compatibility with previous versions of SQLCipher might need investigation / verifications (or a displayed warning if not compatible).

I just made a few experiments myself. It seems that the PRAGMA interface for manipulating the cipher configuration is somehow broken. If I use the C API (as in wxSQLite3) or the SQL functions (SELECT sqlite3mc_config(...);) all works as expected and I was able to open original SQLCipher databases (even the master.db file provided by @netmikey.

Any thoughts or ideas ?

Well, I will have to analyze what's going wrong when setting cipher configuration parameters via the PRAGMA interface. I will look into this issue within the next few days.

I just made a few experiments myself. It seems that the PRAGMA interface for manipulating the cipher configuration is somehow broken. If I use the C API (as in wxSQLite3) or the SQL functions (SELECT sqlite3mc_config(...);) all works as expected [...]

Nice to know !
I'm already working on a fix for the issue and will provide it within the next couple of days.
BTW, when using the legacy configuration parameter, the order of PRAGMA commands matters. That is, if you want to change some of the default settings of a legacy cipher scheme that must be done after setting the legacy parameter value.

Commit e27c041b298d9afae56c5947f881cedbb2e20e2a should fix the issue.
@Willena:
I tested it successfully myself with original SQLCipher 4 databases. However, I'd like to ask you to test it, too, and report back. After confirmation that the issue is indeed fixed, I will make a new release. TIA.

@utelle I tested on few databases I had and things are working now 😄 !
Thanks for being that quick at answering issues

BTW, when using the legacy configuration parameter, the order of PRAGMA commands matters. That is, if you want to change some of the default settings of a legacy cipher scheme that must be done after setting the legacy parameter value.

Is it written somewhere ? I don't remember reading it in the documentation. I made changes to my code as it did not follow the right execution order for pragmas...

I tested on few databases I had and things are working now 😄 !

Great. Thanks for confirming. I will then make an official release tomorrow (or the day after).

Thanks for being that quick at answering issues

You are welcome. Thanks for reporting such issues.

BTW, when using the legacy configuration parameter, the order of PRAGMA commands matters. That is, if you want to change some of the default settings of a legacy cipher scheme that must be done after setting the legacy parameter value.

Is it written somewhere ? I don't remember reading it in the documentation.

Well, it is not yet mentioned in the documentation, but certainly should be mentioned. Currently only the SQLCipher scheme is affected, because it is the only one, for which several versions (with differing defaults) are supported.

I made changes to my code as it did not follow the right execution order for pragmas...

For selecting the default variants of SQLCipher scheme versions you only need to issue the PRAGMA legacy=<SQLCipher version> command, because under the hood it will set the defaults correctly. Only if one wants to change the defaults, additional PRAGMA commands are required - and it is only logical that they have to be issued afterwards. That is, typically you need only 2 PRAGMA commands:
-- Select SQLCipher version 4 database encryption scheme
PRAGMA cipher='SQLCipher';
PRAGMA legacy=4;

If URI parameters are used to set cipher configuration parameters, the order doesn't matter, because the code takes care and checks for the legacy parameter first.

I made a new release, version 1.1.2, today, and I added a note to the PRAGMA legacy command documentation.


Feature: enable deduplication of d3-selection for shared browser events

d3-selection has global state (the event) that is obtained
using require("d3-selection").event. If multiple copies of this
js module exists in the frontend, they will not read eachother's
events.
This is required for 3rd party libraries, such as bqplot-image-gl
to be able to read the last d3 event.
Since Jupyter Lab will deduplicate direct dependencies, we only need
to add the d3-selection as an explicit dependency, and it will be
shared with other extensions.

@maartenbreddels LGTM should we merge?

Yeah, and do a release, with the latest bqplot-image-gl, we now need this, or get in the JS console:
Error: Shared module d3-selection doesn't exist in shared scope default

Since we don't ship it anymore, and bqplot is a provider of it.


SWAP of landmarks

Hi
I see in the data_augmentation function of train.cpp, there is macro SWAP. I am worried that it destroys the data integrity. May I know why this is being done ?
Thank you

Hi, what the code do in data_augmentation function is just flip the face. When your flip a face, all landmarks will change their position to (1-x, y), and symmetrical landmarks also will change their meaning. For example, left eye center is turned to be right eye center. The data has all landmarks in an order, 5 landmarks for example, left eye center, right eye center, nose, left mouth corner, right mouth corner, then, we need to swap the symmetrical landmarks after changing their position.
Hope you get the idea of why we need SWAP. :-)

Hey, Thanks for the quick reply. I do get the idea now :)


two authors with same ORCID

Hello
we end up with two authors associated to the same ORCID when processing this document : https://mdpi-res.com/d_attachment/foods/foods-11-00274/article_deploy/foods-11-00274-v2.pdf
in this case, the DOI was extracted so I think we should fully rely on the consolidated metadata, the annotations are not always reliable..
Cheers
Achraf

Hi Achraf !
Indeed, without consolidation we have the author "Elena Bozzetta" associated to the ORCID of the previous author, while she does not have an ORCID on this paper. We should prevent having one ORCID attributed to more than one person.
With consolidation everything is correct now, following your fix in #838


⚠ Bofrost.at has degraded performance

In e1f0a3b, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1757 ms

Resolved: Bofrost.at performance has improved in 8baee85 after 7 minutes.


AutoGluon issues on Mac OSX

Running AutoGluon on Mac OSX, you may see Segmentation fault: 11 errors whose stack trace points to libmxnet.so.  If you get such errors, please use Linux instead.  You can also try building MXNet from source with appropriate configuration options if you really want to use Mac OSX (which is not fully supported yet).  The error occurs because the current pip installed MXNet does not work properly with AutoGluon on Mac.  We are now working with the MXNet team to update the MXNet version to resolve this issue.
In particular, TabularPrediction tasks with hyperparameter_tune = True are known to produce this error, as are ImageClassification tasks.  However, TabularPrediction tasks with hyperparameter_tune = False should work on Mac OSX.
Instructions for building MXNet from source on Mac:  https://mxnet.apache.org/get_started/osx_setup

Pleas reassign

I have tested after latest HPO fixes and it appears that hyperparameter_tune=True is now stable on MacOS (Including for Neural Network). I have not built mxnet from source.
mxnet version: 1.5.1.post0

FWIW, I'm on macOS, built MxNet 1.6 from source with MKL, and used AG 6b247bfea9d504381cc512e36ba1909e6c54c0c3 I was able to run tabular-indepth without issue. Maybe this issue can be closed?

I'll add a clarifying statement to this issue: If you build mxnet from source, it works correctly. Issues only happen if mxnet is not built from source but instead pip installed. Even if you pip installed mxnet, it still works sometimes, but fails seemingly at random.

I got a data point recently that on mac the autogluon unittests will segfault 11 with

the pip installed mxnet
mxnet built from source with cmake

However, the unittests pass with

mxnet built from source with Makefile

Resolved in #920 , now default HPO process will no longer cause issues on MacOS.


Getting Started Help

I am a developer trying to integrate some React components with COGS but have been struggling with the limited documentation on this project.
Can you consider adding a few lines to assist other developers to outline what the project is for, and how the developer should be working.
For example, some core things I am unclear on - should I be colocated on the same LAN as the COGS server? Where do I set the IP address? Or does the 'simulator' act as a Cogs server?
Can I send receive data from Cogs (Eg hints) and can I send data (Eg lights) both via a React application? If not, what is this library for?

Hi @grahambates-code, thanks for the feedback.

I am a developer trying to integrate some React components with COGS but have been struggling with the limited documentation on this project.
Can you consider adding a few lines to assist other developers to outline what the project is for, and how the developer should be working.

Our docs are spread a little and we're working on unifying everything to make it easier to get started.
The quickest way to get set up is with our Create React App Template: https://github.com/clockwork-dog/cra-template-cogs-client
If you're setting up manually there is some help in the README in this repo and some additional setup details here:
https://docs.cogs.show/plugins/custom/

For example, some core things I am unclear on - should I be colocated on the same LAN as the COGS server? Where do I set the IP address? Or does the 'simulator' act as a Cogs server?

Yes, you should be on the same LAN. The Media Master and COGS communicate via local network IP addresses. The Media Master will be assigned an IP address by your network router. If the IP address of you Media Master changes COGS will handle this.

Can I send receive data from Cogs (Eg hints) and can I send data (Eg lights) both via a React application? If not, what is this library for?

Yes! The Custom Content example project shows you how to use "states" or "events" to communicate back and forth to COGS. https://docs.cogs.show/examples/#custom-content

Brilliant, I've got most of it working now combining this project and the one you mentioned, very very cool.


Add requirement for GNU grep on MacOS

Sudomy using grep tool for parsing any data from the source, but the default grep version in MacOS use BSD grep which is not compatible with Sudomy. So this changes will checking the existing grep version, if GNU grep not exist so will abort the script and mandatory to install GNU grep.

Nice, if someone have the problem with that. You can follow this issues https://github.com/Screetsec/Sudomy/issues/1


docs: remove not needed OS tabbing

PR Checklist

[x] Leverages formatting as provided by https://squidfunk.github.io/mkdocs-material/reference/abbreviations/
[x] Have viewed and verified edits in rendered form ensuring proper formatting

What are you changing?

Anything the reviewer should know when reviewing this PR?

I don't know if you want to put somewhere centralized a blurb about running rpc as administrator, that we assume rpc is the name of the executable and have one spot that might show sudo ./rpc amtinfo vs rpc.exe amtinfo


doc: release-notes: Add changelog for ARM64, cache, IPC

Add the changelog entries for ARM64, Cache API and IPC.
Signed-off-by: Carlo Caione<EMAIL_ADDRESS>

@carlocaione This PR now has conflicts.


[Feature][worker] make workers completely independent of the database

Search before asking

[X] I had searched in the issues and found no similar feature requirement.

Description
IMO, the worker server is independent of the database.
ATM is dependent.
Besides TaskPluginManager, are there any other functions that depend on the database?
If the specific task is not dependent, the changes will not be too much.
Use case
make workers completely independent of the database
Related issues
No response
Are you willing to submit a PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

@zhuangchong can you take a look and give some suggestion?

@huagetai
Currently the worker server is completely independent of the database. Maybe I didn't understand what you said. Would you like to express your design plan and ideas?

@zhuangchong
ATM:
public class TaskExecuteThread implements Runnable, Delayed {
    ...
     @Override
      public void run() {
           ...
           TaskChannel taskChannel = taskPluginManager.getTaskChannelMap().get(taskExecutionContext.getTaskType());
           ...
      }
}

@Component
public class TaskPluginManager {
    ....
     private final PluginDao pluginDao;
     ....
}

In addition, due to the dependence on the dolphinscheduler-service package, spring will automatically inject the datasource


Linting for enum types with string values

Bug Report

TSLint version:
TypeScript version:
Running TSLint via: (pick one) CLI / Node.js API / VSCode / grunt-tslint / Atom / Visual Studio / etc

TypeScript code being linted
Currently from TypeScript 2.4 we have the options to use enums like:
enum Colors {
    Red = "RED",
    Green = "GREEN",
    Blue = "BLUE",
}

This however gives still gives the warning even with the newest version ot the linter:
[ts] Type '"RED"' is not assignable to type 'Colors'
with tslint.json configuration:
Standart tslint file generated from angular-cli.
Is there an option to fix this or is it an option that is coming
Actual behavior
Expected behavior
Do not show this warning.

I don't really understand your problem description. Where is the error located, it this the whole repro code, what version of typescript are you using?
From what I understand I would say that this an error coming from typescript rather than tslint.
I'd guess you have the wrong version of typescript installed.
When I last used the angular-cli to generate a project, it didn't support<EMAIL_ADDRESS>and used<EMAIL_ADDRESS>instead to compile my app.

+1 to this issue.
Following angular/angular-cli#6827 . It tells that you can upgrade Rxjs to 5.4.2. However, at this time on package.json, the version used is ^5.1.0, so 6.0.0-alpha.0. Theoritically, you just have to set typescript to ~4.1.0 . It compiles, but tslint displays the enum string syntax as a warning. Futhermore, Typescript 4.2 is unstable with angular-cli, so except for an experimental project about it, you should not use it.
Waiting this issue fixed (and waiting the support on angular-cli), you can use this sexy trick to implement your own string enum : http://angularfirst.com/typescript-string-enums/

@MLobry looking at the linked angular-cli issue, it seems like<EMAIL_ADDRESS>is now supported.
Please update typescript to 2.4.1 to use string enums.
If that doesn't help, I'd need an example repository to reproduce the error.

closing as this is not a tslint-related issue

Of course this is a tslint issue.  The error says as much.  I'm on TypeScript version 2.5.3, TsLint 5.7.0, and cannot assign string values to enums exactly as described above.

@banderberg We need the exact error message, the rule name and the code that's causing it. Otherwise we can only guess what's wrong.

Hi @ajafff , here is exact error I can see in hint:

TS2322:Type '"REQUESTER"' is not assignable to type 'Roles'.

For next code:
export enum Roles {
    Requester = 'REQUESTER'
}

Typescript 2.6.1
Tslint 5.8.0

That is a typescript compiler error. You can tell because the error string begins with "TS".


Public test server JDBCConnectionException

NOTE: Before filing a ticket, please see the following URL:
https://github.com/hapifhir/hapi-fhir/wiki/Getting-Help
Describe the bug
When I try to access the resources in the public server as https://hapi.fhir.org/baseR4/Patient/, I am getting the exception below:
"HAPI-0389: Failed to call access method: org.springframework.transaction.CannotCreateTransactionException: Could not open JPA EntityManager for transaction; nested exception is org.hibernate.exception.JDBCConnectionException: Unable to acquire JDBC Connection"
To Reproduce
Steps to reproduce the behavior:

Go to 'Patient resource end-point'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Environment (please complete the following information):

HAPI FHIR Version
OS: [e.g. iOS]
Browser [e.g. chrome, safari]

Additional context
Add any other context about the problem here.

Any chance that this will be fixed soon?

This was fixed


Fix ssh-agent for SCP

This should improve ssh-agent for SCP.
Closes #270

Same as the last PR that did this - we really need tests #272 :disappointed:


feat: Use AWS aurora v2

@bbernays I need some help with that. Looks like the aurora v2 is working but Im not able to access it from quicksight, tried to follow this guide - https://dev.to/aws-builders/connect-quicksight-to-rds-in-a-private-vpc-4nl9 but got lost.

For cost purposes given CQ workload it is best to use standard aurora rds (not serverles) + spot instances


Documentation

Shouldn't this read this "Technically, if you sent a list of 100 different feed URLs to the  FeedValidatorActor  then it would create 100 different  FeedValidatorActor  instances to process each feed in parallel."
as
"Technically, if you sent a list of 100 different feed URLs to the  FeedValidatorActor  then it would create 100 different  FeedCoordinatorActor  instances to process each feed in parallel."

yes, thank you

thanks for the note. it actually creates instances of the FeedParserCoordinator, I'll update that now


[compat] Add support for 'lastResort' handler introduced in Python 3.2

Fixes https://github.com/mapproxy/mapproxy/issues/499

Could someone please review this PR?

is this still an issue?

should be resolved by removal of python 2 support


Issue#135

Description of changes
Changed the  component as well as signup.js to allow an "enter" key down submittal of the form in addition to the normal button click submittal.
Issue Resolved
This solves issue #135

I like the idea of handling it as a property, like @alexspence said.

I changed all found instances of FormButton to onSubmit which I think is best practice for form UX. A user should be able to click or press enter to submit a form anywhere on the site.


In pruning_main.py best_mar12_3.pt should be saved as webqsp.pt?

In EmbedKGQA/KGQA/RoBERTa/ there are two main programs. In pruning_main.py the commented out saved model is best_mar12_3.pt, is this same as webqsp.pt, ?
Should the Roberta model be saved as webqsp.pt?

best_mar12_3.pt is same as webqsp.pt,


Failing operation when panic occurs in the step

Description
Changes proposed in this pull request:

RunStep in staged_manager handles panic
RunStep in upgrade_cluster/manager handles panic
RunStep in upgrade_kyma/manager handles panic

Related issue(s)

/test all

/retest

/retest


geoip: set default dir to '/var/lib/geoip-databases'

Motivation for this change
Set the default directory to '/var/lib/geoip-databases', the default directory of geoip-updater service.
It might be not the perfect choice, but

current default directory is ${geoip}/share/GeoIP which is in read-only nix store, it has no files nor possibility to put them in.
some software just use the default and do not allow to customize the geoip directory

Things done

[ ] Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] Linux

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Fits CONTRIBUTING.md.

Won't this (now) clash with the geoipDatabase handling in postInstall? I think perhaps this path should be an argument to the derivation.

@bjornfor maybe like this? https://github.com/NixOS/nixpkgs/pull/27412/commits/04de1eda1f36c8de48abf091839a47b371cd3e6a

@volth: No, that's not what I had in mind. I was hoping for a derivation argument, like the existing "geoipDatabase". Even better if the two can be one. So "geoipDatabase" could default to "/var/lib/geoip-databases", but could also be set to a derivation providing databases (like the geoipWithDatabase package does with geolite-legacy).

@bjornfor I do not see how they could be united into one option, current geoipDatabase is a build-time option, the source location of database to copy in into nix store (and keep read-only and not updatable), while the new option would be a run-time writable directory.
If geoipDatabase would default to "/var/lib/geoip-databases", that means at build time, the current version of updateble database would be copied to the nix store (perhaps even with temporary files if if happens to overlap the update process) thus making the geoip derivation not deterministic.

not if you put it in quotes "/var/lib/geoip-databases" vs /var/lib/geoip-databases

@Mic92 sorry, not "copy", but "symlink"
  postInstall = ''
    DB=${toString geoipDatabase}
    if [ -n "$DB" ]; then
      ln -s $DB/share/GeoIP $out/share/GeoIP
    fi
  '';

if geoipDatabase would be "/var/lib/geoip-databases", ${geoip}/share/GeoIP would be a silly symlink from nix store to "/var/lib/geoip-databases/share/GeoIP"

if you really wants is as param.... https://github.com/NixOS/nixpkgs/pull/27412/commits/0f2f779f4ada890629c25a8ec4355eeee2de31fc

I think this parameter does not provide additional value. @bjornfor actually I don't why it clashes in the first place. /var/lib/geoip-databases was only used, if no database file was passed to this package.

@Mic92: I think I got the @bjornfor idea: https://github.com/NixOS/nixpkgs/compare/master...be71e3895d171df5656e96dc448350e0248a5e04

@bjornfor does current PR match your idea?

Yes.

Let's merge?

Applied to master (8b4f5376ff1f1ad6898febe99941bd9a4c890f6b).


Lapis revive units

The pwnology TM farming scripts will auto revive with lapis according to Mcgillby's comment here: https://www.reddit.com/r/FFBraveExvius/comments/62x2vg/custom_tm_farming_scripts_for_memu/dfu6kkx/
We should confirm the bug and see what can be done to fix.

It shouldn't be too hard to confirm the bug, i'll go to a hard dungeon and let the original tm macro script run, give me 5 minutes

Cool, thanks. And if you manage to identify the coordinates causing the issue, well, that's bonus points!
Then we're also going to have to figure out what event those coordinates are meant to be there for. There's a lot of eventualities that pwnology and Kitthix are accounting for (which is half the reason it's so worthwhile to be fixing their scripts).

Bug confirmed, just used 100 lapis to revive units in mission. I think the click is the one supposed to say YES when you want to refill lapis, but for the coordinates... i don't think i can help you here. Here's a screenshoot i've taken of the reviving screen. the button pressed in the screenshoot are the ones normally cycled, when the script try to refill lapis once in a couple of runs, it hits the YES button. Technically, if we can avoid the click that presses the AUTO button we would be good, because having the units 1 and 3 to attack, the others will remain idle, not ending your turn and actually "pausing" the battle

Any chance you can you give me a screenshot of the lapis refill screen so I can overlay them?

here it is screenshoot

Is there another screen after you hit 'Use Lapis'?

i'll make one right away, but i don't have any lapis right now on memu

You're right, i forgot there was another confirmation screen, here it is

Well i'm off to bed for now Kas, it's 2 am in the morning here :) i'll leave the fixed macro run all night and see if i get some strange behaviour in the morning. I'll get back to you asap for everything that i'll read here. We should decide something for the mcgillb situation, too. later!

Thanks for all the help!

Ah no problem, i'm here to help!, anyway, i've noted a strange thing. the tm macro, every 4 or 5 cycles (when he's trying to refill lapis, basically), hits the AUTO button, which is probably the source of a good chunk of our problems. I've digged a little in the old thread and i've found this
Maybe we should just remove it? it's been causing only troubles in the last couple of days. And now, goodnight for real :)

That's the kicker, is that I know why Pwnology put it there and I'm trying to think of a truly viable alternative that deals with the issue he was fixing without introducing another issue altogether.

But I don't understand, what's the problem if 2 characters attacks the same monster? It will not mess up the macro since it's modular, the monsters usually dies with one hit.

The problem occurs when there are two monsters and what is called a race condition occurs. A race condition means the second unit is set to attack before the first unit has registered as attacking and killing the monster, meaning both attack the same monster and then without an "AUTO" click or some other means of fixing the problem, the units just stand there infinitely staring down the second monster.

Hm, I was not aware of this problem honestly, now the auto click make sense. What if we sightly delay the two units attack clicks? This will slow down farming a bit, but it can be an acceptable compromise if the alternative is a potential loss of lapis

What if people just turned off Continuous Auto Battle? The issue could still exist but would be so much less potentially devastating.

Could work, but in my test run before, the tm party was wiped in a single encounter. Given the frequency of the auto button getting pressed, we would still have a lot of problems

Mcgillby solved this by eliminating the auto press and replacing it with
another unit attacking. Might be the best idea. I just wish there were a
good way to fix a crash to the main screen, which is what puts us in this
shitty situation in the first place.
On Apr 4, 2017 9:01 PM, "biribussi"<EMAIL_ADDRESS>wrote:

Could work, but in my test run before, the tm party was wiped in a single
encounter. Given the frequency of the auto button getting pressed, we would
still have a lot of problems
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/kastyr1984/FFBE-Macros/issues/3#issuecomment-291695214,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ARPJh1Q3lQfe0CGvEJDCvC1NxgceBJkCks5rsuf2gaJpZM4Mzjjy
.

hey guys mcgillby here. ive fixed all the issues with the script. as soon as that script gets fixed or replaced I will be nice. I really take this shit seriously. im sorry about freaking out but this should have been fixed a long time ago. anyways ive fixed the script and patched the errors. i still feel like we should work together and make  a single combined effort. i dont mind going at it myself either. anyways here it is
https://pastebin.com/d12VadNc

biribussi, I appriciate that your using your lapis and testing it, it shows me that you take this seriously. You can clearly see how dangerous this build can be (I still handled it wrong I know) and I couldnt be more happy to see you guys taking the steps to fix it.

You're right, i forgot there was another confirmation screen, here it is

Seems to me that the USE LAPIS for refilling and the YES from reviving are pretty much in the same position, probably that's click causing the issue. I noted that reviving units does not have a second confirmation screen like the refill one
that is not the confirmation screen  this is The button placement is slightly higher then the screenshot given by biribussi

a suggestion for further testing is too start a new account, you can do the story for about an hour and have 5000 lapis. and you get too watch the refresh happen every 20 runs. when you run out of easily farmable lapis, throw it away and start a new account.

I'm gonna go out on a line and say that biri didn't mind spending his 100 lapis for the cause but now that you mention it I might send a donation his way to cover some testing costs. Good idea. Back to bed now!

if we still have a button press over revive, there is still a miniscule chance that someone will start it with continuous auto battle on in the wrong content ( its happened to me before, starting in the wrong content, not auto-battle I never lost any lapis, just a night of farming) if the button press for revive happens, it will continue until 0 lapis, it needs to be fixed. you want to move it to hit the top of the confirmation button, but still miss the espers on the depart screen.

goodnight, i got the day off work tomorrow so im going to do some more testing

I just wish there were a
good way to fix a crash to the main screen, which is what puts us in this
shitty situation in the first place.

Crashes are... tricky to work around. I studied them some months ago working with team pwnology, my conclusion at the time was that crashes are a combination of natural emulator instability, poor software optimization and a memory leak caused by the app-emulator system. I've recently found some advanced settings that would allow a better management of the ram menory used by the emulator, maybe we'll find a way to reduce them. I'll look into it


Import file from a child directory

It seems import statement does not support importing files from a child directory:
$ risor -c "import test/foo"
parse error: invalid expression

location: line 1, column 12
import test/foo

Is this a bug or a different syntax is expected?

@dnbsd this may work for you:
risor --modules test -c "import foo"


Left mouse click on the left part of the button "Default sorting" doesn't toggle sorting modes cyclically

Description

This kind of UI element is usually used to toggle between modes if click was done on the left part of the element (next sorting mode in a cycle should be taken).
If user wants some certain mode, he still can click small triangle and choose the certain mode from the list.
But sometimes it is eaisier "to switch to the next mode cyclically", it is expected and it is not available now.
It simplifies usage with the item, I suggest to add this behavior.

The second part of the issue is that it would be nice to change the image of the icon slightly on each click to indicate the current mode for a user.
He would see the current mode clearly on the button itself.
Similarly how it is now done on a hotkey Ctrl + 2 when some cell is selected in a Result set: the icon with Up/Down arrow is appeared, indicating the current mode of sorting.

https://github.com/user-attachments/assets/389883e1-5e6d-468f-aaae-ba43df9ecf13
DBeaver Version
Community Edition 24.1.5-202408122322
Operating System
Windows 10
Database and driver
PostgreSQL 16
PostgreSQL JDBC Driver 42.7.4
Steps to reproduce
No response
Additional context
No response

Most probably it wasn't supposed that this button would be clickable. We want to disable clicking on it or make the click on the button change the sorting and button icon.


Sidebar on the left is buggy, hexo hey customization fonts suggestion

Sidebar Bug
Whether in Windows (10 · Chrome)or Mac OS (El Captain · Chrome), after I click the posts on the sidebar, the sidebar become so strange on the web page.
Here is the screenshot on the demo.

Customization Font
As for another thing, I suggest to develope a optional configuration which is users could customize their favorite font for hexo hey. When I engaged in Chinese language, I found out the default font is not as good as for the UI design, I hope I can config it as my own.

Sorry about my English.

I've test the ui in Windows and Mac OS both, and i can't reproduce it now, if so, it would be an issue for angular-materiel.
For the second, the config section only custom the back-end behave so i can't do that, but i will change the font-family in the front-end directly :smile:

Is there any possibility that I can fix the Angular Material issue?
As for fonts, where can I find the font-family?


Adding github workflows for both clang-format and google tests

Added 2 github workflows:

clang-format.yml automatically formats every source file in the codebase on pushes to main that modify the codebase.
google-tests.yml automatically runs the exec with google tests. Will run all tests added by the users in the future and refuse to push the commit if the tests failed. (Work on all branches but only on codebase/assets/tests changes)
Workflow tried and tested.

Excluded pull-requests from clang-format workflow


[Suggestion] Support hyphenation

Since Gemini is all about text, I believe readability would be greatly enhanced if Lagrange supported hyphenation for paragraph text.
I am not sure how much work this is though.

Hyphenation is not an easy feature to implement, as it depends on reliably knowing the language of the text and there also needs to be a database of hyphenated words or an algorithm that determines where to put the hyphens. One could require the lang MIME attribute to be set, but unfortunately that is not extremely widely used.
There are probably existing open source libraries that handle some or all of this. Adding one as an optional build dependency might be doable.
In the meantime, it is possible to use the MIME hooks feature to filter text through an external tool that inserts Unicode invisible hyphen characters in appropriate places. (Although I'm not sure if they are currently used correctly by the HarfBuzz-based text renderer.)


fix(OptionsTile): fix styling of summary and chevron when nesting options tile

Closes #5560
Change selectors for the OptionTile's summary and chevron elements to use BEM modifiers.
What did you change?

change the selectors for the CSS rules applicable to summary and chevron to use BEM modifiers, instead of looking at parent elements to determine the open and closing states.

How did you test and verify your work?

The effects of the fix are visual, so (manually) compared how the chevron looks like and behaves during opening/closing, verified that the summary is visible on a nested options tile when it is collapsed.

I have read the DCO document and I hereby sign the DCO.

recheck

Sorry for the churn. I hope I got my user IDs sorted out now.


New Version: OO-Software.Defrag.Professional version 28.1.10015

Checklist for Pull Requests

[x] Have you signed the Contributor License Agreement?
[ ] Is there a linked Issue?

Manifests

[x] Have you checked that there aren't other open pull requests for the same manifest update/change?
[x] This PR only modifies one (1) manifest
[x] Have you validated your manifest locally with winget validate --manifest <path>?

[x] Does your manifest conform to the 1.6 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

=== manifests\o\OO-Software\Defrag\Professional\28.1.10015 === Missing Properties value based on version 16.0.367: Installers

What does this mean?


Create: Temperature converter

Create script which with passed temperature and its unit, outputs desired converted value.
Don't pass degree character to input.
Save script as temperature_converter.py in Source directory.
If conversion is not possible display proper message:

temperature is below absolute zero (0 K, -273.15 °C, -459.67 °F) message: temperature is below absolute zero, conversion is not possible
value of temperature is a non numeric value (anything other than numbers and ".") message: format of entered temperature is wrong, not a number
unit is not recognized (anything other than C, F, K) message: unit not recognized

Input
temperature - numeric value of temperature (floating-point) (i.e. 23.65)
unit 1 - string, unit to convert from (i.e. C - Celsius, K - Kelvin, F - Farenheit)
unit 2 - string, unit to convert to (i.e. C - Celsius, K - Kelvin, F - Farenheit)
Output
temperature unit  - temperature with an accuracy of two decimal numbers (i.e. 26.66), abbr. of unit with degree character (if needed) (i.e. °C, °F, K)
Example
255
K
C
-18.15 °C

To test if script is working properly use prepared test case.
Run:
python Tests\test_temperature_converter.py


Integrating awsssmchaosrunner with chaos-mesh

Here is a file called npm-debug.log, it is also should not belongs to this PR.

I read about awsssmchaosrunner and see familiar functions in chaos-mesh. Any chance that we may intergrate aws chaos in networkchaos and stresschaos and use a different selector?

https://github.com/amzn/awsssmchaosrunner/blob/e5524eeaef78dca85ac5a9b4a9634efae75df967/src/main/kotlin/software/amazon/awsssmchaosrunner/attacks/DependencyPacketLossAttack.kt#L20
This is a bit weird to me 🤦

Yeah, the core of awsssmchaosrunner is many shell scripts. So I wonder that if we could rewrite it someday.

https://github.com/amzn/awsssmchaosrunner/blob/e5524eeaef78dca85ac5a9b4a9634efae75df967/src/main/kotlin/software/amazon/awsssmchaosrunner/attacks/DependencyPacketLossAttack.kt#L20
This is a bit weird to me facepalm

Wow! Really familiar codes :laughing:


About Evaluating

Hi，
Thanks for your help. I just finished training with 4 x 3090. After evaluating, I see that the result has a big different from yours.
I wander if you can tell me where can I change the tokenizer settings and other hyperparameters.
Thanks!

Hi, I have the same question.
The VQ-VAE was trained for 200 epochs on 8 v100 GPUs. The batch size is 1 for per GPU. After training I got the results in the following:

Then for the second stage, the checkpoint of VQ-VAE (epoch_200.pth) was loaded. The OccWorld was trained for another 200 epochs on 8 v100 GPUs. After training I run the eval_metric_stp3.py and got the final results:

However, the result seemed to be different from that announced in the paper:

I have the same question

I also have the Nan problem when using 4 RTX 4090 with batch 1.
When I use 4 RTX 4090 with batch 2, I avoid the nan problem, but I can't get similar results.
I find only 8 RTX 4090 works fine, any suggestions? @wzzheng

Thank you for your interest! I'm very sorry for replying so late. I have provided the training and testing logs for the github repository code. The results maybe a bit different due to random numbers. I also provide the model used in the article and its evaluation log. Also, recommend selecting the best vqvae model for the occworld training.


MAINT: malloc checks for C source files

Adds checks in case malloc returns NULL in the C source files. Based on issue #9999

@makeroflight are you planning to continue work on this? Would be great if you could address the open comment.

Thanks for the contribution, @hrhwilliams.  This PR has been superseded by https://github.com/scipy/scipy/pull/10443 and https://github.com/scipy/scipy/pull/10451, so closing.

Thanks for the contribution, @hrhwilliams.  This PR has been superseded by https://github.com/scipy/scipy/pull/10443 and https://github.com/scipy/scipy/pull/10451, so closing.


Speed of auto scroll to end

I want to ask how to make scroll to end slow ?

Currently this is not possible, because we are using FlatList, which doesn't provide us the control over speed of scrolling to end.


Allow multi-line comments

Store the contents as non-html string, but when rendering preserve new
lines.

:+1:
We need to add the line breaks to the email notification too. Separate PR?


Chapter 8, (b : bs)?

Hello!
In the folding with monads section, there is this example code.
foldM f a (b : bs) = do
  a' <- f a b
  foldM f a' bs

As far as I understand, the (b : bs) syntax is no longer correct PureScript?

Yes, it's valid again, but it means something different. Previously it was used to destructure arrays, but now it's an alias for the Cons constructor on lists. The alias is defined in Data.List.


ssh keys sometimes not present

I am doing a build multiple times to tweak my system. Sometimes, i have noticed that the ssh keys in /etc/ssh/ are not generated. In one of the tries, there wasnt any key at all and therefore i wasnt able to login into the RPI.
For instance in my current build, i do see these messages
Creating config file /etc/ssh/sshd_config with new version
Creating SSH2 RSA key; this may take some time ...
2048 SHA256:Xt90dVJCurKQ5BD5Zvq7Z4B5pcQicvgaulKra/jo/tI root@debian (RSA)
Creating SSH2 ECDSA key; this may take some time ...
256 SHA256:RmIOfx6tdG3XO+3LkqzCka5vzOAdyhGGHdv0tyNAKIs root@debian (ECDSA)
Creating SSH2 ED25519 key; this may take some time ...
256 SHA256:RGoRqA6nhropKepp/nceXvPCGF/vnTHsaxSSZpn9flE root@debian (ED25519)
qemu: Unsupported syscall: 384
Created symlink /etc/systemd/system/sshd.service -> /lib/systemd/system/ssh.service.

However, after boot the RPI i dont see the ECDSA and ED25519 keys,
# ls -la /etc/ssh/
total 568
drwxr-xr-x  2 root root   4096 Oct 16 01:35 .
drwxr-xr-x 82 root root   4096 Oct 16 02:03 ..
-rw-r--r--  1 root root 553122 Jun 18 05:41 moduli
-rw-r--r--  1 root root   1723 Jun 18 05:41 ssh_config
-rw-r--r--  1 root root   3284 Oct 16 01:36 sshd_config
-rw-------  1 root root   1679 Oct 16 01:35 ssh_host_rsa_key
-rw-r--r--  1 root root    396 Oct 16 01:35 ssh_host_rsa_key.pub

Is this expected behaviour?
My plugins.txt is
plugins/Raspbian/copies-and-fills
plugins/Raspbian/kernel
plugins/Raspbian/rpitools
plugins/Foundation/apt-key
plugins/stretch/extrapackages
plugins/stretch/noPersistentNetGen
plugins/stretch/oldNetNaming
plugins/alsa
plugins/common
plugins/dbreconfig
plugins/disableroot
plugins/extrapackages
plugins/fake-hwclock
plugins/fsckboot
plugins/hostname
plugins/kodi
plugins/mythtvlight
plugins/resizefs
plugins/rpi-next
plugins/sshkeys
plugins/swap
plugins/tmpfs

they should never be present they are generated on first boot so you can use the image on multiple systems.  If it's bailing before the end they might still exist, but when then image is done they should never be there with the plugin/sshkeys being run.

Which script generates the keys on  the pi on firstboot?
It seems the key generation is a bit random. I have another instance here where the only key in /etc/ssh is ssh_host_dsa_key and ssh_host_dsa_key.pub. The rsa/ecdsa/ed25519 keys were not generated.
I get a 'Connection reset' when i try to ssh, probably due to lack of rsa host key.

it shouldn't be it's using the openssl script to build em
https://github.com/TheSin-/rpi-img-builder/blob/master/plugins/sshkeys/files/etc/systemd/system/sshd_check_keys.service

Maybe the sleep time is not enough,
ExecStart=/bin/sleep 15

Or too much and the restart happens too early?

On an idle system i get 6 seconds,
# time ssh-keygen -A
ssh-keygen: generating new host keys: RSA DSA ECDSA ED25519

real    0m6.646s
user    0m6.630s
sys     0m0.010s

Are the ExeStart scripts meant to run sequential or parallel. If it is the former than the sleep time should be more.

I made two builds,

ExecStart=/bin/sleep 30 produced rsa and dsa keys
ExecStart=/bin/sleep 8 correctly produced all the keys

So, shorter time is better.

wonder if the timer is required at all, the way systemd works I'm not sure it would be.  I took this script from some place maybe I should rework it.  I won't be able to run tests till next week but I think I'll just need to use preStart and Start

try that commit see if it helps with consistency and let me know

Unfortunately, the new changes made it worse. I dont get a key generated on the RPI.
Environment=TERM=linux
Type=oneshot
ExecStartPre=/usr/bin/ssh-keygen -A
ExecStart=/bin/sleep 10
ExecStartPost=/usr/sbin/service sshd restart
StandardError=syslog
RemainAfterExit=no

it wouldn't happen till the last reboot.
as it reboots it'll remove files from /var/run/system-...  sshkeys won't get till all those files are gone.  So maybe it needs one more reboot to gen, I'll revisit the logic, that last files might be getting removed after the keys try to gen but no reboot happens.
if that isn't it can you boot and run systemctl -l status sshkeys.service for me and paste the results.

There were no files /var/run/system-*. I rebooted but did not get any keys. Here are photos i captured after the reboot.

why is your fs ReadOnly??
it's trying to run but the fs is RO

I have no idea. This is a clean boot after dding the image. The dd should have wiped out the previous filesystem. It is surprising that the disk check has kicked in somehow.

did you enable the readonlyfs plugin?
Also fsck=yes is in the default codlin.txt, being and SD card I like to check every boot

I rebooted again, and this time it generated the keys.

No, readonlyfs is disabled.

okay just making sure.  Cause I'll have to rework that one for the changes in sshkeys ;)
I don't like that you had to manually reboot, I'll look deeper into the order


CDN in a Box: CentOS 8.3 fixes

What does this PR (Pull Request) do?

[x] This PR is not related to any Issue

This PR fixes problems in CDN in a Box caused by breaking changes in CentOS 8.3. The issue building the trafficportal CDN in a Box image has already been fixed in nodesource/distributions#1147.
Fixes the Error: Unknown repo: 'PowerTools' error that you will get when trying to build the trafficops-perl CDN in a Box image.
The centos:8 Docker image updated to CentOS version 8.3.2011 yesterday. CentOS 8.3 changed the PowerTools repo ID from PowerTools to powertools, and repo IDs are case-sensitive.
I couldn't find any mention of this in the Red Hat 8.3 release notes, though a related Fedora Websites documentation bug is still open.
If we really wanted, we could support CentOS 8 versions older than 8.3.2011 for trafficops-perl in CDN in a Box by doing --enablerepo=[Pp]ower[Tt]ools, but let's not.
Which Traffic Control components are affected by this PR?

CDN in a Box

What is the best way to verify this PR?

Pull a newer centos:8 image
docker pull centos:8

Build Traffic Ops Perl
docker-compose build trafficops-perl

If this is a bug fix, what versions of Traffic Control are affected?

master (c0126fca5d)
5.0.0 (RC2)

The following criteria are ALL met by this PR

[x] Only CDN in a Box is affected, no test beyond testing CDN in a Box is necessary
[x] Usage of CDN in a Box is unchanged, documentation is not necessary
[x] An update to CHANGELOG.md is not necessary
[x] This PR includes any and all required license headers
[x] This PR DOES NOT FIX A SERIOUS SECURITY VULNERABILITY (see the Apache Software Foundation's security guidelines for details)

Removed the fix for rpm.nodesource.com, as nodesource/distributions#1147 already fixes that issue.

I'm guessing the CI caching of perl dependencies hides this error.

That sounds correct. A quick search reveals that File::Stat does not exist but File::stat does. I wonder how long that's been the case.

The trafficops-perl image builds successfully for me with no caching. Still looking into this...

Yeah I blew away my centos7-based local dir, and it built successfully.


Popup incessantly: other applications manage brightness

Before opening the issue, have you...?

[X] Searched for existing issues
[X] Read through the Monitor-Troubleshooting Wiki
[X] Updated MonitorControl to the latest version (if applicable)

Describe the issue
I keep getting the popup that tells me to close other apps that manage brightness except I don't have any.
Expected behavior
No popup I guess?
Anything else?
No response
Environment Information (please complete the following information)
- macOS version: 12.6 / 13
- Mac model: Macbook Air M1
- MonitorControl version: 4.1.0 build 7034
- Monitor(s): iiyama prolite ge2488hs
- Monitor Cable(s)/Connection(s): hdmi
- Apple Silicon/M1 (yes or no): yes

You can ask the app not to warn about it again - but if you have this popup, then good chance you have some other conflicting app running (BetterDisplay, f.lux, etc). If not, then I have no idea why this is happening (the prompt is shown when the expected color table set by the app differs from what the app actually finds).
You can try using BetterDisplay if this is a returning issue as in that app I did not implement a warning like this. :)
(note: if you use BetterDispaly or MonitorControl via HDMI, do not forget to disable DDC brightness control as that does not work through the HDMI port of M1 macs).

Should I disable DDC brightness if my display is connected to HDMI via a USB-C hub?

Ohh sorry, I see you have an M1 Air. No, the problem is only for built-in HDMI ports, but an Air does not have one. If your dock properly translates DDC i2c communication over HDMI (most do) then you should not disable it. :) My bad!

I don't have any other applications that affect brightness or screens. Maybe it's the screen's fault? When it turns on I sometimes have to move the slider a bit to get the right brightness

I don't think so, this specific warning is only happens when there is a change caused in the color table (this defines how each colors look on your screen) by a third party app.

Unfortunately not. What you might want to do is to make sure no apps are launched (either as a Login Item or as an agent) and then start them one-by-one and see when the issue appears. But not many apps should do color table manipulation so if you figure out a new app that is interfering, let me know so I can add it to the bad boys list lol. :)

I use Night Shift and True Tone from macOS, is this a problem?

No, those should be applied over the color table manipulated screen and should not interfere.


Fix for iOS reverse camera button crash and Android plugin install

fixes iOS crash when reverse camera button is pushed
fixes Android plugin install failure if another plugin in the project also requires camera

You can change the afterDelay to 0 as it now the execution will be delayed until the block is called.
Other than that LGTM

bump


podman-tui-image Containerfile

Adding Containerfile for building podman-tui container image.
The image is built using latest Fedora and then podman-tui is built and installed.
Signed-off-by: Navid Yaghoobi<EMAIL_ADDRESS>

LGTM


do you support also Dash?

do you support also Dash?

no, but you can follow this example to create NBitcoin.Dash

Ok I will create and test it


Decouple individual listener initialization during startup

Title: Decouple individual listener initialization during startup
Description:
Currently, all listeners are started at once during initialization. This means if Listener A is warming, Listener B will not start. This includes static listeners.
This behavior is discussed a bit in https://github.com/envoyproxy/envoy/issues/6904 (though its also largely focusing on how this is exposed in config_dump).
This ends up being operationally risky. We expose stats through a (static) listener, but these cannot be accessed for pods that are stuck warming - when things are going wrong is exactly the time when we really want to see metrics to understand what is going on. Additionally, we serve health checks through (static) listeners. These should fail, but it would be nice to able to fail because we explicitly said "This is not ready yet for " rather than just refusing the connection entirely.
In general I am most concerned with static listeners starting, though I think that is fairly specific to the way we set things up today and doesn't seem like a general case.
One (of, I am sure, many) thing to keep in mind is there are dependencies between listeners for bind_to_port=false listeners. Likely these would need to be tied together in some way for initialization purposes

cc @kyessenov @ggreenway @alyssawilk

The idea makes sense to me, but as you've noted, you'll have to do this with great care, and probably not change the default behavior, or else you'll break peoples startup behavior.

+1 to reasonable idea and implementing with great care.  If well designed we might be able to get away with reload protecting if the differences are expected to be generally helpful and not harmful but worst case we can always add a knob.


Add language picker to notebook cells.

In order to support polyglot notebooks (i.e. .NET Interactive) we need to be able to configure cell languages individually. So, this change adds a language picker to the lower right corner of the cell, similar to what VS Code has. Language lookup is based off of the current provider and kernel display name, so if we're using Interactive we get all of its languages, but when using the Python 3 kernel we only get Python as an option.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
4 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.02%) to 48.278%

Files with Coverage Reduction
New Missed Lines
%

extensions/notebook/src/jupyter/serverInstance.ts
4
75.13%

Totals

Change from base Build<PHONE_NUMBER>:
-0.02%

Covered Lines:
14819

Relevant Lines:
29138

💛  - Coveralls


Remove cmd/gsutil image

Based on #1503
This image wrapped any image with gsutil in the path (typically,
google/cloud-sdk), and invoked it with our own custom Go wrapper.
That's unnecessary, since gsutil can be invoked directly as necessary,
using any image where gsutil is in the PATH (google/cloud-sdk is the
default).
This change should be entirely transparent to end users.
Operators can override the image being used if they so desire, to point
to any image that contains gsutil.
Historical note: cmd/gsutil was added in https://github.com/tektoncd/pipeline/pull/321, ostensibly to fix https://github.com/tektoncd/pipeline/issues/124, though the specific rationale for the wrapper script might be lost to time. 🕥
Submitter Checklist
These are the criteria that every PR should meet, please check them off as you
review them:

[y] Includes tests (if functionality changed/added)
[y] Includes docs (if user facing)
[y] Commit messages follow commit message best practices

See the contribution guide for more details.
Double check this list of stuff that's easy to miss:

If you are adding a new binary/image to the cmd dir, please update
the release Task to build and release this image.

Reviewer Notes
If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.
Release Notes
Use google/cloud-sdk directly to invoke gsutil in support of internal operations, instead of our own wrapper around that image.

/test pull-tekton-pipeline-integration-tests

/test pull-tekton-pipeline-integration-tests

/test pull-tekton-pipeline-integration-tests
❄

/lgtm

Looks like our gsutil wrapper may have been introduced to solve an issue similar to this one.
We had an explicit issue around this here

It sounds like gcloud expects users to call gcloud auth activate-service-account --key-file $GOOGLE_APPLICATION_CREDENTIALS before making calls.
We shouldn't introduce an image to run this for users, I think. Instead we should document this and make sure users know that gcloud expects this of them.

Sounds good to me.  We... probably need to consider this a breaking change in 0.9 tho right?  I'll update the release notes.

Yeah. If we guarantee this behavior going forward we should add a test.


[Tests] Audit History Entry tests

Some unit tests for history entry creation have been completed. audit what is being tested to ensure that all paths are being tested, including history entry items that should contain a parent item. i.e. when software added to computer, it's parent_pk is updated.
source mr is !17

mentioned in commit 8c1be6797483f365d9842c299afd353366d263d4

mentioned in commit 9bece0a811826b5577ac214c38dad11bb8249e7f

mentioned in commit d1b6c96d7211278f4caff17164faf37bc43ab396

mentioned in commit f29ec63f46fdba4273c3df7b88afc6959c0515eb

marked this issue as related to #15

mentioned in commit 367c4bebb67c24ffcc2ade19abfeac6089a1e702

mentioned in commit 789c035a03058cdf80b200df75c51d948fd6faf2


Missing the first element of the x-axis

I have an issue same as this example: https://apexcharts.com/javascript-chart-demos/column-charts/column-with-negative-values/
The first element of the x-axis ("2011-01-01") was disappeared when I use xaxis.labels.rotates = -90

Noted.
There is a function which calculates if labels exceed available grid area so that it can draw without overlapping. And I think that function is returning false positives sometimes for rotated labels.
I will fix that

Fixed it in the https://github.com/apexcharts/apexcharts.js/commit/4c658eb848321159378ecac06c5830f0b6311159

I am using "apexcharts": "^3.17.0", but still i have the same issue.

I am not able to see the first label. It's the same case with all types of graphs.

I have something very similar, the first element in categories is undefined


Farm detected as a bot for ad-clicking

Hi,
Kind of an hilarious message here. Not about you not getting your price, but about what they're saying. 1. They already know there's only one bot existing and this is this farm.
2. Obviously who will say the bot he's using if he's getting no reward.
I'm sorry for you, I cannot do something for now. Maybe I can try to randomize the ad-clicking to make them more "human & real™". As there is no security at all on the ad-clicking process, it was faking a ad click whenever it was possible. If they think a ad every 60min is not human, I'll make it human for them xD

lol  thats what i was thinking also, you give them nothing then want somthing ? xD lmao i woudnt tell them ether way.a humenizer would be a good idea

You could also try to stop farming for them to retrieve the prize and then start over xD

seeing if that works xD


BUG-2 Issue with sorting blog posts on date

User Statement
As a Content Creator, I would like for my newest blog posts to appear at the top of the blog feed so that returning System Users can access the latest content instantly.
Engineering Response
We will adjust the blog engine to display blog posts in order, starting from the newest post at the top and descending towards the oldest post at the bottom.
Technical Breakout
In the generateBlogFeed() function within BlogEngine.php there is a line that sorts on the date attribute. This is currently sorting by month, then day, and then year. This needs to be adjusted to sort on the largest period of time first.
Acceptance Criteria

[ ] Blog posts are rendered in the blog feed in descending order by date, starting with the newest blog post at the top and ending with the oldest blog post at the bottom

I've introduced a new method called generateSortableDateTime() to the XMLEngine class that provides a better solution for sorting on date and time. A better solution would make use of PHP's date and time object interfaces rather than sorting on a string.


#15243: Add retries to cpp tests

Ticket
#15243
Problem description
CPP tests are unstable and fail ND often on main
What's changed
Retry to improve stability. Need to be able to reproduce so we can raise to runtime team
Checklist

[ ] Post commit CI passes
[ ] Blackhole Post commit (if applicable)
[ ] Model regression CI testing passes (if applicable)
[ ] Device performance regression CI testing passes (if applicable)
[ ] New/Existing tests provide coverage for changes

passing cpp tests: https://github.com/tenstorrent/tt-metal/actions/runs/11940411942/job/33284197881

If retries work, maybe we can reproduce it by repeating the cpp tests in a custom dispatch, maybe also try to run it on the same machine too
If this fixes the issue, doesn't this mean its a issue caused by the tests?

Let's put this on hold for now until we get more signals from recent CPP test splitting @bkeith-TT

I discussed with @bkeith-TT today, and we're thinking of re-working this into an auto-retrier that lives outside as a service and uses the API to retry failed jobs on main.


feat(core-api): include deployed contracts in consortium data model

Is your feature request related to a problem? Please describe.
Right now if I want to invoke a contract on a Web3/eth ledger through the API client, I need to provide the ABI in my request and the contract address.
It's also quite easy to make a mistake when specifying which contract to use for what since it's not very intuitive for humans to differentiate long, randomized hex strings.
Describe the solution you'd like
We could provide a shorthand for this by storing this information as part of the consortium definition so that the request definition for a contract invocation could reference the contract ID as defined by the consortium definition.
The challenge is to define a Contract model entity that can accurately represent contracts in general for all ledgers, not just for a single one them.
The usage of this feature must be entirely optional meaning that any contract can be invoked on any ledger through the Cactus API client, no matter what.
Describe alternatives you've considered
The alternative is in place as of right now which is to mandate the callers to provide the ABI of the contract they are invoking.
cc: @takeutak @sfuji822 @jonathan-m-hamilton

For EVM:
Contracts could be specified in a json file that has an array with all the contracts (ABI, address, and name). They could be loaded as objects so they can be invoked in a similar way on different blockchains


Viewer

Redo of the Diagnostics Viewer PR, this time merged with devel and aimed at devel

Sam
Can you stop by & help me sort out a git issue?
Thanks
Jim
From: Sam Fries <notifications<EMAIL_ADDRESS>Reply-To: UV-CDAT/uvcmetrics <reply<EMAIL_ADDRESS>Date: Friday, May 6, 2016 at 12:56 PM
To: UV-CDAT/uvcmetrics <uvcmetrics<EMAIL_ADDRESS>Subject: [UV-CDAT/uvcmetrics] Viewer (#183)
Redo of the Diagnostics Viewer PR, this time merged with devel and aimed at devel

You can view, comment on, or merge this pull request online at:
https://github.com/UV-CDAT/uvcmetrics/pull/183
Commit Summary

Swapped 41 to 4A
Make it so capitalization doesn't matter
Added viewer script to metrics
Merge branch '41_to_4a' into viewer
Finished 41->4a
Added missing file, fixed bug where viewer required path argument
Made paths relative, made plotset images actually load in the index page
Fixed plotset 2 filenames and made defaults for raw_inputs work better
Merge remote-tracking branch 'origin/devel' into viewer
Cleaned up some loose ends, updated images to be UVCDAT generated

File Changes

M setup.pyhttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-0 (17)
A share/viewer/bootstrap.min.csshttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-1 (6)
A share/viewer/bootstrap.min.jshttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-2 (7)
A share/viewer/imgs/SET1.pnghttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-3 (0)
A share/viewer/imgs/SET10.pnghttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-4 (0)
A share/viewer/imgs/SET11.pnghttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-5 (0)
A share/viewer/imgs/SET12.pnghttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-6 (0)
A share/viewer/imgs/SET13.pnghttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-7 (0)
A share/viewer/imgs/SET14.pnghttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-8 (0)
A share/viewer/imgs/SET15.pnghttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-9 (0)
A share/viewer/imgs/SET2.pnghttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-10 (0)
A share/viewer/imgs/SET3.pnghttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-11 (0)
A share/viewer/imgs/SET4.pnghttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-12 (0)
A share/viewer/imgs/SET4A.pnghttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-13 (0)
A share/viewer/imgs/SET5.pnghttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-14 (0)
A share/viewer/imgs/SET6.pnghttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-15 (0)
A share/viewer/imgs/SET7.pnghttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-16 (0)
A share/viewer/imgs/SET8.pnghttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-17 (0)
A share/viewer/imgs/SET9.pnghttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-18 (0)
A share/viewer/jquery-2.2.3.min.jshttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-19 (4)
A share/viewer/viewer.csshttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-20 (28)
A share/viewer/viewer.jshttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-21 (14)
M src/python/frontend/amwgmaster.pyhttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-22 (22)
M src/python/frontend/metadiags.pyhttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-23 (2)
M src/python/packages/amwg/amwg.pyhttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-24 (6)
A src/python/viewer/init.pyhttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-25 (2)
A src/python/viewer/htmlbuilder.pyhttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-26 (229)
A src/python/viewer/plotsets.pyhttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-27 (505)
A src/python/viewer/viewerhttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-28 (1)
A src/python/viewer/viewer.pyhttps://github.com/UV-CDAT/uvcmetrics/pull/183/files#diff-29 (107)

Patch Links:

https://github.com/UV-CDAT/uvcmetrics/pull/183.patch
https://github.com/UV-CDAT/uvcmetrics/pull/183.diff

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHubhttps://github.com/UV-CDAT/uvcmetrics/pull/183


Multiple file downloads

From @rukku on November 25, 2015 11:11
For when a user wants to download multiple files at the same time. Something like a shopping cart? Or at check boxes per scene. Then the user can just click the download button once.
Copied from original issue: phl-microsat-dpad/libra#11

Can probably be shown in UI as an "Add bands to cart" button here:

...and an "Add bundle to cart" button here:


Ag-grid theme background

What happened?
Currently the iX ag-grid theme does set the grid background to transparent in order to deal with various background colors.
This can cause confusion and might not yield a determenistic outcome.

[ ] Set background
[ ] Fix broken properties e.g.:

--ag-checkbox-checked-color: var(--theme-color-input--focus);
--ag-range-selection-border-color: var(--theme-color-input--focus);

What type of frontend frameware are you seeing the problem on?
Angular, React, JavaScript
Which version of iX do you use?
1.1.1
Code to produce this issue.
No response

As discussed with @silviowolf background of the AGGrid stays transparent


DEMO application crashes with Xcode 11.2

I am not sure what is going wrong, but after I upgraded to Xcode 11.2, the demo application now crashes at:
extension EncodableRecord where Self: Encodable {
    public func encode(to container: inout PersistenceContainer) {
        let encoder = RecordEncoder<Self>(persistenceContainer: container)
        **try! encode(to: encoder)**
        container = encoder.persistenceContainer
    }
}

with
Thread 1: EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
I will assume that this is easily reproducible. If it is not, I can provide more information. The only relevant difference between when it was working and now seems to be the version of Xcode.
I am using the tip of master -- 416dda2
I did notice that the Demo app was using GRDB 4.4.0. I attempted to use GRDB 4.5.0, but the results were unchanged...still crashes.

Hello @williehallock802,
Thanks for reminding me of this. This crash is an Xcode bug. See https://github.com/groue/GRDB.swift/issues/640 for more information.
Please try the workaround explained in the above link: in the demo app Xcode project, select the app target, and set DEAD_CODE_STRIPPING to NO. It should fix the crash.
I should apply this workaround in the demo app so that other users do not face the same disappointment.

The workaround for the Xcode bug has been pushed to the master branch.


IOS foreground mode not update location as realtime as in android

In IOS, I have to wait longer than expect to get location from callback function. As in android device, I got the location similar to HTML5.geolocation.watchPosition.
Your Environment

Plugin version: 2.1.2
Platform: iOS
Device manufacturer and model:
Cordova version (cordova -v): 6.1.1
Cordova platform version (cordova platform ls): ios 4.1.1
Plugin configuration options:

[
      desiredAccuracy: 0,
      stationaryRadius: 5,
      distanceFilter: 1,
      activityType: 'AutomotiveNavigation',
      // debug: true, // <-- enable this hear sounds for background-geolocation life-cycle.
      // stopOnTerminate: false, // <-- enable this
      locationProvider: backgroundGeolocation.provider.ANDROID_ACTIVITY_PROVIDER,
      interval: 3000, // <!-- poll for position every minute
      fastestInterval: 500,
]

Expected Behavior

In foreground mode, I think we should able to get location as soon as possible like android did. In ios, the location will be callback around 20 sec.

 It's a maximum, meaning it can be much sooner (depends on option.desiredAccuracy and provider conditions).

Can you suggest the config then?
I want android and iOS to track location of user when in a car to draw a driving lines. So, the foreground and background should be realtime gps tracking.

ANDROID_ACTIVITY_PROVIDER is not suitable for cars, as you sit in it can be recognized as STILL activity (stationary).
I would go for something like:
{
      desiredAccuracy: 100,
      stationaryRadius: 30,
      distanceFilter: 1,
      activityType: 'AutomotiveNavigation',
      interval: 3000,
      saveBatteryOnBackground: false
}

Also back to android sampling. Thanks to log I found nice argument while timed based sampling is better then fixed number of samples. Currently iOS waits 15seconds and android waits for 5 location samples.
First 5 samples might be only from network, which is not as accurate as GPS. Thanks to users logs, I can actually confirm this:
Sampling on fixed number might result in network provider only results, beacasue when GPS is "cold" it might not get fix so quickly.
[2016/08/09 23:02:37] DistanceFilterLocationProvider:Location change: Location[network 49.******,5.****** acc=38 et=+2m52s269ms {Bundle[mParcelledData.dataSize=448]}] isMoving=false
[2016/08/09 23:02:17] DistanceFilterLocationProvider:Location change: Location[network 49.******,5.****** acc=38 et=+2m32s188ms {Bundle[mParcelledData.dataSize=448]}] isMoving=false
[2016/08/09 23:02:17] DistanceFilterLocationProvider:Location change: Location[network 49.******,5.****** acc=38 et=+2m32s188ms {Bundle[mParcelledData.dataSize=448]}] isMoving=false
[2016/08/09 23:01:59] BackgroundGeolocationPlugin:App is no longer visible
[2016/08/09 23:01:59] BackgroundGeolocationPlugin:App will be paused multitasking=true
[2016/08/09 23:01:57] DistanceFilterLocationProvider:Location change: Location[network 49.******,5.****** acc=38 et=+2m12s237ms {Bundle[mParcelledData.dataSize=448]}] isMoving=false
[2016/08/09 23:01:57] DistanceFilterLocationProvider:Location change: Location[network 49.******,5.****** acc=38 et=+2m12s237ms {Bundle[mParcelledData.dataSize=448]}] isMoving=false
[2016/08/09 23:01:56] DistanceFilterLocationProvider:Setting pace: false

Hi I have a similar issue iOS. I am working on an app for my research project that needs to track GPS location every second.
I have the following settings:
{
      desiredAccuracy: 0,
      stationaryRadius: 1,
      distanceFilter: 0,
      saveBatteryOnBackground: false,
      activityType: 'AutomotiveNavigation',
}

When I start a trip it seems to work fine ... the location changes in real-time. But once I switch from walking to running, or when I drive a car, or when I am on the bus the location tracking only updates every 15-90 seconds, which makes it impossible to detect any stops (e.g. at red lights or bus stops).
For my application I do not care about saving battery life. The goal is to get accurate real-time data for study analysis. I'm also not planning to submit the app to Google Play Store, nor to the Apple App Store.
Could you please show me a way to by-pass any of the energy saving functions and to maximize accuracy and real-time update?

@pbraun there could be multiple problems, but I cannot recommend this plugin for your use case as it's focusing on battery life.
The biggest one is distanceFilter, you'll need to disable it in code.
The seconds one is iOS specific, which does not allow you to poll for locations in defined time intervals, but rather meters.
In your case, I would go for something much simpler than this plugin. HTML5 or cordova-plugin-geolocation, which don't have any fancy features.

@mauron85 Thank you for the quick reply. I tried using the cordova-plugin-geolocation, there were two issues with it: 1. for me it didn't work on Android, and 2. it doesn't allow background logging. That is why I switched to this plugin.
Thank you for the cordova-background-geolocation-lt suggestion. I will give that a try.

yeah it should work better with option disableElasticity which basically turns off distanceFilter.
Anyway as there is more and more people asking for this kind of behaviour, I'm thinking about creating bare version to suit your needs.

A bare version would have its own advantages too. What do you think about adding more config options to get more fine grained control? E.g. choose whether to use only the traditional Location Manager or the Activity Types, turn off distance filter and region monitor ...
Do you have more thoughts about the bare version?

What do you think about adding more config options to get more fine grained control?

normally I'm usually against too many options, because I hate reading long docs and preferer reasonable defaults, BUT it will probably make sense in bare version.

E.g. choose whether to use only the traditional Location Manager or the Activity Type

what do you prefer? I'm not decided about this.

turn off distance filter and region monitor

sure, those will be removed from bare version
My requeriments to the bare version:

should provide minimum functionally
act as foundation to more advanced plugins, meaning those plugin has to use code from bare and only extend it to add adtitional functionality (no duplicate code bases).

I agree with you.
I think it would be good to have the option to use the standard location services as well as the significant changes feature. Having them in one plugin would be convenient but if it's too complicated to put them together maybe it would be better to have two different plugins for it.
I've never worked with native iOS code but I tried to patch the iOS code for this plugin to use the highest possible accuracy and turned off the distance filter but it still didn't have any affect. Maybe I did something wrong.
In LocationManager.m in the configure function I tried to set:
locationManager.distanceFilter = kCLDistanceFilterNone;
locationManager.desiredAccuracy = kCLLocationAccuracyBestForNavigation;

Is it even possible to use these options but this set the activity type?

FYI, Since I couldn't quickly figure out how to natively improve the accuracy I found a work around that kind of works for my app. As per your suggestion I used the HTML 5 geolocation feature, which works well for real-time updates, but it doesn't run in the background by itself. I found out that with the "cordova-plugin-background-geolocation" installed and configured, for some reason, it allows me to run the HTML5 geolocation code in the background. I only had to configure the plugin then call start and stop onsuccess, just once. The only disadvantage now is that every time I use the app the HTML5 geolocation feature asks for GPS permission. For some reason the WebView doesn't remember the user's decision. For my use-case that's not a big issue though.

In LocationManager.m in the configure function I tried to set:

locationManager.distanceFilter = kCLDistanceFilterNone;
locationManager.desiredAccuracy = kCLLocationAccuracyBestForNavigation;
You also need to disable recalculation of distance filter in LocationManage.m line 585
// replace following line:
// locationManager.distanceFilter = [self calculateDistanceFilter:[lastLocation.speed floatValue]];
// with this:
locationManager.distanceFilter = _config.distanceFilter;

and then replace line 590 - 595 with:
locationManager.distanceFilter = _config.distanceFilter;
[self startUpdatingLocation];

Thank you for the feedback on patching LocationManager.

I would be interested as well. I vote for the disable distance filter option too.

You can now use RAW_LOCATION_PROVIDER to achieve that.


fix APYChart Storybook UI component

APY Chart Storybook UI component is not working

https://github.com/Anchor-Protocol/anchor-web-app/pull/191


Create get-ghcup.sh, fixes #36

This follows the style of the rustup installation script that has a friendly welcome message, tells you what the script is going to do, and then asks if you'd like to proceed. This is meant to be a one-command setup of everything you need to get started with Haskell.
I already had this written so adding it for comparison to #40. Would still need to update README and clean a few things up. Let me know what you think!

Thanks, the messages in this PR are better, but I think feature wise https://github.com/haskell/ghcup/pull/40 is a bit more comprehensive, so maybe I'll improve the user facing messages.


Problems about training referring image segmentation

Hello, thank you very much for releasing the source code.  After training the detection task and saving the model to exps/refcoco/r101_det/checkpoint_best.pth, I intend to train the segmentation task further, but I encounter an error. Can you tell me how to solve this error?
The error occurs when I run GPUS_PER_NODE=2  tools/run_dist_launch.sh   2  configs/refcoco/RefTR_refcoco_101.sh.
The content of RefTR_refcoco_101.sh is
EXP_DIR=exps/refcoco/r101
python -u main_vg.py \
    --pretrained_model "exps/refcoco/r101_det/checkpoint_best.pth"\
    --num_feature_levels 1\
    --masks\
    --lr 1e-5\
    --lr_mask_branch_proj 10\
    --dataset refcoco_unc\
    --train_split train\
    --test_split val testA testB\
    --dec_layers 6\
    --backbone resnet101\
    --aux_loss \
    --img_size 640\
    --max_img_size 640\
    --epochs 40\
    --lr_drop 30\
    --output_dir ${EXP_DIR} \

This is the error information.
+ GPUS=2
+ RUN_COMMAND=configs/refcoco/RefTR_refcoco_101.sh
+ '[' 2 -lt 8 ']'
+ GPUS_PER_NODE=2
+ MASTER_ADDR=<IP_ADDRESS>
+ MASTER_PORT=29500
+ NODE_RANK=0
+ let NNODES=GPUS/GPUS_PER_NODE
+ python ./tools/launch.py --nnodes 1 --node_rank 0 --master_addr <IP_ADDRESS> --master_port 29500 --nproc_per_node 2 configs/refcoco/RefTR_refcoco_101.sh
+ PY_ARGS=
+ EXP_DIR=exps/refcoco/r101_det
+ EXP_DIR=exps/refcoco/r101
+ python -u main_vg.py --pretrained_model exps/refcoco/r101_det/checkpoint_best.pth --num_feature_levels 1 --masks --lr 1e-5 --lr_mask_branch_proj 10 --dataset refcoco_unc --train_split train --test_split val testA testB --dec_layers 6 --backbone resnet101 --aux_loss --img_size 640 --max_img_size 640 --epochs 40 --lr_drop 30 --output_dir exps/refcoco/r101
+ PY_ARGS=
+ EXP_DIR=exps/refcoco/r101_det
+ EXP_DIR=exps/refcoco/r101
+ python -u main_vg.py --pretrained_model exps/refcoco/r101_det/checkpoint_best.pth --num_feature_levels 1 --masks --lr 1e-5 --lr_mask_branch_proj 10 --dataset refcoco_unc --train_split train --test_split val testA testB --dec_layers 6 --backbone resnet101 --aux_loss --img_size 640 --max_img_size 640 --epochs 40 --lr_drop 30 --output_dir exps/refcoco/r101
| distributed init (rank 0): env://
backend=nccl, init_method=env://, world_size=2, rank=0
| distributed init (rank 1): env://
backend=nccl, init_method=env://, world_size=2, rank=1
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
git:
  sha: N/A, status: clean, branch: N/A

Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertModel: ['cls.predictions.transform.dense.weight', 'cls.predictions.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight', 'cls.predictions.decoder.weight', 'cls.predictions.transform.LayerNorm.bias']
- This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertModel: ['cls.seq_relationship.weight', 'cls.predictions.transform.dense.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.decoder.weight', 'cls.predictions.bias', 'cls.predictions.transform.LayerNorm.weight']
- This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Model dim: 256
Language model dim: 768
Using multi phrase loss
number of params: 174134685
Steps per training epoch:  3769
Using pretrained MODEL exps/refcoco/r101_det/checkpoint_best.pth to init the model.
Unexpected keys:  []
Missing keys:  ['bbox_attention.q_linear.weight', 'bbox_attention.q_linear.bias', 'bbox_attention.k_linear.weight', 'bbox_attention.k_linear.bias', 'mask_head.lay1.weight', 'mask_head.lay1.bias', 'mask_head.gn1.weight', 'mask_head.gn1.bias', 'mask_head.lay2.weight', 'mask_head.lay2.bias', 'mask_head.gn2.weight', 'mask_head.gn2.bias', 'mask_head.lay3.weight', 'mask_head.lay3.bias', 'mask_head.gn3.weight', 'mask_head.gn3.bias', 'mask_head.lay4.weight', 'mask_head.lay4.bias', 'mask_head.gn4.weight', 'mask_head.gn4.bias', 'mask_head.lay5.weight', 'mask_head.lay5.bias', 'mask_head.gn5.weight', 'mask_head.gn5.bias', 'mask_head.out_lay.weight', 'mask_head.out_lay.bias', 'mask_head.adapter1.weight', 'mask_head.adapter1.bias', 'mask_head.adapter2.weight', 'mask_head.adapter2.bias', 'mask_head.adapter3.weight', 'mask_head.adapter3.bias']
Start training
Traceback (most recent call last):
  File "main_vg.py", line 439, in <module>
    main(args)
  File "main_vg.py", line 371, in main
    model, criterion, data_loader_train, optimizer, lr_scheduler, device, epoch, args.clip_max_norm)
  File "/mnt/RefTR-main/engine_vg.py", line 41, in train_one_epoch
    loss_dict = criterion(outputs, targets)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/mnt/RefTR-main/models/criterion.py", line 186, in forward
    losses.update(self.get_loss(loss, outputs, targets, num_boxes, **kwargs))
  File "/mnt/RefTR-main/models/criterion.py", line 164, in get_loss
    return loss_map[loss](outputs, targets, num_boxes, **kwargs)
  File "/mnt/RefTR-main/models/criterion.py", line 150, in loss_boxes
    box_ops.box_cxcywh_to_xyxy(target_boxes)))
  File "/mnt/RefTR-main/util/box_ops.py", line 66, in generalized_box_iou
    assert (boxes2[:, 2:] >= boxes2[:, :2]).all()
AssertionError

Hi, this probably indicates that there is something wrong with your annotation that the ground truth box passed in is not in the form of cxcywh, may I know which annotation file you are using for multi-task training?

Thanks for your quick response. I use data/unc/unc_val.pth as annotation file for training grounding task, and no errors are reported when training this task. The same annotation file is utilized for multi-task training, but the error occurs. So I'm really puzzled by this error.
I print the boxes2's information.
# print(boxes2)
tensor([[0.5094, 0.0209, 0.4891, 0.9728],
        [0.0016, 0.0983, 0.9500, 0.8996],
        [0.0031, 0.1104, 0.1937, 0.5729],
        [0.7486, 0.2625, 0.2495, 0.7266],
        [0.5219, 0.1843, 0.4094, 0.5718],
        [0.7781, 0.2389, 0.2000, 0.3911],
        [0.5984, 0.3154, 0.2453, 0.3575],
        [0.7422, 0.0141, 0.2563, 0.7518],
        [0.5500, 0.1897, 0.2500, 0.7588],
        [0.2280, 0.1040, 0.4700, 0.6480],
        [0.0531, 0.1734, 0.1297, 0.6407],
        [0.0750, 0.2371, 0.4641, 0.7606],
        [0.0000, 0.0329, 0.9312, 0.9640],
        [0.6453, 0.2132, 0.2062, 0.4118],
        [0.2469, 0.7552, 0.2656, 0.2378],
        [0.0406, 0.3438, 0.2531, 0.2229]], device='cuda:0')

# print(boxes2[:, 2:] >= boxes2[:, :2])
tensor([[False,  True],
        [ True,  True],
        [ True,  True],
        [False,  True],
        [False,  True],
        [False,  True],
        [False,  True],
        [False,  True],
        [False,  True],
        [ True,  True],
        [ True,  True],
        [ True,  True],
        [ True,  True],
        [False,  True],
        [ True, False],
        [ True, False]], device='cuda:0')

# print((boxes2[:, 2:] >= boxes2[:, :2]).all())
tensor(False, device='cuda:0')

I notice some differences between the dataset loader of grounding and segmentation task.
In the ReferSegDataset for segmentation, bbox = np.array(bbox, dtype=int)
In the ReferDataset for groundin, bbox[2], bbox[3] = bbox[0]+bbox[2], bbox[1]+bbox[3]
Could these two different behaviours be the reason for this error?

Sorry for the bug, I use a different annotation file for segmentation since the original annotation do not contain mask ground truth, I'll update the script for that part soon.


DEV-158 | Upgrade cookbooks, gem bundle

Details: https://issues.teracy.org/browse/DEV-158

closed, updated with this PR: https://github.com/teracyhq/dev/pull/112


Statistics output figures

Create figures of statistics output.

Box plots
Density plots (maybe)

Completed in 1d79cb4fc3ae8827bf7306ffc14f2b1ba057485c


HERE API relates to Major Trunk Roads

On investigating the output from the HERE API, by placing various bounding boxes in residential estates, it is "limited" to those major trunk roads (A or M).
Whilst major cities may have more coverage than the suburbs, it may be worthwhile acknowledging this for people who may see more visible coverage of this.
This is from the to-be-deprecated transparent traffic map, highlighting the roads around the Tyne Tunnel.

Whilst it's a different API being hit, the output appears to provide a correlation with that produced from any bounding box return.

Agreed it seems to be the subject of the most support requests for help, maybe we should update the README STEP1 to highlight this and ask people to confirm if any data is returned before proceeding to STEP2

I've created a pull request to update the Readme
#3

Thanks @jeggburt I'll close this off for now and if people continue to comment we can re-open it.

Afternoon!
Is the above issue the reasonI am seeing data for main roads that is outside my bounding box. For example, I am getting returns for DE (streetname) of  B272 which is way outside my bounding box. [  51.4387,-0.1272;51.4227,-0.1134 ]

Afternoon!
Is the above issue the reasonI am seeing data for main roads that is outside my bounding box. For example, I am getting returns for DE (streetname) of B272 which is way outside my bounding box. [ 51.4387,-0.1272;51.4227,-0.1134 ]

Quite probably hence the update to the Readme doc, HERE api does not have every road so you need to verify the DE attributes in your bounding box.

Recommend you check out https://twitter.com/RgntsPkSpeeders and https://github.com/chronakis/speed-monitor-bot/ it has much more details as to the data provided by the HERE API.

This is an interesting thread, and something I've not considered. I have been on other things (you know, like paid work :) ). Now that I've had time to delve into it, I fear that the data for my bot is coming from well outside the area. This is my bounding box, but DE values show A23 and A25 in the surrounding area (https://jsoneditoronline.org/#left=cloud.09233cb3133d45f89b3ac51989dc34ca). However, the data is consistent with those observed within the target area. Were it to include those roads, we would consistently be seeing data of 40-50mph. Is there a way to be more sure about the relevance of the data?

Ok, so looking at the data visualisation at https://speedbot.bikesnbytes.net/, it looks like the data is way out and I'm in a similar situation to this: https://twitter.com/RgntsPkSpeeders/status/1427957749963542533. Oops.
Back to the drawing board...


draw_gridlines called but function is commented

index.js
enter_exit_update.js
are still calling draw_gridlines

Thanks, we can clean this up (we disabled the drawing of gridlines). Feel free to submit a pull request if you would like.

We're reenabling draw_gridlines, thanks for the issue.

Hi,
Will the example page be synced ?
By the way the example page still has the slider issue in Firefox (I saw
you fixed it in the code).
Thanks again
2018-03-22 11:17 GMT-04:00 Nicolas Fernandez<EMAIL_ADDRESS>
We're reenabling draw_gridlines, thanks for the issue.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/MaayanLab/clustergrammer/issues/11#issuecomment-375344074,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADbamwsFOy_V7i65nDnarYZA-eAV9VKxks5tg8CWgaJpZM4QvNEz
.

Hi,
Are you referring to these example pages?
http://maayanlab.github.io/clustergrammer/scrolling_tour
http://maayanlab.github.io/clustergrammer/
We have just updated them by fast forwarding the gh-pages branch to the master branch.
Best,
Nick

Hello,
Yes these pages.
In Firefox the sliders' triangles and the clusters' trapezoids are missing.
2018-03-22 11:58 GMT-04:00 Nicolas Fernandez<EMAIL_ADDRESS>
Hi,
Are you referring to these example pages?
http://maayanlab.github.io/clustergrammer/scrolling_tour
http://maayanlab.github.io/clustergrammer/
We have just updated them by fast forwarding the gh-pages branch to the
master branch.
Best,
Nick
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/MaayanLab/clustergrammer/issues/11#issuecomment-375359013,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADbam7Zw23DxogR-O4YsuAX65HaIfAq1ks5tg8oSgaJpZM4QvNEz
.

Yes, Firefox appears to have a problem displaying SVG paths in Clustergrammer (the slider triangle and trapezoids are drawn using SVG paths). We recommend using Chrome, but we will look into the issue more in the future.

Hi Nicolas,
I am pretty sure it was fixed in your code because I was about to do a pull request with this:
-- var output_string = 'M' + start_x + ',' + start_y + ', L' + mid_x + ', ' + mid_y + ', L' + final_x + ',' + final_y + ' Z';

++ var output_string = 'M' + start_x + ',' + start_y + ' L' + mid_x + ', ' + mid_y + ' L' + final_x + ',' + final_y + ' Z';

in these files:
clustergrammer\src\dendrogram\build_svg_dendro_slider.js
clustergrammer\src\dendrogram\make_dendro_crop_buttons.js
clustergrammer\src\dendrogram\make_dendro_triangles.js
but turned out the changes were there unless I looked too fast at those subtle changes.
So here they are if you need them !

Thanks, do those changes help with rendering triangles/paths in Firefox?

Yes, specifically for SVG path in Firefox.
2018-03-29 12:04 GMT-04:00 Nicolas Fernandez<EMAIL_ADDRESS>
Thanks, do those changes help with rendering triangles/paths in Firefox?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/MaayanLab/clustergrammer/issues/11#issuecomment-377285223,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADbam-4gZZfZY1tvcvManqYhcP6AZZVGks5tjQYZgaJpZM4QvNEz
.

Cool, if you want to do a pull request I can check it on my end and merge it.

That worked great! Thanks again 👍I'll be releasing a new version with this fix.