Add verbose option for logging

This PR adds a new variable to enable verbose logging on the cron job for easier identification of issues as well as review of any purge/clean-up on snapshots.

Hi @PleaseStopAsking - Don't worry about the lint errors, I will fix those. Since it last ran Ansible-Lint has had loads of updates so new rules are being triggered and causing failures.

@dbrennand I went ahead and fixed the lint issues reported as they did not require any breaking changes. Feel free to run workflows again time permitting.

LGTM


Inject type from join via raw sql

Howdy! After reading the source and the fine documentation, I've been searching for how to properly inject types when using raw sql in a join. In my case, I'd like to join in an array that will be defined at runtime (query compile time):
  const batchToQueue = await pg
    .selectFrom('EmbeddingsMetadata as em')
    .selectAll('em')
    .leftJoinLateral(
      sql<{v: {model: string}}>`unnest(ARRAY[${sql.join(configuredModels)}]::varchar[])`.as<'v'>(
        'v(model)' as 'v'
      ),
      (join) => join.onTrue()
    )
    .leftJoin('Team as t', 'em.teamId', 't.orgId')
    .select('t.orgId')
    .select('v.model')  // <-- TypeScript error occurs here
   ...

This sql<{v: {model: string}}>... is most certainly incorrect as the trip of this expression is really a string[]. So, forgive that desperate attempt.
Many thanks for a great module!

Is this what you're after https://kyse.link/K1s08?

@koskimas I'm running into two problems, the first I have a work around for but I wanted to raise it in case I am missing something.
This line:
   .leftJoinLateral(
        sql<{model: string}>`unnest(ARRAY[${sql.join(configuredModels)}]::varchar[])`.as<'v'>(
            'v(model)' as 'v'
        ),
        (join) => join.onTrue()
    )

Will render as:
 LEFT JOIN LATERAL unnest(ARRAY[$1, $2]::VARCHAR[]) AS "v(model)" ON TRUE

In the Postgres dialect, "v(model)" (note double quotes) won't create v.model (accessible with v.*, for example) rather, you'll et a column called v(model). Not the end of the world...one can work around that. Curious if there is a way to drop a literal as?
Second thing I'm running into. I ended up cleaning up my code a bit and using RawBuilder<T> to implement a convenience method:
  function unnestedArray<T>(value: T): RawBuilder<T> {
    if (!Array.isArray(value)) {
      throw new TypeError('Value must be an array')
    }

    return sql`unnest(ARRAY[${sql.join(value)}]::varchar[])`
  }

  const batchToQueue = (await pg
    .selectFrom('EmbeddingsMetadata as em')
    .selectAll('em')
    .leftJoinLateral(unnestedArray(configuredModels).as('model'), (join) => join.onTrue())
    .leftJoin('Team as t', 'em.teamId', 't.id')
    .select('model') // <-- Typescript error

The TypeScript error I get is:
No overload matches this call.
  Overload 1 of 3, '(selections: readonly SelectExpression<DB & { em: EmbeddingsMetadata; } & { model: (string | null)[]; } & { t: Nullable<Team>; }, "em" | "model" | "t">[]): SelectQueryBuilder<...>', gave the following error.
    Argument of type 'string' is not assignable to parameter of type 'readonly SelectExpression<DB & { em: EmbeddingsMetadata; } & { model: (string | null)[]; } & { t: Nullable<Team>; }, "em" | "model" | "t">[]'.
  Overload 2 of 3, '(callback: SelectCallback<DB & { em: EmbeddingsMetadata; } & { model: (string | null)[]; } & { t: Nullable<Team>; }, "em" | "model" | "t">): SelectQueryBuilder<...>', gave the following error.
    Argument of type 'string' is not assignable to parameter of type 'SelectCallback<DB & { em: EmbeddingsMetadata; } & { model: (string | null)[]; } & { t: Nullable<Team>; }, "em" | "model" | "t">'.
  Overload 3 of 3, '(selection: SelectExpression<DB & { em: EmbeddingsMetadata; } & { model: (string | null)[]; } & { t: Nullable<Team>; }, "em" | "model" | "t">): SelectQueryBuilder<...>', gave the following error.
    Argument of type '"model"' is not assignable to parameter of type 'SelectExpression<DB & { em: EmbeddingsMetadata; } & { model: (string | null)[]; } & { t: Nullable<Team>; }, "em" | "model" | "t">'.ts(2769)

This is strange to me as I see "model" in the types right there but I haven't walked the source yet to grok why this TS error is happening.
Anyhow, the first bit is the thing that's more interesting. Definitely a corner case. If you don't care to address it I am fine just working around it for now.


zita-resampler: 1.6.2 -> 1.8.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/zita-resampler/versions.
meta.description for zita-resampler is: "Resample library by Fons Adriaensen"
meta.homepage for zita-resampler is: "http://kokkinizita.linuxaudio.org/linuxaudio/downloads/index.html"
meta.changelog for zita-resampler is: ""
Updates performed

Version update

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 1.8.0 with grep in /nix/store/bd0nlph3743bsk26ghs0y5bsqywq91x7-zita-resampler-1.8.0

directory tree listing: https://gist.github.com/c00307bbed7394751c852db8a3b2a0b5

du listing: https://gist.github.com/c254c5e2ee5a093edc2f6b218788226d

Rebuild report (if merged into master) (click to expand)

30 total rebuild path(s)

10 package rebuild(s)

10 x86_64-linux rebuild(s)
10 i686-linux rebuild(s)
0 x86_64-darwin rebuild(s)
10 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
drumgizmo
ebumeter
guitarix
infamousPlugins
kapitonov-plugins-pack
xtuner
zita-ajbridge
zita-at1
zita-njbridge
zita-resampler

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/bd0nlph3743bsk26ghs0y5bsqywq91x7-zita-resampler-1.8.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A zita-resampler https://github.com/r-ryantm/nixpkgs/archive/0497d984c9e0ef6945ab1cb473f0cefc0035d7a0.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/bd0nlph3743bsk26ghs0y5bsqywq91x7-zita-resampler-1.8.0
ls -la /nix/store/bd0nlph3743bsk26ghs0y5bsqywq91x7-zita-resampler-1.8.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

10 packages built:

drumgizmo
ebumeter
guitarix
infamousPlugins
kapitonov-plugins-pack
xtuner
zita-ajbridge
zita-at1
zita-njbridge
zita-resampler

Maintainer pings
cc @magnetophon for testing.

nixpkgs-review happy for me, non-nixos linux x86_64


Error in integer encoding due to OTP bug

The value -576460752303423488 is a counter example. It doesn't correctly render on OTP 17.1, and this issue tracks the problem until we get rid of the bug in OTP.

This one is fixed on never Erlang releases. We should change the EQC tests so they can generate the erroneous value and then remove this issue.


Why Heroku no work?

Ugh

This PR served two purposes. One to fix whatever the hell was wrong with my db setup script. The second was to fix our session store issue on Heroku and run in strict mode so we get const and other things.


Icon mdi:amazon was removed from MDI

Describe the bug
I get this from time to time in my logs and I'm not quite sure if it comes this from Alexa Media Player of from Core / Nabucasa Cloud:
Logger: frontend.js.latest.202111090
Source: components/system_log/init.py:190
First occurred: 16:00:30 (2 occurrences)
Last logged: 16:33:12
Icon mdi:amazon was removed from MDI, please replace this icon with an other icon in your config, it will be removed in version 2021.12.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
System details

Home-assistant (version):
Hassio (Yes/No): (Please note you may have to restart hassio 2-3 times to load the latest version of alexapy after an update. This looks like a HA bug).
alexa_media  (version from const.py or HA startup):
alexapy (version from pip show alexapy or HA startup):
Amazon 2FA is enabled (y/n). We will not debug login issues if unanswered:

Logs
Please provide logs. We'll be most likely asking for them anyway.
Additional context
Add any other context about the problem here.

We don't use that icon.


Added magic drc report to RDB conversion

Description
This pull adds a simple python script to convert the saved magic drc report to rdb format.
So it can be opened with Klayout or Synopsys RVE for debug.
Note

It creates a pseudo rdb format with some placeholder test to make report readable by Klayout of SNPS tool

@ganeshgore: Merged. Thanks!
P.S: can you PR this to openlane as well? (or can I use it there?)

I am not using openlane for my design yet, so I am unsure in which step I should integrate this. But the script is portable, you can definitely use it in openlane. if you get any problem, let me know.
NOTE : The script assumes the input magic.drc.log file is generated with following code.
https://github.com/efabless/open_mpw_precheck/blob/cf3b95ae6e1185435d3511ecb9f41919e80b2a60/drc_checks/magic_drc_check.tcl#L32-L48

@ganeshgore: We are using the same output convention there as well. So, I can easily integrate it. Thank you very much!


WIP: CF convention parser

[ ] Closes #127
[x] Tests added
[x] Passes pre-commit run --all-files
[ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
[ ] New functions/methods are listed in api.rst
[ ] Double check dependencies if we decide to require cf-xarray

This actually works quite neatly out of the box thanks to cf-xarray.
Using this I can take a CMIP6 dataset:

And get a fully functional xgcm grid with a few lines:
kwargs = cf_parser(ds)
grid = Grid(**kwargs)
grid

<xgcm.Grid>
X Axis (periodic, boundary=None):
  * center   x --> outer
  * outer    x_outer --> center
Y Axis (periodic, boundary=None):
  * center   y --> outer
  * outer    y_outer --> center
Z Axis (periodic, boundary=None):
  * center   lev --> outer
  * outer    lev_outer --> center
T Axis (periodic, boundary=None):
  * center   time --> outer
  * outer    time_outer --> center

The parser function modifies the dataset and then passes it + additional kwargs (at the moment only the coords) to the Grid constructor. We could implement this step automatically in the Grid.__init__ if CF conventions are present.

@dcherian I am curious if you have some ideas on testing here. This works well with a single dataset, but I figured you have a lot more experience with edge cases re CF stuff.

Another general question here would be if we want to make cf-xarray a full or optional dependency. cc @rabernat @dcherian @TomNicholas.

Hah I think the edge cases are in the CMIP archive :)

Hah I think the edge cases are in the CMIP archive :)

LOL


Always store metadata in info.meta, not just when using string interpolation.

This change ensures any logged metadata is always accessible from the info object's meta property (i.e. info.meta), which is consistent with the behavior exhibited when using string interpolation/format.splat.
Prior to 3.0, a custom transport could use the log function's meta argument to access any metadata. Now that the log method's signature only has two arguments (info, callback), it makes sense to have a designated meta property from which to access any metadata.

Thanks for this, but this behavior is a by-design change from<EMAIL_ADDRESS>If folks in the community feel strongly about reverting this we can consider it, but merging the enumerable properties of any meta object into the "info" was intentional.
To put it succinctly: "the info is the meta".


added url sharing for graphiql

Description
After each page reload, a new tab is created and I am unable to share my query via URL. In Apollo or Graphene, this functionality is available, so it would be great to have the same behavior implemented in Strawberry as well.
Inspired by renderExample.js.

Types of Changes

[ ] Core
[ ] Bugfix
[x] New feature
[ ] Enhancement/optimization
[ ] Documentation

Issues Fixed or Closed by This PR

Checklist

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[x] I have tested the changes and verified that they work and don't break anything (as well as I can manage).

Hi, thanks for contributing to Strawberry 🍓!
We noticed that this PR is missing a RELEASE.md file. We use that to automatically do releases here on GitHub and, most importantly, to PyPI!
So as soon as this PR is merged, a release will be made 🚀.
Here's an example of RELEASE.md:
Release type: patch

Description of the changes, ideally with some examples, if adding a new feature.

Release type can be one of patch, minor or major. We use semver, so make sure to pick the appropriate type. If in doubt feel free to ask :)
Here's the tweet text:
🆕 Release (next) is out! Thanks to Mykhailo Havelia for the PR 👏

Get it here 👉 https://github.com/strawberry-graphql/strawberry/releases/tag/(next)

Thanks for the PR @Arfey! Currently testing your changes locally. So far appears to be working as expected, good job!

I've also asked someone from the GraphiQL team to take a look 😊


Did not work with drop shadow in this case

When I use drop shadow it just fade like this-  filter: drop-shadow(-2px 1px 11px black);

and  when I use box-shadow I fell like this-

if I am doing something Wrong please help me and let me know .
this is the site where I am testing this library (Customui.codes)

@aatmprakash I'll check it


SupportTVOS Pod

Thanks @RomualdPercereau ! :smile:
Unfortunately, this isn't quite complete. We'll need to add a tvOS target, and update travis-ci.
But, I'll go ahead and do this part.


Trouble installing Ruby on Mac OS Big Sur

I am trying to install rbenv to Mac OS Big Sur:
following are the commands:
Install rbenv via brew
brew install rbenv ruby-build
Added rbenv to bash so that it loads every time you open a terminal
echo 'if which rbenv > /dev/null; then eval "$(rbenv init -)"; fi' >> ~/.zshrc
source ~/.zshrc
Install Ruby
rbenv install 2.7.2
However I am getting following error:
Downloading openssl-1.1.1i.tar.gz...
-> https://dqw8nmjcqpjn7.cloudfront.net/e8be6a35fe41d10603c3cc635e93289ed00bf34b79671a3a4de64fcee00d5242
Installing openssl-1.1.1i...
Installed openssl-1.1.1i to /Users/subratrout/.rbenv/versions/2.7.2

Downloading ruby-2.7.2.tar.bz2...
-> https://cache.ruby-lang.org/pub/ruby/2.7/ruby-2.7.2.tar.bz2
error: failed to download ruby-2.7.2.tar.bz2

BUILD FAILED (macOS 11.1 using ruby-build 20201225)

Inspect or clean up the working tree at /var/folders/p1/rt785hd96dbfmjqp3nbvvy740000gp/T/ruby-build.20210119132640.73931.iyjaox
Results logged to /var/folders/p1/rt785hd96dbfmjqp3nbvvy740000gp/T/ruby-build.20210119132640.73931.log

Last 10 log lines:
/Users/subratrout/.rbenv/versions/2.7.2/openssl/share/doc/openssl/html/man7/scrypt.html
/Users/subratrout/.rbenv/versions/2.7.2/openssl/share/doc/openssl/html/man7/SM2.html
/Users/subratrout/.rbenv/versions/2.7.2/openssl/share/doc/openssl/html/man7/ssl.html
/Users/subratrout/.rbenv/versions/2.7.2/openssl/share/doc/openssl/html/man7/X25519.html
/Users/subratrout/.rbenv/versions/2.7.2/openssl/share/doc/openssl/html/man7/X448.html -> /Users/subratrout/.rbenv/versions/2.7.2/openssl/share/doc/openssl/html/man7/X25519.html
/Users/subratrout/.rbenv/versions/2.7.2/openssl/share/doc/openssl/html/man7/x509.html
/var/folders/p1/rt785hd96dbfmjqp3nbvvy740000gp/T/ruby-build.20210119132640.73931.iyjaox ~
~
/var/folders/p1/rt785hd96dbfmjqp3nbvvy740000gp/T/ruby-build.20210119132640.73931.iyjaox ~
curl: (35) error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version

I am not sure what's the issue here.
I have installed both curl and openssl using brew. I suspect there is some incompatibility between these two.
My curl version is 7.74.0 and openssl is 1.1.1i

Hello @subratrout,
I had some problems using rbenv in macOS, try to uninstall rbenv and use rvm. #Guide installation rvm
Follow the steps bellow it will help you with your problem.
Observation: the new versions ruby crashed with versions >~<EMAIL_ADDRESS>
Uninstall your current version openssl and force it

$ brew uninstall --ignore-dependencies<EMAIL_ADDRESS>

Install the openssl version 1.0

$ brew install<EMAIL_ADDRESS>

Set the version

$ brew link<EMAIL_ADDRESS>

Check the version

$ openssl version
OpenSSL 1.0.2t  10 Sep 201

Install ruby 2.7.2 with the params

$ rvm install 2.7.2 --with-zlib-dir=$(brew --prefix zlib<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

Marking as inactive, to be closed at the end 3.4 dev, barring further input


What is the host name and password?

Hi,
I cannot login, and I don't know the host name and the password? What are they?

user：vagrant
pass：vagrant


thttpd: add THTTPD_NFILE_DESCRIPTORS

Summary
using THTTPD_NFILE_DESCRIPTORS instead of NFILE_DESCRIPTORS. This patch associates with https://github.com/apache/incubator-nuttx/pull/3045.
Impact
Testing
daily test

../nuttx/tools/checkpatch.sh -g<PHONE_NUMBER>f816cedfe6f9ff57c699e4d027af09..HEAD
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd.c:100:2: error: Mixed case identifier found
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd.c:101:2: error: Mixed case identifier found
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd.c:132:17: error: Mixed case identifier found
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd.c:132:57: error: Mixed case identifier found
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd.c:133:36: error: Mixed case identifier found
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd.c:134:52: error: Mixed case identifier found
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd.c:135:23: error: Mixed case identifier found
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd.c:135:63: error: Mixed case identifier found
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd.c:530:2: error: Mixed case identifier found
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd.c:600:17: error: Mixed case identifier found
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd.c:600:57: error: Mixed case identifier found
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd.c:611:14: error: Mixed case identifier found
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd.c:618:38: error: Mixed case identifier found
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd.c:623:14: error: Mixed case identifier found
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd.c:628:37: error: Mixed case identifier found
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd.c:635:36: error: Mixed case identifier found
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd.c:636:52: error: Mixed case identifier found
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd.c:646:23: error: Mixed case identifier found
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd.c:646:63: error: Mixed case identifier found
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd.c:733:35: error: Mixed case identifier found
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd.c:742:29: error: Mixed case identifier found
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd_cgi.c:720:2: error: Mixed case identifier found
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd_cgi.c:1100:21: error: Mixed case identifier found
Error: /home/runner/work/incubator-nuttx-apps/incubator-nuttx-apps/apps/netutils/thttpd/thttpd_cgi.c:1100:61: error: Mixed case identifier found
ignore?

Yes, let ignore it.

Yes, let ignore it.

https://github.com/apache/incubator-nuttx-apps/search?q=ClientData&type=code
@xiaoxiang781216 Where is ClientData coming in from?

It exist for long time, do you want to rename it?

@xiaoxiang781216  are you asking if you or @Donny9 should fix it?

Yes, we can rename it if no objection.

@xiaoxiang781216  Please do.

@xiaoxiang781216 Please do.

There are so many such "Mixed case identifier found", Are you sure want to fix them? not just ClientData. Timer, TimerProc, ....

@davids5 the violation is huge, let's merge this patch first and create a new PR late to fix the mix case issue since fs: Allocate file/socket dynamically apache/incubator-nuttx#3045 which is also a huge patch need to merge depends on this PR.


Use normal Collection when property has indexBy

Avoid using <int> index when property has indexBy.
Fixes https://github.com/rectorphp/rector/issues/8791
Fixes https://github.com/rectorphp/rector-doctrine/issues/357

Fixed 🎉 /cc @AlexeyKosov @yguedidi

All checks have passed 🎉 @TomasVotruba I am merging it ;)

Thanks @samsonasik !!
Maybe better to detect the type of the indexed by referenced property, and put either int or string as collection key type?

@yguedidi that's will be too much lookup :), using Collection<ClassName> is just fine for it

Then maybe at least skip changing if there is already an index in the type?

that will be too much for that :)

@samsonasik the thing is that I don't want Rector to change my Collection<string, ClassName> to Collection<ClassName> 😅

Feel free to skip the rule.

Feel free to skip the rule.

I know it's possible, but I find it weird that Rector will break a more precise type (the one with key type define), and in order to avoid that, we have to make our config more complex to maintain by adding a skip configuration (either full rule, either specific to few files)
I think, as a general behavior, rectors shouldn't degrade existing type definitions if they are more precise than what the rectors changes are capable of

It only change when it has unioned type as far as I see: @var Collection|Training[] -> @var Collection<Training>

in my case I have something like
    /**
     * @var Collection<string, Training>
     */
    #[ORM\OneToMany(targetEntity: Training::class, indexBy: "id", mappedBy: "trainer")]
    private Collection $trainings;

that got transformed to
    /**
     * @var Collection<int, Training>
     */
    #[ORM\OneToMany(targetEntity: Training::class, indexBy: "id", mappedBy: "trainer")]
    private Collection $trainings;

@yguedidi that's correctly skipped, see PR fixture

https://github.com/rectorphp/rector-doctrine/pull/360


Use bang as force write

Only write format output to the buffer when there are no unsaved changes or Format was called with bang
I added force as a new param for now, but when we want to add more stuff, it'll probably be nicer to have a config table or similar.

With the latest changes to refactor branch, could you update and we could merge this in?

Updated
and I changed some other things too

we read the buffer only once and keep the state in memory between formatter, then write back at the end.
We need to do this, because one formatter can change the number of lines, which then will make the next formatter read the wrong selection from the buffer. Or will throw an Index out of bounds error on write
only read the bufnr once, so the user can focus a different buffer and things still work
I removed the getRange function. -range=% will always set the start and end line, they will never be nil.
I merged format and internalFormatter as there was basically no logic left in format. Is there a reason to keep it?

Looks good!


Tests failing with v2.0.0-beta8: Could not find com.jetbrains.intellij.platform:test-framework:242.18071.12

What happened?
I got the following error while running tests:
FAILURE: Build failed with an exception.

* What went wrong:
Configuration cache state could not be cached: field `provider` of `org.gradle.configurationcache.serialization.codecs.ProviderBackedFileCollectionSpec` bean found in field `element` of `java.util.Collections$SingletonList` bean found in field `elements` of `org.gradle.configurationcache.serialization.codecs.ResolutionBackedFileCollectionSpec` bean found in field `__classpathSnapshot__` of `org.jetbrains.kotlin.gradle.tasks.KotlinCompile$ClasspathSnapshotProperties` bean found in field `__classpathSnapshotProperties__` of task `:compileTestKotlin` of type `org.jetbrains.kotlin.gradle.tasks.KotlinCompile`: error writing value of type 'org.gradle.api.internal.provider.DefaultProvider'
Configuration cache entry discarded due to serialization error.
> Could not resolve all files for configuration ':testCompileClasspath'.
   > Could not find com.jetbrains.intellij.platform:test-framework:242.18071.12.
     Searched in the following locations:
       - https://repo.maven.apache.org/maven2/com/jetbrains/intellij/platform/test-framework/242.18071.12/test-framework-242.18071.12.pom
       - http://localhost:7348/com.jetbrains.intellij.platform/test-framework/242.18071.12/descriptor.ivy
       - https://cache-redirector.jetbrains.com/intellij-dependencies/com/jetbrains/intellij/platform/test-framework/242.18071.12/test-framework-242.18071.12.pom
       - https://cache-redirector.jetbrains.com/www.jetbrains.com/intellij-repository/releases/com/jetbrains/intellij/platform/test-framework/242.18071.12/test-framework-242.18071.12.pom
       - https://cache-redirector.jetbrains.com/www.jetbrains.com/intellij-repository/snapshots/com/jetbrains/intellij/platform/test-framework/242.18071.12/test-framework-242.18071.12.pom
       - https://cache-redirector.jetbrains.com/plugins.jetbrains.com/maven/com/jetbrains/intellij/platform/test-framework/242.18071.12/test-framework-242.18071.12.pom
     Required by:
         project :

The same build.gradle.kts worked just fine with v2.0.0-beta7.

Plugin version: 2.0.0-beta8
Gradle version: 8.8
Operating system: Ubuntu (via GitHub Actions)

Fixed with #1677


Read() password protected spreadsheet

Can the read()  function read a password protected spreadsheet with a password passed as argument?
Thanks

Leroy, I've added some tests to the automated suite for reading and writing encrypted files, but could you please download the master branch and check it all works in your own environment? Again I must stress that only XML (.xlsx) files are supported.

This works perfectly as described! Thanks Julian!


NaNs after ca. 220 coupling cycles

After refactoring (#45), the couette scenario fails after about 220 coupling cycles if 2-way coupling is enabled:
With SIMPLE_MD and couette.xml.template but with 1 MD instance and ONLY with 2-way coupling
Finish coupling cycle 234
Finish coupling cycle 244
ERROR simplemd::moleculemappings::VelocityStoermerVerletMapping::handleMolecule: Position 0 is out of range
Position: -nan , 21.3675 , 5.01122, molecule: 2679
Velocity: -nan , -0.811837 , 0.0131969, molecule: 2679
OldVelocity: 0.904897 , -0.800832 , 0.019862, molecule: 2679
Force: -nan , -1.75438 , -1.70058, old: -3.47021 , -2.64766 , -0.965466
Old position: 17.8997 , 21.3716 , 5.01118

With LS1_MARDYN (663745bb88c8f85d9df7abb55b32c74bb7dad03a) and couette.xml.template but with 1 MD instance and ONLY with 2-way coupling
Finish coupling cycle 223
Finish coupling cycle 228
ERROR:  20240808T125404  752.102 [0]    NaN detected, exiting.
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
with errorcode 1.

You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------

This is expected behaviour when two-way coupling is enabled for only a single MD instance.
Simulations may only be stable with more samples (>=200).


docs profile for edown

I noticed you use dev profile for including edown. The edoc provider runs under the docs profile, so easier if you put doc specific stuff like edown under that profile, then you won't need as dev, just rebar3 edoc.

oh nice! I mostly copied the code from hackney. i am doing the change now.

done. Thanks!


Catch abbreviations in street names

OpenStreetMap has a strongly held rule against abbreviations, but sometimes abbreviations slip through. In some cases, new users haven’t been made aware of this rule; in other cases, the TIGER expansion bot missed some abbreviations. This tool should catch the abbreviations so that they can be expanded by bot or by hand.
It’s important that the tool be language aware: as a starting point, it could look for English abbreviations in English-speaking regions only, then expand to other regions as lists of abbreviations are compiled for more languages.
/cc @srividyacb

Good!! once you identify those,  and easy way to fix  those issues:  That is for Peru abbreviations
https://gist.github.com/Rub21/cc055320c925c855926e
cc. @maning

More sources of abbreviations:
https://github.com/Project-OSRM/osrm-text-instructions/tree/master/languages/abbreviations/
https://github.com/mapbox/geocoder-abbreviations/
https://wiki.openstreetmap.org/wiki/Name_finder:Abbreviations


License etc

Add License, Notice file and modify Readme file following LLNL open source software guideline
closes #89

Finally got the IM release code. @forsyth2

I'm following guild lines here: https://software.llnl.gov/about/licenses/

I'm following guild lines here: https://software.llnl.gov/about/licenses/ Please feel free to merge

Sounds good!


Error when posting message to myself

 ACTIVITYPUB.OUTBOX: Error when posting activity to local actor https://mypod.store/sro: The resourceUri param must be remote. Provided: https://mypod.store/sro/data/activities/64624542173b0f552f2390c6 (webId https://mypod.store/sro)
In activitypub.outbox service, localPost method, we shouldn't call ldp.remote.store if the emitter is same as recipient.
This doesn't create problems besides this error message.

Maybe fixed by https://github.com/assemblee-virtuelle/activitypods/commit/e5ff5ee5c62ec74e7748d1b95b8b4b4f9bd40e20

The original error occured on the next branch as well.


SIMPY | Solve equations

Solve equations using simpy
Also display equations on screen
Get all steps documented in a Latex document for printing into PDF

Test data:

Test1: Solve the simple equations (existing examples from below link)

https://github.com/vamseeachanta/assetutilities/blob/f7d7bfea6bc87ec6b96c0fa3bf293f228b234569/docs/sub_math

Test2: Pressure loss calculation. https://github.com/vamseeachanta/digitalmodel/issues/57

Displaying equation on screen:

https://docs.sympy.org/latest/tutorials/intro-tutorial/printing.html

displaying equations on screen :
ipython console is not opening as described here https://docs.sympy.org/latest/tutorials/intro-tutorial/printing.html

@samdansk2 , first create a test for simple equation and try all the methods above. Let us do this for a simple equation that was used in simpy link above. Can you do this and send me link to the code?

@vamseeachanta
I am not getting your point sir
How can we write a test for displaying a single equation ?

@samdansk2 , just a simple function to just display the equations in a terminal?

@vamseeachanta
Hello sir good morning , i have researched regarding sympy equation displaying ,
To display math equations visually on screen , there are three ways :

Using Sympy in ipython
Using IPython library in Jupyter notebook
3.. Using Matplotlib in vscode

--> I have tried all of these ways , from three of this i found Ipython with Jupyter notebook and Matplotlib are better , gives more visual resulting equations . i am attaching the screenshots of three methods below , please see it .

Exploring markdown route. See repo folder below:
https://github.com/vamseeachanta/assetutilities/blob/2024aug/docs/sub_latex/markdown

https://github.com/vamseeachanta/assetutilities/blob/574e255e5e13ec955e1ed00f60b027aafbe28783/docs/sub_latex/markdown/sympy_function_2.py
https://github.com/vamseeachanta/assetutilities/blob/574e255e5e13ec955e1ed00f60b027aafbe28783/docs/sub_latex/markdown/sympy_function_2.md


Apply changes to CMake configuration

@mc-nv, I'm assuming that this is a follow up to #371 because it did not solve your problem? I'm fine with the cmake minimum version change, but can you help me understand why the project declaration needs to be moved outside of the if/else branch as we discussed on #371?

@mc-nv, I'm assuming that this is a follow up to #371 because it did not solve your problem? I'm fine with the cmake minimum version change, but can you help me understand why the project declaration needs to be moved outside of the if/else branch as we discussed on #371?

Previous commit has a CMakeLists.txt constraint where minimal required version must be declared as top-level instruction.

Note Call the cmake_minimum_required() command at the beginning of the top-level CMakeLists.txt file even before calling the project() command. It is important to establish version and policy settings before invoking other commands whose behavior they may affect. See also policy CMP0000.

Regarding removing project() command, I simply follow the book approach where according to documentation this approach is more elegant IMO.

Enables support for the named languages in CMake. This is the same as the project() command but does not create any of the extra variables that are created by the project command.

Previous commit has a CMakeLists.txt constraint where minimal required version must be declared as top-level instruction.

The real issue here is that our CMakeLists.txt is really two CMakeLists.txt files stacked on top of one another for convenience. It would be useful not to have to constrain the CMake version to something so recent if we're just going to invoke the build through Docker, but once we're working with the actual bare-metal build, we need 3.26.4. The guidance on when to call cmake_minimum_required is generally correct, but we should interpret that as calling it at the top of each branch of the if/else.
Practically, this is a concern because RAPIDS is almost always at one of the most recent CMake versions, but it would be preferable not to always impose that condition on Triton.
As for the project declaration, I'm not too opinionated on it either way. I slightly prefer the separate project declarations because it emphasizes the above idea that these are really two different build configs, but enable_language works just fine too.

@wphicks  we are happy with any solution that will solve compilation process for Triton.
We can come back to https://github.com/triton-inference-server/fil_backend/pull/372 changes if that's what you willing to see.

we are happy with any solution that will solve compilation process for Triton

Of course! That's number one priority, and I'm happy to get a temporary solution in and then clean up if necessary with further discussion. Could you file an issue with a reproducer for the problem you're seeing so we can track both the initial solution and any necessary follow-up? I want to make sure that any subsequent clean-up we do does not re-introduce the problem, and I'm not entirely sure what the problem is at the moment.

Sorry, @mc-nv, forgot to tag you in the above.

@wphicks Sorry, it was probably my fault for putting the previous PR for review before making sure it fixed the issue in our CI (the original version did, but after applying reviews it didn't). All we need is just a minimum CMake version in the docker build branch to get the build passing.

@Tabrizian Great! Thanks very much for the context. Could you go ahead and post an issue with reproducer now? I'm 100% okay with merging a fix that just gets CI green again, but I want to make sure I understand the issue. CMake's project command has supported VERSION since 3.0.2 at least (and maybe earlier), so after we get CI green, I'd like to investigate further.
For now, I can't confirm that this change gets CI green, since build.py builds are blocked by the current problem with fetching boost. Our other build path does not reproduce the issue.

Let's get this merged: https://github.com/triton-inference-server/fil_backend/pull/372
Green pipeline: https://gitlab-master.nvidia.com/dl/dgx/tritonserver/-/jobs/78848045
I'm going to file a reproducer for this issue to make sure it is covered in CI for future changes.

Accidentally merged this one, but reverted and merged #372. Thanks, @Tabrizian!


Cannot clone this repo

"git clone git@github.com:mysterycommand/vigilant-octo-robot.git" is producing an error in Git Bash on Windows 10x64.
$ git clone git@github.com:mysterycommand/vigilant-octo-robot.git
Cloning into 'vigilant-octo-robot'...
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Met you at Ember JS NYC meetup at Tumblr last Thursday night.
Thank you for help configuring Microsoft Visual Studio Code.

Finally cloned "vigilant-octo-robot" repo and was able to run it locally.

Hey dude! So, each "step" in my little demo from last week is committed to this repo as a branch. You should be able to run like git checkout <branch-name> to see the project as it grows/changes where <branch-name> is one of:

01-a-particle
02-some-particles
03-acceleration
04-other-effects
05-fancy-draw
06-interactive
07-images-animation
08-rotation
09-bursting
10-pooling
11-field-effects
12-more-fields
13-something-practical

The names aren't always super reflective of what changed in between, but they should alt least get you a little idea. You can see exactly what changed between branches by running git diff <first-branch-name> <second-branch-name> (for example, running git diff 01-a-particle 02-some-particles will show you exactly what changed between the 01 and 02 branches.
Hope that helps!

You rock. This is exciting! Thank you for feedback. Take care.


Add history to managed raw studies

The history will be a list of commands (like Variant) which will be added after each study modification.
The history will be reversible. (we can revert a set of last commands).

Do you want to implement an undo/redo mechanism?


Test failure on Chromium Linux

Interesting that it's so close. Maybe we're running into some anti-aliasing? Should we sample from a different location?


Integrate common base mode colors into public dashboard

Added in support for e-mission-common into em-public-dashboard, and then utilized e-mission-common's base mode colors in the dashboard to unify colors across platforms.

Update
After a bit of time trying to find my way around the public dashboard, I managed to...

Get e-mission-common installed for use by the visualization scripts on the dev environment.
Modify the mapping_color_labels function to map combined_mode_values and sensed_values to the colors in e-mission-common.

The way I mapped the colors was a bit weird as I ran into the following issue. In mapping_color_labels, here is an example of the type of modes we are expected to map to colors:
combined_mode_values:
['Gas Car, drove alone', 'E-car, drove alone', 'Bus', 'Train', 'Free Shuttle', 'Taxi/Uber/Lyft', 'Gas Car, with others', 'E-car, with others', 'Bikeshare', 'Scooter share', 'E-bike', 'Walk', 'Skate board', 'Regular Bike', 'Not a Trip', 'No Travel', 'Same Mode', 'Other']

sensed_values:
['WALKING', 'BICYCLING', 'IN_VEHICLE', 'AIR_OR_HSR', 'UNKNOWN', 'OTHER', 'Other']

Here are the BASE_MODES found in e-mission-common that we will be using for the mapping:

BASE_MODES =
BASE_MODES = {
  # BEGIN MotionTypes
  "IN_VEHICLE": {
      "icon": 'speedometer',
      "color": mode_colors['red'],
      "met": NON_ACTIVE_METS,
      # footprint not known; left undefined. later filled in by an average of:
      # CAR, BUS, LIGHT_RAIL, TRAIN, TRAM, SUBWAY
  },
  "BICYCLING": {
      "icon": 'bike',
      "color": mode_colors['green'],
      "met": BIKING_METS,
      "footprint": {},
  },
  "ON_FOOT": {
      "icon": 'walk',
      "color": mode_colors['blue'],
      "met": WALKING_METS,
      "footprint": {},
  },
  "UNKNOWN": {
      "icon": 'help',
      "color": mode_colors['grey'],
      # met and footprint not known; left undefined
  },
  "WALKING": {
      "icon": 'walk',
      "color": mode_colors['blue'],
      "met": WALKING_METS,
      "footprint": {},
  },
  "AIR_OR_HSR": {
      "icon": 'airplane',
      "color": mode_colors['orange'],
      "met": NON_ACTIVE_METS,
      "footprint": AIR_FOOTPRINT,
  },
  # END MotionTypes
  "CAR": {
      "icon": 'car',
      "color": mode_colors['red'],
      "met": NON_ACTIVE_METS,
      "footprint": CAR_FOOTPRINT,
  },
  "E_CAR": {
      "icon": 'car-electric',
      "color": mode_colors['pink'],
      "met": NON_ACTIVE_METS,
      "footprint": E_CAR_FOOTPRINT,
  },
  "PHEV_CAR": {
      "icon": 'car-electric',
      "color": mode_colors['pink'],
      "met": NON_ACTIVE_METS,
      "footprint": PHEV_CAR_FOOTPRINT,
  },
  "E_BIKE": {
      "icon": 'bicycle-electric',
      "color": mode_colors['green'],
      "met": E_BIKING_METS,
      "footprint": E_BIKE_FOOTPRINT,
  },
  "E_SCOOTER": {
      "icon": 'scooter-electric',
      "color": mode_colors['periwinkle'],
      "met": NON_ACTIVE_METS,
      "footprint": E_SCOOTER_FOOTPRINT,
  },
  "MOPED": {
      "icon": 'moped',
      "color": mode_colors['green'],
      "met": NON_ACTIVE_METS,
      "footprint": MOPED_FOOTPRINT,
  },
  "TAXI": {
      "icon": 'taxi',
      "color": mode_colors['red'],
      "met": NON_ACTIVE_METS,
      "footprint": TAXI_FOOTPRINT,
  },
  "BUS": {
      "icon": 'bus-side',
      "color": mode_colors['magenta'],
      "met": NON_ACTIVE_METS,
      "footprint": { "transit": ["MB", "RB", "CB"] }, # fixed-route bus, bus rapid transit, commuter bus
  },
  "AIR": {
      "icon": 'airplane',
      "color": mode_colors['orange'],
      "met": NON_ACTIVE_METS,
      "footprint": AIR_FOOTPRINT,
  },
  "LIGHT_RAIL": {
      "icon": 'train-car-passenger',
      "color": mode_colors['periwinkle'],
      "met": NON_ACTIVE_METS,
      "footprint": { "transit": ["LR"] } # light rail
  },
  "TRAIN": {
      "icon": 'train-car-passenger',
      "color": mode_colors['periwinkle'],
      "met": NON_ACTIVE_METS,
      "footprint": { "transit": ["LR", "HR", "YR", "CR"] } # light rail, heavy rail, hybrid rail, commuter rail
  },
  "TRAM": {
      "icon": 'tram',
      "color": mode_colors['periwinkle'],
      "met": NON_ACTIVE_METS,
      "footprint": { "transit": ["SR"] } # streetcar
  },
  "SUBWAY": {
      "icon": 'subway-variant',
      "color": mode_colors['periwinkle'],
      "met": NON_ACTIVE_METS,
      "footprint": { "transit": ["HR"] } # heavy rail
  },
  "FERRY": {
      "icon": 'ferry',
      "color": mode_colors['taupe'],
      "met": NON_ACTIVE_METS,
      "footprint": { "transit": ["FB"] } # ferry boat
  },
  "TROLLEYBUS": {
      "icon": 'bus-side',
      "color": mode_colors['taupe'],
      "met": NON_ACTIVE_METS,
      "footprint": { "transit": ["TB", "SR"] } # trolleybus, streetcar
  },
  "UNPROCESSED": {
      "icon": 'help',
      "color": mode_colors['grey'],
      # met not known; left undefined
      # footprint not known; left undefined
  },
  "OTHER": {
      "icon": 'pencil-circle',
      "color": mode_colors['taupe'],
      # met not known; left undefined
      # footprint not known; left undefined
  },
};

While there is a clear mapping between values in sensed_values and BASE_MODES, not all values in combined_mode_values have a clear mapping to BASE_MODES (i.e. Gas Car, drove alone vs CAR, E-car, drove alone vs E_CAR). My solution to this was to trim the modes in combined_mode_values to only keep the string on the left side of a comma (Gas Car, drove alone --> Gas Car), which got us closer to the value we cared about. I then used the difflib package from python and called get_close_matches, which scans BASE_MODES to find the closest match, giving us the color we want. This was unfortunately a bit of a long winded way to do this, but I wasn't too sure what other way I could do this without hard coding cases.
Results
Below is a plot generated using the new color mappings.

Note: As of right now I only tested the new colors with labeled data as that is what was provided using the data from the "loading test data" tutorial. I tried downloading my own json dumps for some sensed data, but we never fixed that json dump issue, so I wasn't able to. That could be a good thing to work on next after this.
Tasks Remaining

Using my current mapping implementation, Regular Bike maps to OTHER instead of BICYCLING. Going to need to figure out how to fix that, maybe I have to hardcode it because even Bike --> BICYCLING is a stretch.
Integrate e-mission-common into the production docker-compose.yml.

Further Public Dashboard Questions
I managed to get this fix in, but I still have some questions wrt how the dashboard works.

How do I generate all the plots easily? Right now I've been running each notebook individually, but I keep on running into issues where example_function() doesn't exist, so I have to find example_function(), run that notebook first, and then I can run what I want. I just feel like there is an easier option that I am missing.
This is more out of curiosity, but how does all of this get setup for production? Is all of the software launched using the production docker-compose.yml, like it is for development, and then hosted on a server somewhere? This whole setup is a super interesting way to tie multiple parts of e-mission together in an elegant way.

@Abby-Wheelis for visibility and review

How do I generate all the plots easily?

If you are using the dev dockerfile, you need to run the notebooks in the notebooks to generate the plots (always run the mapping notebook first). You can also use the non-dev dockerfile, and then run the notebooks from the command line: there are some instructions here, or I'm happy to show you. You'll still need to run the mapping notebook first.

Figure out how to install e-mission-common into the production version

I think we already use e-mission-common in the survey notebook, would you be able to use it here in a similar way?
For the matching process, some deployments have custom labels, can you check and make sure that your process will work with those labels too?

I met up with @Abby-Wheelis yesterday and she helped clarify a lot of the confusions I had with the public dashboard, along with giving me some great ideas on how to handle custom labels.
Custom Labels
For the custom labels, instead of doing it in the hacky pattern matching way I previously used, her idea was to use the baseMode field in the unprocessed dynamic_labels variable for the mapping. This was a great idea as we can now be sure that no matter what custom label there is, the mapping will be correct. That change can now be found in my latest pushed commit.
I have only been able to test my code by hardcoding in the dynamic_labels variable using values from
nrel-openpath-deploy-configs/label_options, and then manually checking the mapping to ensure the colors were matched correctly. Once we get the permissions issue cleared up with the mongo dumps I can try it using actual data and generate some plots to see how it looks.
Default Labels
I am still not quite happy with how I handle mapping the default labels to colors, but I don't know if there is a better way to do it. If there are no custom labels, we default to using the labels found in viz_scripts/auxillary_files/mode_labels.csv, which doesn't include a baseMode column. This is why I did my hacky pattern matching solution as there was no clear mapping between the labels provided and the baseModes we have. The only alternative I can think of is to add a baseModes column into mode_labels.csv, but that seems even worse. I'll keep it how it is for the time being as it works, but it could be worth refactoring if we figure out a better way to do it.
Dependency Issue
In terms of utilizing e-mission-common in the dashboard, Abby filled me in on the fact that e-mission-server actually installs it as a dependency, so I got rid of my code that installs it directly on the dashboard. I did notice, however, that e-mission-server is using<EMAIL_ADDRESS>which does not include the base mode features as that was released in 0.5.4. So before these changes go out we need to bump that up in e-mission-server. I can make a PR for that, but I haven't worked on the server before so I am not sure how to test the server to make sure the newer release doesn't break it.

I was finally able to test out the new colors using some better mongo dumps. Below are how all of the plots in generic_metrics.ipynb look using datasets from the Laos and Ride 2 Own studies. I chose those two datasets so we can check to make sure both the dynamic and default labels appear correctly. Let me know what you think about these @Abby-Wheelis and if you'd like to see any others!
Laos Data (Dynamic Labels)

Ride 2 Own (Default Labels)

@louisg1337 The labels look like they're working, so that's good!
Is there anything we can do to make the variations on the same base mode different from one another? In the in-app dashboard I think something like "gas car with others" would show up as a lighter/darker shade of the same red as "gas car drove alone"

@louisg1337

The only alternative I can think of is to add a baseModes column into mode_labels.csv, but that seems even worse. I'll keep it how it is for the time being as it works, but it could be worth refactoring if we figure out a better way to do it.

I had a discussion with @JGreenlee yesterday. In regards with the aspect of Default Labels not having label_options, and it's dependency on using mode_labels.csv. Would it be a good idea to host a default label_options in the e-mission-commons repo itself? With that we can make use of richModes/baseModes without having to rely on either transforming the mode_labels.csv or the using current approach which you've suggested might need some refactoring.

I did notice, however, that e-mission-server is using<EMAIL_ADDRESS>which does not include the base mode features as that was released in 0.5.4

We could make the following changes in the local system, in the environment36.dashboard.additions.yml file in Public Dashboard to make use of the latest changes on the e-mission-common repo.
dependencies:
- pip:
  ...
  - git+https://github.com/JGreenlee/e-mission-common@master

Then re-build the docker with docker-compose -f docker-compose.dev.yml build

I just added https://github.com/JGreenlee/e-mission-common/blob/master/src/emcommon/resources/label-options.default.json to e-mission-common, which should help with this

Is there anything we can do to make the variations on the same base mode different from one another? In the in-app dashboard I think something like "gas car with others" would show up as a lighter/darker shade of the same red as "gas car drove alone"

Yes, we made dedupColors on the phone https://github.com/e-mission/e-mission-phone/blob/90595e9d2a5cdb4acbd47e8e4f224610f56c832d/www/js/components/charting.ts#L165
Ideally we want this to be moved to e-mission-common so both dashboards use the same dedup method.
But the current dedupColors depends on the JS color library; we need a Python equivalent

It seems like Python's built-in colorsys will be good enough for this
https://stackoverflow.com/a/60562502/5110347
So we won't need an extra dependency

I had a discussion with @JGreenlee yesterday. In regards with the aspect of Default Labels not having label_options, and it's dependency on using mode_labels.csv. Would it be a good idea to host a default label_options in the e-mission-commons repo itself? With that we can make use of richModes/baseModes without having to rely on either transforming the mode_labels.csv or the using current approach which you've suggested might need some refactoring.

Yes, this helps tremendously, thanks for suggesting this change! This definitely helps streamline the code and makes it way more consistent. Thank you also @JGreenlee for also implementing it, that'll be the next thing I try to work in.

We could make the following changes in the local system, in the environment36.dashboard.additions.yml file in Public Dashboard to make use of the latest changes on the e-mission-common repo.

Perfect, that sounds like a great way to deal with that issue!

Yes, we made dedupColors for this purpose on e-mission-phone https://github.com/e-mission/e-mission-phone/blob/90595e9d2a5cdb4acbd47e8e4f224610f56c832d/www/js/components/charting.ts#L165

So I was actually working on that this morning, I just pushed out a working version that allows us to lighten each occurrence of the same color. Here is one plot that showcases how it looks.

I definitely do agree though that this should be included in e-mission-common, maybe we can use the code I just pushed out as a baseline to work off of as I also utilized the colorsys package. Also, I do think my implementation will need to be refactored slightly, as I saw in dedupColors it supports infinite occurrences of the same color, while mine only supports 5 shades lighter.

Also, I do think my implementation will need to be refactored slightly, as I saw in dedupColors it supports infinite occurrences of the same color, while mine only supports 5 shades lighter.

Until we unify this, I figured I'd try to fix the above issue. I took a slightly different approach from the dedupColors, which is instead of evenly spacing out an adjustment per color, I based the adjustment off of an inverse exponential decay function.

It was just an easier change to make than to have to refactor everything which I would've had to do if I took the same approach found in dedupColors.
The function can handle an infinite amount of occurrences as it will always stay within the max adjustment value set.

Like stated above, this will be refactored so none of this is set in stone, its just a placeholder for the time being.

We could make the following changes in the local system, in the environment36.dashboard.additions.yml file in Public Dashboard to make use of the latest changes on the e-mission-common repo.

I just implemented this in the latest push, and I've run into an issue. When I try to run the python notebooks, I get this error:
---> 73 async def get_egrid_region(coords: list[float, float], year: int) -> str | None:
     74     """
     75     Get the eGRID region at the given coordinates in the year.
     76     """
     77     if year < 2018:

TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'

The error is apparently because the str | None syntax is only available in Python 3.10 (StackOverflow). Unfortunately, we are using Python 3.9 in this docker environment. I'm not sure if the solution to this is to replace the syntax in e-mission-common or to bump up the version of Python. If it is the latter, I am also not quite sure how to do that. My impression is that Python 3.9 gets installed using the e-mission-server docker image, and if that is the case thats a whole other can of worms to deal with trying to bump up the version there. Would love to hear some feedback of what you guys think!

I think it just needs from __future__ import annotations
I should configure the test environment in e-mission-common to also use Python 3.9 so we can catch these kinds of issues sooner

Thanks for the help with trying to figure that all out @JGreenlee ! It turns out the from __future__ import annotations doesn't work which is unfortunate. I created a PR here that removes the str | None type hints from e-mission-common, which ended up fixing the issue.
Here are also the plots generated with the new coloring logic I used. Let me know what you guys think about it and whether or not the differences between duplicate colors are noticeable enough. If it isn't enough, I can try to refactor it a bit to get more of a difference between colors.

In this new version, they look a bit too similar to me. I vaguely remember going through this same process on the phone, which is what led me to design dedupColors the way I did
dedupColors makes the colors as far apart as it can with respect to the number of duplicates there are. With 2-3 duplicates the colors are easily distinguishable; with 4-5+ duplicates they become harder to distinguish (but still distinct)
If you implement dedupe_colors in Python you should be able to use it on the dashboard. Something like:
colors_mode = dedupe_colors([
    [mode, translations_to_basemodes.get(mode, "UNKNOWN")]
    for mode in combined_mode_values
])

Thats a great point @JGreenlee. I definitely agree the colors are a bit too similar, so I took your advice and implemented dedupe_colors in e-mission-common. The PR for that can be found here. I'm not sure how we are going to convert the python code to javascript as we are using the colorsys python library, so I kept it as a draft until we figure that out. I attached some plots below to showcase how the new colors look, let me know what you guys think. If need be, I can always adjust the variance in lightness that we can go so we can get more drastic differences.
Normal Plot

Three Reds + Two Blues At Once

Four Greens At Once

Four Pinks At Once

I just added https://github.com/JGreenlee/e-mission-common/blob/master/src/emcommon/resources/label-options.default.json to e-mission-common, which should help with this
The .csv files will phased out

The above label_options.default.json doesn't have "translations" object. We need the translations for mapping from MODE.value to mode_clean.
For example: MODE.value = e_car_drove_alone which has baseMode = "E_CAR" to mode_clean = "E-car, drove alone".
Also, I think it would be a good idea to match all the values in mode_labels.csv, like train_and pilot e-bike, which translates to "Train".

On the phone, when translations are not included in the label options, they are taken from the language JSON file
https://github.com/e-mission/e-mission-phone/blob/adb7cf235ded412119581a1ef8bf656d0ec93855/www/i18n/en.json#L160
These are the text that gets shown to the user.
I don't think this lines up with mode_clean, though

I don't understand where the mode_clean values comes from. A mode like train_and pilot e-bike appears to have been a custom label. So did someone manually map it to "Train"?
I thought that the public dashboard just lumped custom modes into "Other"

I realize I'm jumping in mid-process here, so please feel free to correct me if I've misinterpreted!

So did someone manually map it to "Train" at some point?

Yes, I think that is the case, I remember these being useful when I was processing the canbike CO data, so I'm guessing these date from the early stages of the dashboard, but all other custom labels get lumped into "Other", so I think we're safe to do that with all user-provided labels.
As for where they come from - that depends if we're talking about "default" or "custom" labels.
Default labels currently get read from csvs in: mapping_dictionaries.ipynb which are then used as parameters to the functions in scaffolding.py that process the data (ie expanded_ct['Mode_confirm'] = expanded_ct['mode_confirm'].map(dic_re)). It sounds like this is what we're looking to phase out in favor of a single source of default labels/label translations, which makes a lot of sense to me.
If there are custom labels for the deployment, we pass those in as dynamic_labels and use them instead of the mapping dictionaries. If we wanted to centralize the labels, and get rid of our csvs, I think that would not be too hard to do, we could refactor mapping_dictionaries.py.

The above label_options.default.json doesn't have "translations" object. We need the translations for mapping from MODE.value to mode_clean.

I see what you mean, for dynamic labels we expect the translation objects, and for the default labels they're hardcoded into the csvs.

On the phone, when translations are not included in the label options, they are taken from the language JSON file

Can you remind me where this comes from? If it comes from e-mission-translate then that's not something that we currently use on the dashboard, but could probably build in, we just don't offer any other parts of the dashboard in >1 language. Else maybe we could use the default files from nrel-openpath-deploy-configs when a custom file is not specified? Or is the default going to be stored in em-common and not nrel-openpath-deploy-configs?

Can you remind me where this comes from?

For English, it comes from e-mission-phone, www/i18n/en.json, "multilabel". https://github.com/e-mission/e-mission-phone/blob/adb7cf235ded412119581a1ef8bf656d0ec93855/www/i18n/en.json#L160
The other languages are in e-mission-translate.
I think we have a few options:

Have em-public-dashboard retrieve en.json from e-mission-phone
Extract the English "multilabel" translations to a new file (multilabel_en.json?) and put it in e-mission-common. The other languages would stay in e-mission-translate
Instead of having "multilabel" in en.json, es.json etc, always include translations in the label-options json
Move en.json to e-mission-translate and have the public dashboard retrieve en.json from there

As for where they come from - that depends if we're talking about "default" or "custom" labels.

Can we get rid of the concept of "default" labels?
I think there should be 2 kinds of labels: 1) labels specified by the deployment, and 2) labels given by the user

Can we get rid of the concept of "default" labels?
I think there should be 2 kinds of labels: 1) labels specified by the deployment, and 2) labels given by the user

I agree that would make the most sense, but unless we update all of the configs, we're left with 1) deployments with their own file in nrel-openpath-deploy-configs/label_options and 2) those who specify Mulitlabel, but not their own labels (so we use our defaults)

Instead of having "multilabel" in en.json, es.json etc, always include translations in the label-options json

Could we use [example-study-label-options.json](https://github.com/e-mission/nrel-openpath-deploy-configs/blob/main/label_options/example-study-label-options.json) and [example-program-label-options.json](https://github.com/e-mission/nrel-openpath-deploy-configs/blob/main/label_options/example-program-label-options.json) as the files if the deployer does not specify their own? This would definitely streamline the process on the public dashboard end. These also currently include translations, does that mean we have translations in multiple places for the same set of labels?

Can we get rid of the concept of "default" labels?
I think there should be 2 kinds of labels: 1) labels specified by the deployment, and 2) labels given by the user

This would definitely be ideal and is probably how we should do it, but Abby is right with how we may still need "default" labels for any past study that never specified any.

Could we use example-study-label-options.json and example-program-label-options.json as the files if the deployer does not specify their own?

I think this would be a great idea as having everything together in one file just makes things 1) a lot easier to work with and 2) a lot easier for any future developer to understand instead of having to track down multiple files. It will also follow the same format that we have for specifying dynamic labels, so that would be good to be consistent across the board.

These also currently include translations, does that mean we have translations in multiple places for the same set of labels?

Yes, and avoiding this repetition was the rationale for supporting translations from en.json, es.json, etc, rather than always putting all translations in the label-options JSON. We wanted to centralize all the translations as much as possible, this way future deployers would only have to provide translations for modes that we have not already supported in the past.

Could we use example-study-label-options.json and example-program-label-options.json as the files if the deployer does not specify their own?

Yes, this seems functionally the same as adding the translations to label-options.default.json. If we do this, there is no reason to still have them in en.json, es.json, etc.
I think the 'example' label options are intended as more of a resource for deployers to reference when creating their own label options, rather than to be used as an actual default.
But maybe we should just decide between "default" and "example" and use one for both.

Add "translations" to label-options.default.json in e-mission-common, taking them from en.json, es.json, and lo.json

Is there any reason why we would want to host the label-options.default.json in e-mission-common instead of nrel-openpath-deploy-configs's label_options?
We already have other label_options files in nrel-openpath-deploy-configs. It would likely be a good idea to store all label_options, even default one, in a single place.

For the meantime, I've created a PR for e-mission-common which includes adding in the translations to label options. I just wanted to get this in there so I can start working on getting that implemented in case this is the approach we go for.

Is there any reason why we would want to host the label-options.default.json in e-mission-common instead of nrel-openpath-deploy-configs's label_options?

The only reason I can think of is that resources from nrel-openpath-deploy-configs get downloaded at runtime, while resources from e-mission-common can be included as a static dependency.
Keeping it in e-mission-common avoids an extra network call.

The only reason I can think of is that resources from nrel-openpath-deploy-configs get downloaded at runtime, while resources from e-mission-common can be included as a static dependency. Keeping it in e-mission-common avoids an extra network call.

I see. We are doing the network call to download label_options whenever we have custom labels for a program.
So, it's a tradeoff between keeping all the label_options at a single place vs an extra network call.

Default label options issues
I've integrated the use of label_options.default.json into the code, and I am now running into an issue unfortunately. When I try to generate the new plots, I get this error:

I found out this is because some of the translation values in label_options.default.json don't line up with the values being used to store the data. Therefore, when matplotlib tries to map keys to colors, it throws errors. The differences between the data are subtle too, here are some of them.

emcommon labels
current labels

Gas Car Shared Ride
Gas Car, with others

Gas Car Drove Alone
Gas Car, drove alone

Taxi / Uber / Lyft
Taxi/Uber/Lyft

...
...

There are a few more instances of this, especially with the E-car labels. Would it be alright if we changed the translation values in label_options.default.json to make them consistent with what was there before @JGreenlee ?

Dedupe colors issues
With the new dedupe_color( ) in e-mission-common, the colors are appearing too dark. Here is what one of the plots look like.

Gas Car, drove alone and E-car, drove alone are both way too dark. I think that max_adjustment needs to be tweaked a bit so that the range goes from like [0.4, 1.6] to something like [0.8, 2]. Going less than 1 makes the colors dark pretty fast, so I don't know if we should venture that far below 1. And then 2 was the max I used in my previous implementation, and it seemed to be a sweet spot where the color was light, but not too light where it was white. Let me know what you think about both of these @JGreenlee.

I found out this is because some of the translation values in label_options.default.json don't line up with the values being used to store the data. Therefore, when matplotlib tries to map keys to colors, it throws errors. The differences between the data are subtle too, here are some of them.

The fundamental error here is that the colors are mapped to the display modes. They should be mapped from the base mode. The base mode is the internal representation and is the one that should be used for all calculations. It was a bad design to do the mapping from the display mode in the first place.

Both of these issues are just growing pains from trying to unify code between phone & dashboard.
The label options need to be what they are in label-options.default.json. These translations will be used on the phone for the text that gets displayed to the user, so we can't just change it to match the previous implementation of the public dashboard.
The dedupe_color issues are because the charts on the phone don't have any text on the bars, while the dashboard does have (black) text on the bars. So the dashboard cannot afford to darken colors without compromising legibility, but the phone can.
I think we can get around this pretty easily by having max_adjustment be a parameter to dedupe_colors. This way the phone can keep using [0.4, 1.6], while the public dashboard can use something like [1, 2] or whatever you determine to be a good tradeoff of distinguishability between colors and legibility of text.
You can make it an optional parameter with a default value of [1, 2] if you'd like the default behavior to be only "lightening" and not "darkening"

The fundamental error here is that the colors are mapped to the display modes. They should be mapped from the internal mode (e.g. drove_alone, not Gas Car, drove alone). The base mode is the internal representation and is the one that should be used for all calculations. It was a bad design to do the mapping from the display mode in the first place.

This makes sense and thats a great point, mapping it to the internal representation would be better, especially for supporting future translations. I guess where I am still confused is if the plots showcase display modes as the labels, don't we need a display mode to color mapping for the colors to get applied correctly? The error I have been getting is that certain keys don't exist in the color mapping because the are slightly different in label-options.default.json, so I assumed that was how it needed to be done.
Is there a different way to map the colors to the labels in matplot that I've been overlooking? Maybe something like the label values are stored as (drove_alone, "Gas car, drove alone") and we can map colors to the internal representation while the display mode still gets displayed? I'm just a bit confused as to what the process with that would look like, so any help would be appreciated @shankari .

I think we can get around this pretty easily by having max_adjustment be a parameter to dedupe_colors. This way the phone can keep using [0.4, 1.6], while the public dashboard can use something like [1, 2] or whatever you determine to be a good tradeoff of distinguishability between colors and legibility of text.

Thats a great idea, I like the idea of being able to customize it as needed. I just put out a PR with those changes. I added an additional parameter called min_adjustment because using only max_adjustment was making the colors either way too dark if the number was too low, or way too white if it was too high. As there was no happy medium, I just added that so we can keep the min and max independent. I think I also made it so that the functionality remains the same without passing in anything. Let me know if that was what you had in mind @JGreenlee .

Looks good; I tweaked it a little bit, updated the tests to reflect the new parameter, and merged just now

For the time being, I've pushed out a working version of the color mapping that does not utilize label-options.default.json. It seems like if we want to use it, it will require a lot more work refactoring the way we store and pass around the data to be displayed on the plots. That effort will probably also deserve its own branch as it doesn't seem like a quick fix. I'll continue exploring ways in which we can move away from the display representation and more towards the internal representation, but for the time being we have a working version using the old mode_labels.csv file.

I guess where I am still confused is if the plots showcase display modes as the labels, don't we need a display mode to color mapping for the colors to get applied correctly? The error I have been getting is that certain keys don't exist in the color mapping because the are slightly different in label-options.default.json, so I assumed that was how it needed to be done.

Here's what I would suggest at a high-level. To get deeper into this, I'd have to go and dig into the actual code.
Right now, my recollection is that the codebase works like this:

we construct a dataframe with a mode_confirm column. The values in mode_confirm are the internal labels (drove_alone)
the mode_confirm is mapped to a display mode column (Mode_confirm) which has values like Gas car, drove alone
we then create colors based on Mode_confirm
we should create the colors based on mode_confirm instead
if the concern is the strings in the legend, we can just create the legend manually using matplotlib (e.g. https://stackoverflow.com/questions/39500265/how-to-manually-create-a-legend or similar)

In scaffolding.py, we have mapping_labels() which gives mapping from between display mode to display mode i.e.

base mode
display mode

drove_alone
Gas Car, drove alone

e_car_drove_alone
E-car, drove alone

Currently, we are using
Dynamic labels:
mode_values = list(mapping_labels(dynamic_labels, "MODE").values()) if "MODE" in dynamic_labels else []
combined_mode_values = mode_values + replaced_mode_values + ['other']

Default mapping:
combined_mode_values = (list(OrderedDict.fromkeys(dic_re.values())) + ['Other'])

colors_mode = dict(zip(combined_mode_values, plt.cm.tab20.colors[:len(combined_mode_values)]))

If we use mapping_labels(dynamic_labels, "MODE").keys() or dic_re.keys() instead of .values(). This would help us map between base mode and color palette.

Thank you @shankari and @iantei for the help, I appreciate it a lot! I managed to refactor plot_and_text_stacked_bar_chart so that it now generates plots using the internal representations, and then we remap the labels to showcase the display representations. I've changed all of the plots in generic_metrics.ipynb to work with the new refactored function. The changes are simply swapping "Mode_confirm" to mode_confirm, and then tagging along values_to_translations as a parameter to plot_and_text_stacked_bar_chart when necessary. Here are what the plots look like:

Further Tasks
Now that the function has been refactored and we are now using label-options.default.json, all thats left is...

Testing to make sure this new implementation works with dynamic_labels.
Testing other plot generating notebooks to see what needs to be refactored so this new color implementation works.

wrt https://github.com/e-mission/em-public-dashboard/pull/143#issuecomment-2322009427, as we have discussed earlier, the mapping (drove_alone -> Gas car, drove alone) shouldn't have two copies, one in the public dashboard and one in the UI. There should be one standardized mapping.
I agree with Jack that there should really be: (i) platform specified modes and (ii) user specified modes. I am willing to have: (i) platform specified default modes, (ii) platform specified custom modes, and (iii) user specified modes.
But I don't think we should have two copies of the platform specified default modes.

wrt https://github.com/e-mission/em-public-dashboard/pull/143#issuecomment-2322009427, as we have discussed earlier, the mapping (drove_alone -> Gas car, drove alone) shouldn't have two copies, one in the public dashboard and one in the UI. There should be one standardized mapping.

I agree, and I think it would be fairly easy to refactor the way mapping_dictionaries works to pull labels from elsewhere and remove the csvs. Does that change need to be a part of this PR? If it makes sense to do it separately I would be happy to take that on as a priority next week.

Happy to defer it until later. Ideally, at the end of this quarter, we should not have any csvs.

I cloned the above repo for further testing, and found the below cases where code changes need to be made:
Changes required:

Testing other plot generating notebooks to see what needs to be refactored so this new color implementation works. The changes I made in generic_metrics.ipynb are a good example of what needs to be done to any notebook that has errors.

Changes need to be incorporated to the mode_specific_metrics notebook

colors_mode, colors_purpose, colors_sensed = scaffolding.mapping_color_labels(dynamic_labels, dic_re, dic_pur)

TypeError: cannot unpack non-iterable coroutine object

While testing with cortezebikes dataset, I encountered an issue while generating Purpose related trip charts in generic_metrics notebook

Tested the above scenario 2, with changes of integrated Inferred Labels in my another PR - with same dataset.

The generation of Number of trips for each purpose seems fine.
Need to review the code changes which impacted this.

The issue is happening when it's trying to add Recreation/Exercise bar into the Stacked Bar Chart.
It's able to plot for Other and Shopping. But throws the error when it's trying to plot for Recreation/Exercise.

Testing with dynamic labels:
Used cortez-ebikes as the dataset, while changing the  STUDY_CONFIG=usaid-laos-ev and passing dynamic_labels from usaid-laos-ev-label-options
Got the following error:
KeyError                                  Traceback (most recent call last)
Cell In[4], line 1
----> 1 colors_mode, colors_purpose, colors_sensed, values_to_translations = await scaffolding.mapping_color_labels(dynamic_labels, dic_re, dic_pur)

File /usr/src/app/saved-notebooks/scaffolding.py:244, in mapping_color_labels(dynamic_labels, dic_re, dic_pur, language)
    241 values_to_translations = {mode["value"]: labels["translations"][language][mode["value"]] for mode in labels["MODE"]}
    243 # Assign colors to mode, purpose, and sensed values
--> 244 colors_mode = dedupe_colors([
    245     [mode, BASE_MODES[value_to_basemode.get(mode, "UNKNOWN")]['color']]
    246     for mode in combined_mode_values
    247 ], adjustment_range=[1,1.8])
    248 colors_purpose = dict(zip(purpose_values, plt.cm.tab20.colors[:len(purpose_values)]))
    249 colors_sensed = dict(zip(sensed_values, [BASE_MODES[x.upper()]['color'] for x in sensed_values]))

File /usr/src/app/saved-notebooks/scaffolding.py:245, in <listcomp>(.0)
    241 values_to_translations = {mode["value"]: labels["translations"][language][mode["value"]] for mode in labels["MODE"]}
    243 # Assign colors to mode, purpose, and sensed values
    244 colors_mode = dedupe_colors([
--> 245     [mode, BASE_MODES[value_to_basemode.get(mode, "UNKNOWN")]['color']]
    246     for mode in combined_mode_values
    247 ], adjustment_range=[1,1.8])
    248 colors_purpose = dict(zip(purpose_values, plt.cm.tab20.colors[:len(purpose_values)]))
    249 colors_sensed = dict(zip(sensed_values, [BASE_MODES[x.upper()]['color'] for x in sensed_values]))

KeyError: 'baseMode'

Ah, the issue is happening because we are using display labels in expanded_ct.Trip_purpose as Recreation/Exercise while it is saved as exercise: Recreation / Exercise in label-options.default.json

Using expanded_ct.purpose_confirm and updating the mapping_color_labels for purpose similar to mode resolves the issue.
Changing
purpose_values = list(mapping_labels(labels, "PURPOSE").values()) + ['Other'] if "PURPOSE" in labels else []

to
purpose_values = [mode["value"] for mode in labels["PURPOSE"]] + ['Other']

Since I need to make code changes to this PR to resolve the above issues.
I will incorporate these fixes alongside changes with respect to mode_specific_metrics notebook. While also testing to see if these would impact changes in survey_metrics and survey_responses notebooks in a new PR and refer back to it here.

There is an issue with the current approach of using base labels. Whenever we have a label which is not present on label_options, this will cause an issue.
bar = ax.barh(y=bar_label, width=mode_prop, height=bar_height, left=bar_width, label=values_to_translations.get(label, label), color=colors[label])
Even though we provide the default label for the base label whenever the label is unavailable, this will not be able to handle colors[label], which results into error.

This was handled well earlier because we used Display mode, which we mapped either using mode_labels.csv/ purpose_labels.csv or dynamic labels; for the key which didn't had a mapping in Display mode were changed to Other using DefaultDict.
To handle this, we might need to change the base mode column in the dataframe, which I have reservation to modify.
For example, if the expanded_ct.purpose_confirm has "volunteer_work", this has to be changed into "other".

if the expanded_ct.purpose_confirm has "volunteer_work", this has to be changed into "other".

Since volunteer_work is not a part of label_option, this is exactly what we would expect and want to happen, the manual grouping of user-entered labels to our other labels that happens in the csvs is not something we can support long term, and we should stick to one set of default labels.
The goal is to have 1) default labels from emcommon 2) dynamic labels from individual deployments 3) user-entered labels (will be grouped as "Other")
I'm planning to work on this issue of unifying the labels today, though I'm realizing that its more tied up in these changes than I initially realized. @iantei maybe you and I should find a time to meet today to discuss the options for unifying the labels and how we want to split the PRs?

The goal is to have 1) default labels from emcommon 2) dynamic labels from individual deployments 3) user-entered labels (will be grouped as "Other")

That sounds good. The current limitation is we are not grouping the user-entered labels as "Other" with base mode e.g. mode_confirm, purpose_confirm.
@Abby-Wheelis Yes, I felt the changes for removal for csv is closely associated with the PR. Lets have a meeting today, and see how we can proceed further.

Since we now use a dictionary (stemming from emcommon) to look up the translations, we don't need the display mode Mode_confirm in the dataframe anymore. What we DO need is a merged_mode that handles the user-defined labels up front (by merging them into other) so that they do not throw errors when we try to find their translation or their color down the line.

Further looking into the code, the Mode_confirm is used with CO2 footprint calculation and other notebooks as well. So, maybe we can hold on to removing Mode_confirm column yet, rather just add the merge_mode column, aggregates the user defined labels into other label, for now.

We were using same color palette for both display labels - Mode_confirm and Replaced_mode. After the refactoring of the 100% Stacked Bar Chart implementation, which enabled creation of single stacked bar chart per the different trip types (like. Mode_confirm, Replaced_mode), I feel it is better not to merge color palette for Mode_confirm and Replaced_mode (or mode_confirm and replaced_mode). Having color_mode, color_replaced dictionary for color mapping is a better approach, since this limits changes if we were to alter either of the list.
In either case, we do not have baseMode or base_Mode for “no_travel” in replaced_mode. The support for base_mode/ baseMode to no_travel needs to be added in label-options.default.json. It would be helpful to have base_mode available for REPLACED_MODE, like we have it for MODE, in label-options.default.json.

Well, this has been handled gracefully, the no_travel will be been assigned as UNKNOWN mode color palette of #555555. This should still be fine.
    colors_mode = dedupe_colors([
        [mode, BASE_MODES[value_to_basemode.get(mode, "UNKNOWN")]['color']]
        for mode in set(mode_values)
    ], adjustment_range=[1,1.8])


Exporting JSON files doesn't work in IOS Safari

Consider moving to .txt files

This is an important one... I just switched to iOS and I feel like it's a bad idea continuing writing when there's no way of exporting my entries.


ganders, euphemiously

This pull request was generated by the 'mq' tool
[test]
flake rate: 0.4
logical conflict every: 100
sleep for: 300s
[pullrequest]
requests per hour: 30
deps=[e,g]

/trunk merge


chore: update readme

#349
#369  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @NathanFlurry and the rest of your teammates on  Graphite


fix broken links

Description

closes: #381, #402

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

[ ] Targeted PR against correct branch (see CONTRIBUTING.md)
[ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
[ ] Code follows the module structure standards.
[ ] Wrote unit and integration tests
[ ] Updated relevant documentation (docs/) or specification (x/<module>/spec/)
[ ] Added relevant godoc comments.
[ ] Added a relevant changelog entry to the Unreleased section in CHANGELOG.md
[x] Re-reviewed Files changed in the Github PR explorer
[ ] Review Codecov Report in the comment section below once CI passes

There are still 3 links that are broken, but I need some advice for 2 of them:

https://ibc.cosmos.network - this will be fixed when the docs website is live, which I hope will happen soon.
https://media.githubusercontent.com/media/cosmos/ics/master/spec/ics-004-channel-and-packet-semantics/packet-state-machine.png (used here).
https://github.com/cosmos/ics/blob/master/ibc/1_IBC_TERMINOLOGY.md (used here).

I also updated the links that were broken in go and proto files. I didn't touch the generated pb.go files. Let me know if I should do something else regarding this.

* https://ibc.cosmos.network - this will be fixed when the docs website is live, which I hope will happen soon.

This is totally fine, let's ignore for now

* https://media.githubusercontent.com/media/cosmos/ics/master/spec/ics-004-channel-and-packet-semantics/packet-state-machine.png (used [here](https://github.com/cosmos/ibc-go/blob/main/docs/ibc/apps.md#packet-flow-handling)).

We can use this

* https://github.com/cosmos/ics/blob/master/ibc/1_IBC_TERMINOLOGY.md (used [here](https://github.com/cosmos/ibc-go/blob/main/modules/core/spec/01_concepts.md)).

I'm in favor of removing. The document was outdated and deprecated. It can be found here

I also updated the links that were broken in go and proto files. I didn't touch the generated pb.go files. Let me know if I should do something else regarding this.

If you changed links in .proto files, you need to run make proto-all to update the .pb.go and proto docs file

Codecov Report

Merging #415 (40b22e9) into main (06c2d76) will increase coverage by 0.03%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #415      +/-   ##
==========================================
+ Coverage   78.84%   78.87%   +0.03%
==========================================
  Files         117      118       +1
  Lines        9627     9661      +34
==========================================
+ Hits         7590     7620      +30
- Misses       1652     1655       +3
- Partials      385      386       +1

Impacted Files
Coverage Δ

modules/apps/transfer/keeper/msg_server.go
77.77% <ø> (ø)

modules/apps/transfer/keeper/relay.go
87.01% <ø> (ø)

modules/apps/transfer/types/trace.go
82.35% <ø> (ø)

modules/core/02-client/keeper/client.go
98.37% <ø> (ø)

modules/core/04-channel/keeper/handshake.go
87.40% <ø> (ø)

modules/core/24-host/keys.go
0.00% <ø> (ø)

modules/core/24-host/validate.go
100.00% <ø> (ø)

modules/apps/transfer/module.go
58.00% <0.00%> (-1.49%)
:arrow_down:

modules/core/keeper/grpc_query.go
0.00% <0.00%> (ø)

modules/core/05-port/keeper/grpc_query.go
88.46% <0.00%> (ø)

... and 2 more


Add support for KMS_KEY_ID on Redshift UNLOAD

Redshift file unload supports defining KMS_KEY_ID for UNLOAD query. "Specifies the key ID for an AWS Key Management Service (AWS KMS) key to be used to encrypt data files on Amazon S3."

https://docs.aws.amazon.com/redshift/latest/dg/r_UNLOAD.html

Adding support for it in Wrangler command would enable encrypting the data e.g. into another account / other KMS key.
https://github.com/awslabs/aws-data-wrangler/blob/1.0.4/awswrangler/db.py#L1072-L1082

Hi @jar-no1 ! Thanks for the suggestion above!
I added kms_key_id, max_file_size and region to wr.db.unload_redshift() and wr.db.unload_redshift_to_files() on our dev branch.
Could you test it and give us feedback?
To install:
pip install git+https://github.com/awslabs/aws-data-wrangler.git@dev
Development docs

Available on version 1.1.0


Experimental: Docs tools

A little experiment to add basic sassdoc and yard support.

Yeah totally, GDS do some stuff with sassdoc where the use the generated JSON to make some custom templates. Primer do a similar thing with yard so we can pull out bits to use in a more readable way I reckon.


Poc sqlite

Needs discussion; first preview

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Basic APISIX scenarios continue to show correct:
# No routes at the start
$ curl -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' http://localhost:9180/apisix/admin/routes

{"header":{"revision":"15"},"message":"Key not found"}

# Add route 1
$ curl -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '{
  "methods": ["GET"],
  "uris": ["/get"],
  "upstream": {
    "nodes": {
      "httpbin.org:80": 1
    }
  }
}' http://localhost:9180/apisix/admin/routes/1

{"key":"/apisix/routes/1","value":{"priority":0,"uris":["/get"],"methods":["GET"],"update_time":1692857760,"create_time":1692857760,"id":"1","upstream":{"pass_host":"pass","nodes":{"httpbin.org:80":1},"type":"roundrobin","hash_on":"vars","scheme":"http"},"status":1}}

# Add route 2
$ curl -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '{
  "methods": ["GET"],
  "uris": ["/foo"],
  "upstream": {
    "nodes": {
      "httpbin.org:80": 1
    }
  }
}' http://localhost:9180/apisix/admin/routes/2

{"key":"/apisix/routes/2","value":{"priority":0,"uris":["/foo"],"methods":["GET"],"update_time":1692857777,"create_time":1692857777,"id":"2","upstream":{"pass_host":"pass","nodes":{"httpbin.org:80":1},"type":"roundrobin","hash_on":"vars","scheme":"http"},"status":1}}

# List only the 1st routes even though
$ curl -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' http://localhost:9180/apisix/admin/routes

{"createdIndex":16,"value":{"priority":0,"uris":["/get"],"methods":["GET"],"update_time":1692857760,"create_time":1692857760,"id":"1","upstream":{"pass_host":"pass","nodes":{"httpbin.org:80":1},"type":"roundrobin","hash_on":"vars","scheme":"http"},"status":1},"modifiedIndex":16,"key":"/apisix/routes/1"}

# Individual route details show
$ curl -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' http://localhost:9180/apisix/admin/routes/1

{"createdIndex":16,"value":{"priority":0,"uris":["/get"],"methods":["GET"],"update_time":1692857760,"create_time":1692857760,"id":"1","upstream":{"pass_host":"pass","nodes":{"httpbin.org:80":1},"type":"roundrobin","hash_on":"vars","scheme":"http"},"status":1},"modifiedIndex":16,"key":"/apisix/routes/1"}

# Individual route details show
$ curl -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' http://localhost:9180/apisix/admin/routes/2

{"createdIndex":17,"value":{"priority":0,"uris":["/foo"],"methods":["GET"],"update_time":1692857777,"create_time":1692857777,"id":"2","upstream":{"pass_host":"pass","nodes":{"httpbin.org:80":1},"type":"roundrobin","hash_on":"vars","scheme":"http"},"status":1},"modifiedIndex":17,"key":"/apisix/routes/2"}

APISIX Route rules only work for the 1st route
# Now the first route seems to work
$ curl localhost:9080/get
{
  "args": {},
  "headers": {
    "Accept": "*/*",
    "Host": "localhost",
    "User-Agent": "curl/8.1.2",
    "X-Amzn-Trace-Id": "Root=1-64e6f5f0-65954e3038b40cdd62b74565",
    "X-Forwarded-Host": "localhost"
  },
  "origin": "<IP_ADDRESS>, <IP_ADDRESS>",
  "url": "http://localhost/get"
}

# Second route does not seem to work
$ curl localhost:9080/foo

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>404 Not Found</title>

Not Found

The requested URL was not found on the server.  If you entered the URL manually please check your spelling and try again.

Routes are shown in the raw etcdctl calls:
$ etcdctl --endpoints=localhost:12379  get /apisix/routes --prefix  -w=fields

"ClusterID" : 0
"MemberID" : 0
"Revision" : 17
"RaftTerm" : 0
"Key" : "/apisix/routes/1"
"CreateRevision" : 16
"ModRevision" : 16
"Version" : 0
"Value" : "{\"priority\":0,\"uris\":[\"/get\"],\"methods\":[\"GET\"],\"update_time\":1692857760,\"create_time\":1692857760,\"id\":\"1\",\"upstream\":{\"pass_host\":\"pass\",\"nodes\":{\"httpbin.org:80\":1},\"type\":\"roundrobin\",\"hash_on\":\"vars\",\"scheme\":\"http\"},\"status\":1}"
"Lease" : 0
"Key" : "/apisix/routes/2"
"CreateRevision" : 17
"ModRevision" : 17
"Version" : 0
"Value" : "{\"priority\":0,\"uris\":[\"/foo\"],\"methods\":[\"GET\"],\"update_time\":1692857777,\"create_time\":1692857777,\"id\":\"2\",\"upstream\":{\"pass_host\":\"pass\",\"nodes\":{\"httpbin.org:80\":1},\"type\":\"roundrobin\",\"hash_on\":\"vars\",\"scheme\":\"http\"},\"status\":1}"
"Lease" : 0
"More" : false
"Count" : 2


Change location of target/rls directory

Is there a way to accomplish this? Quoting https://github.com/rust-lang/rls/issues/870#issuecomment-389286606

Language clients need to use the LSP workspace/didChangeConfiguration to pass the rust.target_dir key

Yes, you can follow the documentation here to add a JSON file with custom workspace settings.

Thanks! I created VSWorkspaceSettings.json in .vs and tried various properties but they don't seem to get picked up. Any ideas?
{
  "rust.target_dir" : "test_target/rls"
}

as well as plain target_dir and RustLanguageExtension.target_dir


Editing and mongoose were not able to find anymore.

I edited a document and saved. It stays there, exactly right when I see through adminMongo. However, mongoose is not able to find it anymore. Whenever I edit a document, mongoose stops finding it.
What could it be?
Thanks

adminMongo  uses Mongojs mainly to connect. Are you using Mongoose in your own app? If so, it's likely an issue with your own app, not adminMongo.

mrvautin, it looks like when it's edited/saved through admin mongo, it changes something on the document structure that my app is not able to list anymore. How can it be a problem at app if before editing at adminMongo it was able to list, and after editing ad adminMongo not anymore?

I've just downloaded another mongodb client (I'd indeed rather adminMongo), however, on the new mongodb client, after editing document, everything keep working. There is something on adminMongo corrupting some indexes or something like that..

I've just found out!!
Check it:
My original document is:
{
name: string,
company: reference / foreign key
}
after edited through adminMongo it changed to:
{
name: string,
company: string
}
There is a problem at adminMongo editing.

This is related to issue #31. You will need to set an object ID using extended JSON formatting. Eg:
 { "$oid": "<id>" }
Full documentation can be found here: https://docs.mongodb.org/manual/reference/mongodb-extended-json/

mrvautin, it shouldn't be user's responsibility. Since adminMongo detects that field as ObjectID, on edit mode it should be pre-configured as: "myAttributeId" : { "$oid": "" } to be edited, not like: "myAttributeId" : "string" ...
Dont you think?

I agree. This is why I linked it to the other issue. Once that is fixed, this issue will also be fixed.


Added back to Top button and improved code formatting

closes #410
Added back to top functionality and improved code formatting

Ok sir I will make new PR
On Wed, 20 Jan 2021, 20:07 Smaranjit Ghose<EMAIL_ADDRESS>wrote:

Closed #414 https://github.com/smaranjitghose/doc2pen/pull/414.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/smaranjitghose/doc2pen/pull/414#event-4228236179, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/APLHP3QABGGDDBEKIJCV7EDS23TCTANCNFSM4WKKKXAQ
.

Ok sir I will make new PR
On Wed, 20 Jan 2021, 20:07 Smaranjit Ghose<EMAIL_ADDRESS>wrote:

Closed #414 https://github.com/smaranjitghose/doc2pen/pull/414.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/smaranjitghose/doc2pen/pull/414#event-4228236179, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/APLHP3QABGGDDBEKIJCV7EDS23TCTANCNFSM4WKKKXAQ
.