How to prevent UNK from being generated at inference time?

For applications other than MT (e.g., image captioning, conversation modeling, etc.) it's crucial to prevent UNK from being generated at inference time. For greedy decoding, I could fix this by modifying the sample() function of the GreedyEmbeddingHelper class to simply choose top_2 (instead of argmax) indices with the highest prob and choose the one that's not UNK. However, the same idea doesn't work for beam search. I'm modifying the choose_top_k() function of the inference.beam_search as follows:
def choose_top_k(scores_flat, config):
  K = config.beam_width
  UNK_ID = int(config.vocab_size) - 3
  # Instead of top-K, retrieve top-K+1 in case the UNK token appears in top K.
  next_beam_scores, word_indices = tf.nn.top_k(scores_flat, k=K+1)

  # Get a Boolean condition tensor which can indicate the UNK index
  condition = tf.not_equal(word_indices[:K], [UNK_ID])

  # For any possible UNK token, copy the value of word_indices[K] instead.
  selected_indices = tf.where(condition, word_indices[:K], [word_indices[K]]*(K) )
  selected_scores = tf.where(condition, next_beam_scores[:K], [next_beam_scores[K]]*(K) )
  return selected_scores, selected_indices

When I decode using the updated beam search, the UNK token still appears in the predictions. Is there anywhere else in the beam search inference that might re-introduce the UNK token?
My inference script is simply the following:
  python -m bin.infer \
  --tasks "
    - class: DecodeText
    - class: DumpBeams
      params:
        file: ${PRED_DIR}/beams.npz" \
  --model_dir $MODEL_DIR \
  --model_params "
    inference.beam_search.beam_width: 2"\
  --input_pipeline "
    class: ParallelTextInputPipeline
    params:
      source_files:
        - $DEV_SOURCES" \
  > ${PRED_DIR}/predictions.txt

Thanks

Look here

Thanks for the link, however, I had seen this UNK replacement before and it's not a solution for applications such as image captioning or conversational modeling. What I would need is not an UNK 'replacement' but a 'prevention' of UNK generation at inference time.

@nasrinm If I am understanding correctly you want unk_replace option for beam search?
unk_replace which is UNK token replacement using the copy mechanism is a parameter for DecodeText Task during inference. In the current implementation it uses attention mechanism to generate the attention_scores for the encoded input. But to my understanding it currently only works for beam of width 1 (basically without beam search).
I am also interested in using unk_replace using copy mechanism for beam search (beam_width > 1). Although I think using the current implementation it's not possible to do so. Why? Read this lines from here

The array file can be loaded used numpy and will contain a list of arrays with shape [target_length, source_length]

Which is verify if you look at task DumpAttention inside file dump_attention.py at function _get_scores (which dump numpy attention scores) that output numpy array is a list of array of shape [target_length, source_length]. Therefore the code right now generates attention scores for only one target (not for multiple targets as in case of beam search). Infact Issue #174 is generated because of the same limitation of the current implementation as it does not generate attention scores for multiple beams.
Anyways I would appreciate if someone here in the community can suggest good way to implement this.

@nasrinm I think your idea and code is all right then. But how you solve the problem then ?

@nasrinm Finding a bug. The scores_flat is flatten from [batch_size, vocab_size]. So unk index appear in multiple indexs: vocab_size-3, 2vocab-3, ..., batch_sizevocab_size -3.They must be all masked.
A simple fix is mask the score([batch_size, vocab_size]) 's unk index to 0, So they can't choosen by top_k.
And this is one simple way to handle unk problem. Another is use attention score to replace the unk when do beam search, this may change a bit codes and structure then. Wish to exchange. Thanks.


fix: ensure that tests db only gets configured in test mode

Description
This pull request (PR) ensures that pytest uses the test database during execution. Additionally, I have made the following changes:

Included a backoff decorator to retry calling the pull_docker_image function logic in case a DockerError exception occurs.
Added docstrings to the remove_db function for better documentation.
Merged the latest updates from the remote main branch into this branch to prevent any potential merge conflicts.

@mmabrouk this pr looks good and is ready to be merged. 👍🏾

Data are no longer being stored in the primary (v2) database, but instead in the test database. Could you please investigate this matter further, @techemmy?

Okay


Get error when trying to delete file

get error on any delete request
TypeError: Cannot read property 'toString' of null
at /home/bogdan/projects/dms/server/node_modules/lodash-id/src/index.js:37:51
at baseFindIndex (/home/bogdan/projects/dms/server/node_modules/lodash/lodash.js:794:11)
at findIndex (/home/bogdan/projects/dms/server/node_modules/lodash/lodash.js:7250:14)
at Function.find (/home/bogdan/projects/dms/server/node_modules/lodash/lodash.js:5071:21)
at Function.getById (/home/bogdan/projects/dms/server/node_modules/lodash-id/src/index.js:35:17)
at /home/bogdan/projects/dms/server/node_modules/json-server/lib/server/mixins.js:26:25
at /home/bogdan/projects/dms/server/node_modules/lodash/lodash.js:4917:15
at baseForOwn (/home/bogdan/projects/dms/server/node_modules/lodash/lodash.js:3002:24)
at /home/bogdan/projects/dms/server/node_modules/lodash/lodash.js:4886:18
at Function.forEach (/home/bogdan/projects/dms/server/node_modules/lodash/lodash.js:9334:14)

Can you share the source code of your index.js?

Well, I had the exact same error at the exact same line numbers and debugged the hell out of it and came to a conclusion that no foreign key with id in it's name should be null. The check throws an error when it finds one, so I set it to -1 and then it workes ok.
This problem basically  annoys anyone having a zero or many or zero or one relations in their tables, since it doesn't return any results on deletion. At "node_modules/json-server/lib/server/mixins.js:22:9" a check for null in value besides the foreign key, would be enough.

Just in case someone else stumbles over this. I solved this issue for me by adding some debug prints to find the offending values in our mock-db.
node_modules/json-server/lib/server/mixins.js:
          const refName = pluralize.plural(key.replace(new RegExp(`${opts.foreignKeySuffix}$`), '')); // Test if table exists

          if (db[refName]) {
            console.log("refName " + refName);
            console.log("value " + value);
            console.log("key " + key);
            // Test if references is defined in table
            const ref = _.getById(db[refName], value);

            if (_.isUndefined(ref)) {
              removable.push({
                name: collName,
                id: doc.id
              });
            }
          }


Add serde impls to LogLevel and LogLevelFilter

Fixes #143
Adds serde as an optional dependency and puts the Serialize and Deserialize traits behind cfg_attr. I wasn't sure what version I should specify. 1.0.7 was the latest when I checked, but maybe it's too specific? Would >= 1.0 be better?
Saw the post on the internals forum, wanted to help out!

Thanks! Could you also add some CI configuration to test this on Travis/AppVeyor? Also, I think the dependency here is safe to just be serde = "1.0" as >= is actually a little too eager as it may or may not be compatible with 2.0 (we're not sure if it will be).

@aergonaut are you still working on this or would you need to hand it off to someone else?

I would still like to work on this! I will try to come back to it by the end of the week.

No problem, let us know if you get stuck.

@dtolnay @alexcrichton I think this is ready now! I added cargo test --features serde to the Travis and Appveyor configs. I also fixed up the feature flag usage so it actually works.


tox: pin ceph_dev_sha1 value for ceph-volume

Since this commit [1] there's a race condition in ceph-volume when using
multiple LV in the same VG for the OSDs causing multiple scenarios to
fail.
This temporary workaround pin the ceph dev sha1 to the previous shaman
build working.
[1] https://github.com/ceph/ceph/commit/ec2e53f
Signed-off-by: Dimitri Savineau<EMAIL_ADDRESS>

iscsigws are using ceph_dev_sha1 variable for shaman setup and because the sha1 provided is only valid for ceph build then it fails on tcmu-runner...


"Jupyter: Run Cells Above Current Cell" runs each cell 3 times

Applies To

[ ] Notebooks (.ipynb files)
[X] Interactive Window and/or Cell Scripts (.py files with #%% markers)

What happened?
When I call "Jupyter: Run Cells Above Current Cell" (jupyter.runallcellsabove.palette) the interactive window shows each cell running 3 times, not just once as expected.
This happens whether I use the keyboard shortcut or manually clicking from VSCode "show all commands" (see screenshots). This happens both locally and on a remote machine I connect to using Remote-SSH.

VS Code Version
Version: 1.74.2 (Universal)
Jupyter Extension Version
v2022.11.1003412109
Jupyter logs
info 17:59:26.818: ipykernel version & path 5.3.4, ~/opt/miniconda3/envs/bc/lib/python3.6/site-packages/ipykernel/__init__.py for /Users/risno/opt/miniconda3/envs/bc/bin/python
info 17:59:26.904: Got new session 0ce36080-36bf-4290-b0e2-d28c0e05a0f6
info 17:59:26.904: Started new restart session
info 17:59:26.904: Dispose Kernel process 13420.
info 17:59:26.940: UpdateWorkingDirectoryAndPath in Kernel
info 17:59:44.684: Generated code for 1 = <ipython-input-1-963935bb3827> with 4 lines
info 17:59:46.541: Generated code for 2 = <ipython-input-2-de19de053bc8> with 3 lines
info 17:59:46.693: Generated code for 3 = <ipython-input-3-de19de053bc8> with 3 lines
info 17:59:46.884: Generated code for 4 = <ipython-input-4-801d77656feb> with 3 lines
info 17:59:47.131: Generated code for 5 = <ipython-input-5-801d77656feb> with 3 lines
info 17:59:47.393: Generated code for 6 = <ipython-input-6-801d77656feb> with 3 lines
info 17:59:47.464: Generated code for 7 = <ipython-input-7-d2f06e104278> with 3 lines
info 17:59:47.545: Generated code for 8 = <ipython-input-8-d2f06e104278> with 3 lines
info 17:59:47.663: Generated code for 9 = <ipython-input-9-d2f06e104278> with 3 lines

Coding Language and Runtime Version
Python v3.7.9
Language Extension Version (if applicable)
No response
Anaconda Version (if applicable)
No response
Running Jupyter locally or remotely?
Remote

That is strange, but I'm not seeing this myself.
Have you seen any other run commands misbehaving like this?

Have you seen any other run commands misbehaving like this?
"Jupyter: Run All Cells" and "Jupyter: Run Current Cell and Below" seem to have the same behaviour. If it's just one line or one cell, it executes just once as expected. The problem seems to be when the command runs multiple cells at the same time.


vscode: 1.56.1 -> 1.56.2

Motivation for this change
Upgrades vscode from 1.56.1 to 1.56.2
This PR was automatically generated by nixpkgs-upkeep.

CI workflow that created this PR.
Internal tag: d59694bb5aa94fb7402e0a9cf3dc7218.

Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 122928 at 4f4486db run on x86_64-linux 1

3 packages built successfully:

vscode
vscode-fhs
vscode-with-extensions

Could you also merge https://github.com/NixOS/nixpkgs/pull/122927?
On Fri, 14 May 2021, 10:24 Jörg Thalheim, @.***> wrote:

Merged #122928 https://github.com/NixOS/nixpkgs/pull/122928 into master.
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
https://github.com/NixOS/nixpkgs/pull/122928#event-4745829433, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ABV7PJ5AGYGT3SAPGGAROHLTNTM3DANCNFSM443P5BTA
.

done.


xz kmod compression

@telphan @floion @agherzan @lorenzo-stoakes
Adding 'xz' support to kmod adds 1kb in size.

LGTM

LGTM

Hmm, the title of this issue says "xz", but the commits seem to add "gzip" support?

@lurch Hi. Indeed I fixed that. This was most probably a typo. @michal-mazurek to confirm.

Closing this because we encountered some issue with debian based images which don't have support for compressed kernel modules. Fixing this involves hacks. In the meanwhile we enlarged the rootfs partition to cope with these space requirements.


TokensAnalyzer - isConstantInvocation PHP 8 issue

closes https://github.com/FriendsOfPHP/PHP-CS-Fixer/issues/5935

Coverage decreased (-0.005%) to 92.342% when pulling 3a18c3205eee02317b813d42833ca7d96f55dab8 on SpacePossum:master_5935 into f785f8817258cb07e94a751263cb6668854baa09 on FriendsOfPHP:master.

Thank you!


Internal error: illegal access (debugging ES6 proxy)

I'm using node-inspector for node 0.11.13
I start node --debug-brk ..., open node-inspector and enter the code in Chrome. Everything's fine.
Then I press 'Continue' in Chrome and, after some time, in console:
Internal error: illegal access

There is no stack at all.
Is there a way to trace communication with v8 to see at which stage it happened? It looks like a node-inspector bug.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

FWIW, the upcoming io.js 1.0.0 release should ship a much more recent version of V8, you should give it a try and see if the issue has been fixed. https://iojs.org/

This is a V8 problem (in its runtime debugger/reflection APIs), not a node-inspector problem per se.
For a project I work on, we patched the following functions in V8: Runtime_GetLocalPropertyNames, Runtime_GetLocalElementNames, Runtime_DebugGetProperty, Runtime_DebugConstructedBy to deal better with JSProxy, then rebuilt node from the patched source.
Those are the V8 3.14 (node 0.10.x) functions and they don't exist under those names in V8 3.28 (node 0.12.0), but I don't think the issue has actually been fixed as of V8 3.28. I haven't tried io.js. I did just try a quick test of node 0.12.0 with Proxy and node-inspector and still see Illegal access exceptions.

Thanks for that info @metamatt
I guess our only hope is to wait for a V8 fix. However, even when that happens, we will need node-inspector to support io.js so we can get at the newest v8. Looks like that may come just in time (https://github.com/node-inspector/node-inspector/issues/523)!

I just ran into this problem... Is there any fix yet?

Also seeing this error with docker to docker debugging

same problem.
node version v6.2.0

This happens on Chrome browser's debugger! with Proxy Object.

Not sure if webstorm uses node-inspector, but this is resolved in node 7.7+


event.bind is not a function.

hi there, i have just got your bot and i get this error in the console:
ERROR Unhandled rejection: TypeError: event.bind is not a function
can anyone help.

nevermind, fixed

How did you fix it?

By following the guide properly and ensuring that all event files had a proper module.exports block .

Oh yeah, that was the mistake. Ty.


Set Eclipse variable for user home on all OSs and use that for WPILIB…

… location
Closes #15 (%HOME% set on Windows breaks C++ project builds)

Someone should definitely test this on a non-Windows system before merge


Enable 'serve' tag-set in create_module_spec function call

It would be useful to export modules in a format that can be consumed by TensorFlow Serving. Servables would increase code reuse and further enable distributed workloads.
Example Usage (note passing "serve" instead of "train"):
hub.create_module_spec(
    module_fn,
    tags_and_args=[({"serve"}, {"is_training":False})],
    drop_collections=None)

Any workarounds for this currently? I want to serve a hub module using Tensorflow Serving.

I have not tested this, but I suspect you could load the module into a empty graph and then export the graph with savedmodelbuilder. I suspect that you could automate that process.

For now one has to do it manually, example:
import tensorflow as tf
import tensorflow_hub as hub

with tf.Graph().as_default():
  module = hub.Module("http://tfhub.dev/google/universal-sentence-encoder/2")
  text = tf.placeholder(tf.string, [None])
  embedding = module(text)

  init_op = tf.group([tf.global_variables_initializer(), tf.tables_initializer()])
  with tf.Session() as session:
    session.run(init_op)
    tf.saved_model.simple_save(
        session,
        "/tmp/serving_saved_model",
        inputs = {"text": text},
        outputs = {"embedding": embedding}
    )

Each modules differs slightly from others in input/output names, additionally each serving use case might have different requirements (e.g. raw features in vs serialized tf.Example protos). By having users creating the graph they want to serve (e.g. as done above) seems more flexible than require users to guess what Servo config to use and/or modify the client side each time they change the module being served.

Closing as this is now obsolete.
In TF-2 users should create reusable saved models with tf.saved_model.save().

@andresusanopinto quick question, does it mean the pre-trained model on tensorflow hub won't have the serving_default signature by default and the user need to re-export it? Thanks

A TF2 SavedModel can both have signatures for deployment to TensorFlow Serving and tf.functions for reuse in a Python TensorFlow program.
See https://www.tensorflow.org/hub/tf2_saved_model#advanced_topic_what_to_expect_from_the_savedmodel_after_loading and https://www.tensorflow.org/guide/saved_model

@arnoegw thanks for the pointer. I noticed some inconsistency of some TF2 models that some have serving_default signature while some does not. For example:

Only default signature:

https://tfhub.dev/google/faster_rcnn/openimages_v4/inception_resnet_v2/1
https://tfhub.dev/google/openimages_v4/ssd/mobilenet_v2/1

No signature: "https://tfhub.dev/google/imagenet/mobilenet_v2_100_224/feature_vector/4"
Only serving_default https://tfhub.dev/tensorflow/resnet_50/classification/1

My understanding is serving_default is not present, the model cannot be served as-is in TF serving. Let me know if I should open a new issue to track this.

@chenliu0831, there's nothing wrong with those examples:

The following are hub.Module assets for TF1 that use SavedModel's SignatureDef to define the signatures that hub.Module can invoke. They cannot be served as-is, because the hub.Module format and SavedModels for serving had some small but crucial differences (which were removed for TF2).
https://tfhub.dev/google/faster_rcnn/openimages_v4/inception_resnet_v2/1
https://tfhub.dev/google/openimages_v4/ssd/mobilenet_v2/1
The following (and many more with similar names) is a TF2 SavedModel meant only for reuse in a Python TensorFlow program (put a to-be-learned layer on top of the features), so it has a @tf.function __call__ but not SignatureDefs:
https://tfhub.dev/google/imagenet/mobilenet_v2_100_224/feature_vector/4
The following is a TF2 SavedModel meant both for reuse and stand-alone serving, so it has both a @tf.function __call__ and a serving signature:
https://tfhub.dev/tensorflow/resnet_50/classification/1

@arnoegw Ah thanks for clarifying. I give a bad example for the second case since that's a feature vector.
I spot check another few on TF hub with the TF2 filter and with classification variants and looks like their signature map is empty as well:

https://tfhub.dev/google/imagenet/mobilenet_v2_100_224/classification/4
https://tfhub.dev/google/imagenet/nasnet_large/classification/4
https://tfhub.dev/google/imagenet/mobilenet_v2_035_160/classification/4

In the detail page of above models I think they all show up as "TF2.0 Saved Model" format.


Ingress removal

Ingress will no longer be supported and will be removed
For more information ,please refer to internal issue : 3574

PR has been merged to develop branch : https://github.com/IBM/kubernetes-driver/pull/87


Fall back to "tsc" if glob doesn't find a tsc binary.

This is a simple fix for Meteor + Windows.

thank you


Check why some entries are flattened

For example :
HNBKFXBALXUFRY-YOGCQVOKSA-N,Cc1ccc2n1[C@@H]1[C@@H]3O[C@]([C@H](C)O)([C@H](O)C2)[C@H]1c1ccc(C)n1[C@@H]3C
and
HNBKFXBALXUFRY-NCVFPUPRSA-N,Cc1ccc2n1[C@@H]1[C@@H]3O[C@]([C@H](C)O)([C@@H](O)C2)[C@H]1c1ccc(C)n1[C@@H]3C
Result, after pickaxe process in :
{
  "_id": "Cc6d77db1bdda75b9feca389ea4939835a09c1c0f",
  "ID": "HNBKFXBALXUFRY-NCVFPUPRSA-N",
  "SMILES": "Cc1ccc2n1C(C)C1OC3(C(C)O)C(O)Cc4ccc(C)n4C1C23",
  "InChI_key": "HNBKFXBALXUFRY-UHFFFAOYSA-N",
  "Type": "Starting Compound",
  "Generation": 0,
  "Expand": true,
  "Reactant_in": [
    "Cc6d77db1bdda75b9feca389ea4939835a09c1c0f_0"
  ],
  "Product_of": []
}

Since we add metadata according to InChI_key and not ID these type of entries to not get populated

Should be this.
However if I remember correctly we commented this line see https://github.com/commons-research/MINE-Database/blob/005678689d531931823166d10fb8ca67e4fcfa7b/mine_database/pickaxe_commons.py#L307
Could you add the previous compounds to the lotus_10 test set and try to understand what's the issue ?


feat(#216): allow locations to be specified as positional args in insert

Fixes #216

Current dependencies on/for this PR:

main

PR #222   👈

This stack of pull requests is managed by Graphite.

Merge activity

Jan 31, 11:24 AM: @MartinBernstorff merged this pull request with Graphite.


Update http4s-circe, http4s-dsl, ... to 0.21.26

Updates

org.http4s:http4s-circe
org.http4s:http4s-dsl
org.http4s:http4s-ember-client
org.http4s:http4s-ember-server

from 0.21.20 to 0.21.26.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.http4s" } ]

labels: library-update, semver-patch

Superseded by #34.


Mod Load Order Question...

Not a Bug per say!
But when using the Fixed Launcher, Community Patch always slides above the TW modules and I move it below them. It should load after them? Correct? To overwrite their broke stuff with fixes?

Community Patch's intended sort order is as high (early) as possible, not later.
It doesn't overwrite anything by replacing files, and other mods should be able to overwrite/replace it's files.


Print the path of installed versions when running the installed command

The current behaviour prints the versions that are installed:
$ xcodes installed
12.2
12.3 (Selected)
12.4 Release Candidate (12D4e)

It may be useful to also print the path they're installed at, perhaps like:
$ xcodes installed
12.2    /Applications/Xcode-12.2.0.app
12.3 (Selected)    /Applications/Xcode-12.2.0.app
12.4 Release Candidate (12D4e)    /Applications/Xcode-12.2.0.app

There should be a tab character in between the current output and the path so it's easier to split the lines into their components.

I can’t remember why I wanted this. Maybe to grep and pipe to xcode-select? There’s a select command in xcodes now though.

I've added some more information to the issue and will re-open it, as it might be useful for some use cases like getting a path to set the DEVELOPER_DIR environment variable in scripts.

I've added some more information to the issue and will re-open it, as it might be useful for some use cases like getting a path to set the DEVELOPER_DIR environment variable in scripts.


Interpolation of meteo data using Scipy RBFInterpolator

This pull request allows for data to be interpolated using the Scipy RBFInterpolator.
The function currently implemented in io_knmi and is used by EvaporationObs.from_xy(method='interpolation') or the ObsCollection.from_knmi(meteo_var=['EV24'], method='interpolation')
The method is significantly slower compared for nearest station if a time series for only one location is collected. For multiple locations (of stations) the difference in computation time becomes much smaller.
For a comparison see notebook examples/04_interpolation.ipynb

So I removed all the IPython magic commands from the KNMI notebook but the test still fails here. When I run pytest locally there is no issue at all so I don't know what is going wrong..
Also the execution time of the Notebook is brought down by commenting out the last two cells.

Two other things are going wrong at the moment.

If you give a startdate='1980-01-01', enddate='2022-07-21' you get a timeseries from 1987-03-26 till 2022-07-21 no matter what. This also goes wrong in the notebook. If the startdate and enddate are both set before 1987-03-26 or after 2022-07-21 this error does not occur.

Some code was deleted so the ObsCollection (obtaining timeseries for multiply locations) is now very very slow. This happens because now when a timeseries is interpolated, an interpolator object has to be created for every timeseries and every timestep. Previously when multiple timeseries needed to be extracted this was done simultaneously for all locations on every timestep.

Something I'll look into but this has to be fixed before merging.

The two problems with the startdate/enddate and multiple locations are resolved right now.
Only the tests still fail.

The test are passing now. There were some problems with an outdated pinned version of pandas and a not working newest version of netCDF4. The pull request is ready to merge!


Honor git submodules when using g:grepper.dir = 'repo'

If I perform Grepper -tool git while being in a submodule, the result list is always empty.
That is because submodules do not create a .git directory; on the other hand, s:compute_working_directory looks exactly for a .git directory to determine where to launch the git grep command - and it always finds the topmost, parent repository.
Unfortunately, git grep does not search through submodules by default - you have to use --recurse-submodules flag. Currently, if I launch grepper while browsing the submodule files, they are completely omitted from search.
I propose two things:

Let grepper.git.grepprg use --recurse-submodules by default.

Even better, I suggest that s.compute_working_directory does something like this:
      " This hack actually works for me

      if dir == 'repo'
        return system("pushd " . expand('%:p:h') . "; git rev-parse --show-toplevel")

Which will additionally always limit the search to the current submodule - we could introduce another option for this, named "git_repo". Note that, if we just provide grepprg with --recurse-submodules instead, we always search through the entire repository along with other, unrelated submodules.

Last but not least, thank you for an amazing plugin!

Okay, it now tries git -C <current dir> rev-parse --show-toplevel before looking though g:grepper.repo.
Thanks for the suggestion!

Awesome! Works just as intended 👍


Error running timer ‘good-scroll--render’: (wrong-type-argument number-or-marker-p nil)

Describe the bug
I get an error when I press shift-tab to close the org-mode document
Error running timer ‘good-scroll--render’: (wrong-type-argument number-or-marker-p nil) [208 times]
As you can see, the error was executed 208 times
Version and environment

OS: macOS
Emacs version 28.1.4

When more pictures are inserted in the org document, it will reproduce when scrolling or inserting, like this

In addition, sometimes, when I use C-x C-f to open the directory, it cannot be opened, and the following problem is reported:
good-scroll--point-at-top-p: Args out of range: 12101, 1, 3618
But no specific pattern has been found.


Implement fargate profile

Description of your changes

Fixes #407
I have:

[x] Read and followed Crossplane's contribution process.
[x] Run make reviewable test to ensure this PR is ready for review.

How has this code been tested

Manual testing (example/eks)
Unit testing for the EKS FargateProfile controller and client

Oh and it'd be great if you can squeeze some of the commit history to have fewer commits.


[$250] BUG: Background color does not cover status bar area

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Open NewDot and login
Go to ay chat
Long press on any comment to open the docked menu

Expected Result:
Background color should cover status bar on top
Actual Result:
Does not cover the area
Workaround:
Visual
Platform:

Where is this issue occurring?

Mobile Web - Safari

Version Number: 1.2.19-2
Reproducible in staging?: y
Reproducible in production?: y
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos:

Expensify/Expensify Issue URL:
Issue reported by: @roryabraham
Slack conversation: https://expensify.slack.com/archives/C01GTK53T8Q/p1666776603827639
View all open jobs on GitHub

I was able to reproduce. Waiting on the Slack discussion to decide whether to label external or no.

I'm not really sure that this is a bug or something we want to fix. When that menu is open, there is an opacity overlay over the rest of the app, so it's not really a "background color" that we would want to extend to the status bar area. I'm going to close this, but also tagging @Expensify/design just for confirmation on my theory.

Yeah, I don't think we can do much about that. Ideally the opacity/BG overlay would stretch over the status bar area but I don't think it's possible.

Ideally the opacity/BG overlay would stretch over the status bar area but I don't think it's possible.

I agree that this would be ideal, and I think it probably is possible? Reopening this

Or, maybe it's only possible on native but not mobile web. Let's see what proposals we get?

Not overdue, waiting for proposals

Proposal
In Safari, it is possible to modify the status bar's color using the theme-color value for the name attribute of the <meta> tag. In particular, I propose a patch along the lines of:
diff --git a/src/pages/home/report/ContextMenu/PopoverReportActionContextMenu.js b/src/pages/home/report/ContextMenu/PopoverReportActionContextMenu.js
index 49e1a2f97..b1a92168c 100644
--- a/src/pages/home/report/ContextMenu/PopoverReportActionContextMenu.js
+++ b/src/pages/home/report/ContextMenu/PopoverReportActionContextMenu.js
@@ -1,4 +1,5 @@
 import React from 'react';
+import { Helmet } from "react-helmet";
 import {
     Dimensions,
 } from 'react-native';
@@ -288,6 +289,11 @@ class PopoverReportActionContextMenu extends React.Component {
     render() {
         return (
             <>
+                {this.state.isPopoverVisible ? (
+                    <Helmet>
+                        <meta name="theme-color" content="#E6E3E3" />
+                    </Helmet>
+                ) : null}
                 <PopoverWithMeasuredContent
                     isVisible={this.state.isPopoverVisible}
                     onClose={this.hideContextMenu}

This change gives the following results:

https://user-images.githubusercontent.com/13868083/202913335-4c359c7f-b2a2-4086-af9f-d816630f56df.MP4
Please note that the final implementation would dynamically compute the required theme-color to account for different themes. One caveat is that the color change of the status bar is not animated, the backdrop of the popover is, though. This causes a slight visual misalignment that is also visible in the above video. Finally, the solution currently relies on react-helmet to dynamically inject the <meta> tag.

I don't think that animation is causing too much of an issue. Rather than using react-helmet, can you please change your proposal to not use it?

@tgolen I will make sure to accomplish the same as proposed without using react-helmet and get back to you as soon as possible.

Proposal (Update)
The update proposal now fixes the status bar color as follows:
From ef6a201350ace58dd1d9d5f9530cbefadff45d12 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20Fu=CC=88rst?=<EMAIL_ADDRESS>Date: Tue, 22 Nov 2022 01:25:14 +0100
Subject: [PATCH] Adapt status bar color to modal backdrop

---
 .../PopoverReportActionContextMenu.js         |  6 ++++
 src/styles/StyleUtils.js                      | 35 +++++++++++++++++++
 web/index.html                                |  1 +
 3 files changed, 42 insertions(+)

diff --git a/src/pages/home/report/ContextMenu/PopoverReportActionContextMenu.js b/src/pages/home/report/ContextMenu/PopoverReportActionContextMenu.js
index 2ac4cf1042..a9ef374a9d 100644
--- a/src/pages/home/report/ContextMenu/PopoverReportActionContextMenu.js
+++ b/src/pages/home/report/ContextMenu/PopoverReportActionContextMenu.js
@@ -9,6 +9,7 @@ import withLocalize, {withLocalizePropTypes} from '../../../../components/withLo
 import PopoverWithMeasuredContent from '../../../../components/PopoverWithMeasuredContent';
 import BaseReportActionContextMenu from './BaseReportActionContextMenu';
 import ConfirmModal from '../../../../components/ConfirmModal';
+import * as StyleUtils from '../../../../styles/StyleUtils';

 const propTypes = {
     /** Flag to check if the chat participant is Chronos */
@@ -148,6 +149,8 @@ class PopoverReportActionContextMenu extends React.Component {
             this.onPopoverHide = onHide;
         };
         this.getContextMenuMeasuredLocation().then(({x, y}) => {
+            document.querySelector('meta[name=theme-color]').content = StyleUtils.getThemeColor();
+
             this.setState({
                 cursorRelativePosition: {
                     horizontal: nativeEvent.pageX - x,
@@ -213,6 +216,9 @@ class PopoverReportActionContextMenu extends React.Component {
         if (_.isFunction(onHideActionCallback)) {
             this.onPopoverHideActionCallback = onHideActionCallback;
         }
+
+        document.querySelector('meta[name=theme-color]').content = '';
+
         this.setState({
             reportID: '0',
             reportAction: {},
diff --git a/src/styles/StyleUtils.js b/src/styles/StyleUtils.js
index d05acdfa8f..075e8124da 100644
--- a/src/styles/StyleUtils.js
+++ b/src/styles/StyleUtils.js
@@ -444,6 +444,40 @@ function getPaymentMethodMenuWidth(isSmallScreenWidth) {
     return {width: !isSmallScreenWidth ? variables.sideBarWidth - (margin * 2) : undefined};
 }

+/**
+ * Approximates the theme color for a modal based on the app's background color,
+ * the modal's backdrop, and the backdrop's opacity.
+ *
+ * @returns {String} The theme color as an RGB value.
+ */
+function getThemeColor() {
+    const hexadecimalToRGB = hexadecimal => _.map(hexadecimal.replace(/^#?([a-f\d])([a-f\d])([a-f\d])$/i,
+        (m, r, g, b) => `#${r}${r}${g}${g}${b}${b}`)
+        .substring(1).match(/.{2}/g), x => parseInt(x, 16));
+
+    const [backgroundR, backgroundG, backgroundB] = hexadecimalToRGB(themeColors.appBG);
+    const [backdropR, backdropG, backdropB] = hexadecimalToRGB(themeColors.modalBackdrop);
+
+    // This value is hardcoded in `BaseModal.js`.
+    const backdropOpacity = 0.5;
+
+    const normalizedBackdropR = backdropR / 255;
+    const normalizedBackdropG = backdropG / 255;
+    const normalizedBackdropB = backdropB / 255;
+    const normalizedBackgroundR = backgroundR / 255;
+    const normalizedBackgroundG = backgroundG / 255;
+    const normalizedBackgroundB = backgroundB / 255;
+
+    const normalizedApproximatedR = ((1 - backdropOpacity) * normalizedBackdropR) + (backdropOpacity * normalizedBackgroundR);
+    const normalizedApproximatedG = ((1 - backdropOpacity) * normalizedBackdropG) + (backdropOpacity * normalizedBackgroundG);
+    const normalizedApproximatedB = ((1 - backdropOpacity) * normalizedBackdropB) + (backdropOpacity * normalizedBackgroundB);
+    const approximatedR = Math.floor(normalizedApproximatedR * 255);
+    const approximatedG = Math.floor(normalizedApproximatedG * 255);
+    const approximatedB = Math.floor(normalizedApproximatedB * 255);
+
+    return `rgb(${approximatedR}, ${approximatedG}, ${approximatedB})`;
+}
+
 /**
  * Parse styleParam and return Styles array
  * @param {Object|Object[]} styleParam
@@ -561,6 +595,7 @@ export {
     getMiniReportActionContextMenuWrapperStyle,
     getKeyboardShortcutsModalWidth,
     getPaymentMethodMenuWidth,
+    getThemeColor,
     parseStyleAsArray,
     combineStyles,
     getPaddingLeft,
diff --git a/web/index.html b/web/index.html
index ca117f328d..638a139d4a 100644
--- a/web/index.html
+++ b/web/index.html
@@ -4,6 +4,7 @@
     <meta charset="utf-8" />
     <title>New Expensify</title>
     <meta name="description" content="Corporate cards, reimbursements, receipt scanning, invoicing, and bill pay. One app, all free.">
+    <meta name="theme-color" content="">
     <meta property="twitter:card" content="summary">
     <meta property="twitter:site" content="@expensify">
     <meta property="og:type" content="website">
--
2.23.0

The only remaining minor issue is that the backdrop opacity of the modal is hardcoded since it is hardcoded in BaseModal.js as well.

Proposal looks good to me, i didn't know this was possible, nice!
A couple of things

I'm not sure this is needed for Android, i couldn't find models where status bar color could be changed. But there might be, so I'm not sure if we need to apply this only for Safari
Would it be possible to have a more generalized approach? I suppose there are other cases where this bug might appear, opening any other fullscreen modal

Proposal (Update)
@tgolen @eVoloshchak the updated proposal now fixes the status bar color in a more general way:
From 5c2fda45dabe5e33f46b9ec7c32ae5afa34cc930 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20Fu=CC=88rst?=<EMAIL_ADDRESS>Date: Sun, 27 Nov 2022 14:05:38 +0100
Subject: [PATCH] Adapt status bar color to modal backdrop

---
 src/components/Modal/BaseModal.js | 10 ++++++++++
 src/styles/StyleUtils.js          | 33 +++++++++++++++++++++++++++++++
 src/styles/variables.js           |  1 +
 web/index.html                    |  1 +
 4 files changed, 45 insertions(+)

diff --git a/src/components/Modal/BaseModal.js b/src/components/Modal/BaseModal.js
index 91b1867a6f..0de83c7faf 100644
--- a/src/components/Modal/BaseModal.js
+++ b/src/components/Modal/BaseModal.js
@@ -9,6 +9,8 @@ import themeColors from '../../styles/themes/default';
 import {propTypes as modalPropTypes, defaultProps as modalDefaultProps} from './modalPropTypes';
 import * as Modal from '../../libs/actions/Modal';
 import getModalStyles from '../../styles/getModalStyles';
+import getPlatform from '../../libs/getPlatform';
+import CONST from '../../CONST';

 const propTypes = {
     ...modalPropTypes,
@@ -53,6 +55,10 @@ class BaseModal extends PureComponent {
         if (callHideCallback) {
             this.props.onModalHide();
         }
+
+        if (this.props.fullscreen && getPlatform() === CONST.PLATFORM.WEB) {
+            document.querySelector('meta[name=theme-color]').content = '';
+        }
     }

     render() {
@@ -94,6 +100,10 @@ class BaseModal extends PureComponent {
                         Modal.setModalVisibility(true);
                     }
                     this.props.onModalShow();
+
+                    if (this.props.fullscreen && getPlatform() === CONST.PLATFORM.WEB) {
+                        document.querySelector('meta[name=theme-color]').content = StyleUtils.getThemeColor();
+                    }
                 }}
                 propagateSwipe={this.props.propagateSwipe}
                 onModalHide={this.hideModal}
diff --git a/src/styles/StyleUtils.js b/src/styles/StyleUtils.js
index d05acdfa8f..0dd0405a80 100644
--- a/src/styles/StyleUtils.js
+++ b/src/styles/StyleUtils.js
@@ -444,6 +444,38 @@ function getPaymentMethodMenuWidth(isSmallScreenWidth) {
     return {width: !isSmallScreenWidth ? variables.sideBarWidth - (margin * 2) : undefined};
 }

+/**
+ * Approximates the theme color for a modal based on the app's background color,
+ * the modal's backdrop, and the backdrop's opacity.
+ *
+ * @returns {String} The theme color as an RGB value.
+ */
+function getThemeColor() {
+    const backdropOpacity = variables.modalFullscreenBackdropOpacity;
+    const hexadecimalToRGB = hexadecimal => _.map(hexadecimal.replace(/^#?([a-f\d])([a-f\d])([a-f\d])$/i,
+        (m, r, g, b) => `#${r}${r}${g}${g}${b}${b}`)
+        .substring(1).match(/.{2}/g), x => parseInt(x, 16));
+
+    const [backgroundR, backgroundG, backgroundB] = hexadecimalToRGB(themeColors.appBG);
+    const [backdropR, backdropG, backdropB] = hexadecimalToRGB(themeColors.modalBackdrop);
+
+    const normalizedBackdropR = backdropR / 255;
+    const normalizedBackdropG = backdropG / 255;
+    const normalizedBackdropB = backdropB / 255;
+    const normalizedBackgroundR = backgroundR / 255;
+    const normalizedBackgroundG = backgroundG / 255;
+    const normalizedBackgroundB = backgroundB / 255;
+
+    const normalizedApproximatedR = ((1 - backdropOpacity) * normalizedBackdropR) + (backdropOpacity * normalizedBackgroundR);
+    const normalizedApproximatedG = ((1 - backdropOpacity) * normalizedBackdropG) + (backdropOpacity * normalizedBackgroundG);
+    const normalizedApproximatedB = ((1 - backdropOpacity) * normalizedBackdropB) + (backdropOpacity * normalizedBackgroundB);
+    const approximatedR = Math.floor(normalizedApproximatedR * 255);
+    const approximatedG = Math.floor(normalizedApproximatedG * 255);
+    const approximatedB = Math.floor(normalizedApproximatedB * 255);
+
+    return `rgb(${approximatedR}, ${approximatedG}, ${approximatedB})`;
+}
+
 /**
  * Parse styleParam and return Styles array
  * @param {Object|Object[]} styleParam
@@ -561,6 +593,7 @@ export {
     getMiniReportActionContextMenuWrapperStyle,
     getKeyboardShortcutsModalWidth,
     getPaymentMethodMenuWidth,
+    getThemeColor,
     parseStyleAsArray,
     combineStyles,
     getPaddingLeft,
diff --git a/src/styles/variables.js b/src/styles/variables.js
index 409c9be576..461995d4a0 100644
--- a/src/styles/variables.js
+++ b/src/styles/variables.js
@@ -42,6 +42,7 @@ export default {
     emojiLineHeight: 28,
     iouAmountTextSize: 40,
     mobileResponsiveWidthBreakpoint: 800,
+    modalFullscreenBackdropOpacity: 0.5,
     tabletResponsiveWidthBreakpoint: 1024,
     safeInsertPercentage: 0.7,
     sideBarWidth: 375,
diff --git a/web/index.html b/web/index.html
index ca117f328d..638a139d4a 100644
--- a/web/index.html
+++ b/web/index.html
@@ -4,6 +4,7 @@
     <meta charset="utf-8" />
     <title>New Expensify</title>
     <meta name="description" content="Corporate cards, reimbursements, receipt scanning, invoicing, and bill pay. One app, all free.">
+    <meta name="theme-color" content="">
     <meta property="twitter:card" content="summary">
     <meta property="twitter:site" content="@expensify">
     <meta property="og:type" content="website">
--
2.23.0

@dnlfrst, the more general approach is awesome!

Judging from the official documentation of Google Chrome (and CIU), the status bar color should also work on Android as long as the native dark mode is not enabled.

You are right, just confirmed it working on Chrome, my mistake was using the dark mode
I'm happy with the proposal functionality-wise, there's only a couple of stylistic issues that need to be addressed, @tgolen listed them all here:

move opacity to the style/variables.js file,

move the document methods to a web-only platform file

see if getThemeColor could be simplified at all

Could you update your proposal with these? If you need help or more context - just tag me, I'll try to be more readily available

Looks like this one is moving forward! I'm going to remove and re-add the external label to create an Upwork job.

Proposal (Update)

move opacity to the style/variables.js file

This is already part of the above proposal.

see if getThemeColor could be simplified at all

This is addressed with the below proposal.

move the document methods to a web-only platform file

Intuitively, I do not think that it makes sense to move the document.* calls to CustomStatusBar. Rather, I have now moved the web-specific calls to Modal. @eVoloshchak let me know what you think about this.
From 658586a0ccba9750bc0e8cd75e01bc00f633e737 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20Fu=CC=88rst?=<EMAIL_ADDRESS>Date: Tue, 29 Nov 2022 11:48:16 +0100
Subject: [PATCH] Adapt status bar color to modal backdrop

---
 src/components/Modal/index.web.js | 29 ++++++++++
 src/styles/StyleUtils.js          | 94 +++++++++++++++++++++++++++++--
 src/styles/variables.js           |  1 +
 web/index.html                    |  1 +
 4 files changed, 120 insertions(+), 5 deletions(-)
 create mode 100644 src/components/Modal/index.web.js

diff --git a/src/components/Modal/index.web.js b/src/components/Modal/index.web.js
new file mode 100644
index<PHONE_NUMBER>..86a1610095
--- /dev/null
+++ b/src/components/Modal/index.web.js
@@ -0,0 +1,29 @@
+import React from 'react';
+import withWindowDimensions from '../withWindowDimensions';
+import BaseModal from './BaseModal';
+import {propTypes, defaultProps} from './modalPropTypes';
+import * as StyleUtils from '../../styles/StyleUtils';
+
+const Modal = props => (
+    <BaseModal
+            // eslint-disable-next-line react/jsx-props-no-spreading
+        {...props}
+        onModalHide={() => {
+            if (!props.fullscreen) { return; }
+
+            document.querySelector('meta[name=theme-color]').content = '';
+        }}
+        onModalShow={() => {
+            if (!props.fullscreen) { return; }
+
+            document.querySelector('meta[name=theme-color]').content = StyleUtils.getThemeColor();
+        }}
+    >
+        {props.children}
+    </BaseModal>
+);
+
+Modal.propTypes = propTypes;
+Modal.defaultProps = defaultProps;
+Modal.displayName = 'Modal';
+export default withWindowDimensions(Modal);
diff --git a/src/styles/StyleUtils.js b/src/styles/StyleUtils.js
index d05acdfa8f..fd7c89b813 100644
--- a/src/styles/StyleUtils.js
+++ b/src/styles/StyleUtils.js
@@ -202,6 +202,22 @@ function getBackgroundColorStyle(backgroundColor) {
     };
 }

+/**
+ * Converts a color in hexadecimal notation into RGB notation.
+ *
+ * @param {String} hexadecimal A color in hexadecimal notation.
+ * @returns {Array} `null` if the input color is not in hexadecimal notation. Otherwise, the input color as RGB value.
+ */
+function hexadecimalToRGB(hexadecimal) {
+    const components = /^#?([a-f\d]{2})([a-f\d]{2})([a-f\d]{2})$/i.exec(hexadecimal);
+
+    if (components !== null) {
+        return _.map(components.slice(1), component => parseInt(component, 16));
+    }
+
+    return null;
+}
+
 /**
  * Returns a background color with opacity style
  *
@@ -210,13 +226,10 @@ function getBackgroundColorStyle(backgroundColor) {
  * @returns {Object}
  */
 function getBackgroundColorWithOpacityStyle(backgroundColor, opacity) {
-    const result = /^#?([a-f\d]{2})([a-f\d]{2})([a-f\d]{2})$/i.exec(backgroundColor);
+    const result = hexadecimalToRGB(backgroundColor);
     if (result !== null) {
-        const r = parseInt(result[1], 16);
-        const g = parseInt(result[2], 16);
-        const b = parseInt(result[3], 16);
         return {
-            backgroundColor: `rgba(${r}, ${g}, ${b}, ${opacity})`,
+            backgroundColor: `rgba(${result[1]}, ${result[1]}, ${result[1]}, ${opacity})`,
         };
     }
     return {};
@@ -444,6 +457,76 @@ function getPaymentMethodMenuWidth(isSmallScreenWidth) {
     return {width: !isSmallScreenWidth ? variables.sideBarWidth - (margin * 2) : undefined};
 }

+/**
+ * Approximates a color in RGBA notation with an equivalent color in RGB notation.
+ *
+ * @param {Array} foregroundRGB The three components of the foreground color in RGB notation.
+ * @param {Array} backgroundRGB The three components of the background color in RGB notation.
+ * @param {number} opacity The desired opacity of the foreground color.
+ * @returns {Array} A color in RGB notation that approximates the foreground color with the given opacity on top of the background color.
+ */
+function approximateRGBAWithRGB(foregroundRGB, backgroundRGB, opacity) {
+    const [foregroundR, foregroundG, foregroundB] = foregroundRGB;
+    const [backgroundR, backgroundG, backgroundB] = backgroundRGB;
+
+    return [
+        ((1 - opacity) * backgroundR) + (opacity * foregroundR),
+        ((1 - opacity) * backgroundG) + (opacity * foregroundG),
+        ((1 - opacity) * backgroundB) + (opacity * foregroundB),
+    ];
+}
+
+/**
+ * Denormalizes the three components of a color in RGB notation.
+ *
+ * @param {number} r The first normalized component of a color in RGB notation.
+ * @param {number} g The second normalized component of a color in RGB notation.
+ * @param {number} b The third normalized component of a color in RGB notation.
+ * @returns {Array} An array with the three denormalized components of a color in RGB notation.
+ */
+function denormalizeRGB(r, g, b) {
+    return [Math.floor(r * 255), Math.floor(g * 255), Math.floor(b * 255)];
+}
+
+/**
+ * Normalizes the three components of a color in RGB notation.
+ *
+ * @param {number} r The first denormalized component of a color in RGB notation.
+ * @param {number} g The second denormalized component of a color in RGB notation.
+ * @param {number} b The third denormalized component of a color in RGB notation.
+ * @returns {Array} An array with the three normalized components of a color in RGB notation.
+ */
+function normalizeRGB(r, g, b) {
+    return [r / 255, g / 255, b / 255];
+}
+
+/**
+ * Approximates the theme color for a modal based on the app's background color,
+ * the modal's backdrop, and the backdrop's opacity.
+ *
+ * @returns {String} The theme color as an RGB value.
+ */
+function getThemeColor() {
+    const backdropOpacity = variables.modalFullscreenBackdropOpacity;
+
+    const [backgroundR, backgroundG, backgroundB] = hexadecimalToRGB(themeColors.appBG);
+    const [backdropR, backdropG, backdropB] = hexadecimalToRGB(themeColors.modalBackdrop);
+    const normalizedBackdropRGB = normalizeRGB(backdropR, backdropG, backdropB);
+    const normalizedBackgroundRGB = normalizeRGB(
+        backgroundR,
+        backgroundG,
+        backgroundB,
+    );
+    const approximatedThemeRGBNormalized = approximateRGBAWithRGB(
+        normalizedBackdropRGB,
+        normalizedBackgroundRGB,
+        backdropOpacity,
+    );
+    const approximatedThemeRGB = denormalizeRGB(...approximatedThemeRGBNormalized);
+
+    return `rgb(${approximatedThemeRGB.join(', ')})`;
+}
+
 /**
  * Parse styleParam and return Styles array
  * @param {Object|Object[]} styleParam
@@ -561,6 +644,7 @@ export {
     getMiniReportActionContextMenuWrapperStyle,
     getKeyboardShortcutsModalWidth,
     getPaymentMethodMenuWidth,
+    getThemeColor,
     parseStyleAsArray,
     combineStyles,
     getPaddingLeft,
diff --git a/src/styles/variables.js b/src/styles/variables.js
index 409c9be576..461995d4a0 100644
--- a/src/styles/variables.js
+++ b/src/styles/variables.js
@@ -42,6 +42,7 @@ export default {
     emojiLineHeight: 28,
     iouAmountTextSize: 40,
     mobileResponsiveWidthBreakpoint: 800,
+    modalFullscreenBackdropOpacity: 0.5,
     tabletResponsiveWidthBreakpoint: 1024,
     safeInsertPercentage: 0.7,
     sideBarWidth: 375,
diff --git a/web/index.html b/web/index.html
index ca117f328d..638a139d4a 100644
--- a/web/index.html
+++ b/web/index.html
@@ -4,6 +4,7 @@
     <meta charset="utf-8" />
     <title>New Expensify</title>
     <meta name="description" content="Corporate cards, reimbursements, receipt scanning, invoicing, and bill pay. One app, all free.">
+    <meta name="theme-color" content="">
     <meta property="twitter:card" content="summary">
     <meta property="twitter:site" content="@expensify">
     <meta property="og:type" content="website">
--
2.23.0

@eVoloshchak Can we prioritize reviewing this proposal and help keep this moving forward? Thank you!

Nice! @sakluger you're currently assigned. Do you want to take this one, or should I assign myself to the issue?

Let's get a PR up for this one. 50% bonus is merged within 3 days!

@JmillsExpensify thanks for keeping this one moving forward, I can handle it from here!

Let's get a PR up for this one. 50% bonus is merged within 3 days!

A first draft of the PR is now underway at #13215.

Please make sure to add a comment explaining that this line is for changing the status bar color above document.querySelector('meta[name=theme-color]').content (and link this issue), since the code is not self-explanatory

I suppose we still need to call props.onModalHide() and props.onModalShow() in the corresponding BaseModal methods

The draft already addresses both issues.

🟢 on the proposal. I think we should also bump this issue up to $500 @JmillsExpensify.
There are a few things I see that I will address in the PR itself.

I received an offer on Upwork at $250, should I already accept that or does this bump need to apply before?

Looks like the linked PR was merged but has not been deployed yet. I'm going to switch this one to a weekly while we wait to make sure there are no issue after deploy.

This was merged 7 days after assignment, there is no bonus or penalty. I will pay out tomorrow as long as there are no regressions.
I started a conversation in Slack in #bug-zero around whether we need regression tests for general UI elements, I'll update here when I have more details from that thread.

No new regression test steps needed for this since Applause looks for cosmetic problems when running through the full set of tests.

OK, sure. I've checked them off. I actually don't think they apply because this wasn't really a "bug" as much as it was a "new feature". It was never something that existed before, so there was nothing that broke. It was just not something that was ever done in the original design of the app.

Ah, that makes sense! I wasn't sure if it was something we broke or hadn't considered before, didn't want to make any assumptions. Thanks for the checkmarks 👌


Can't change the type of text (normal, heading, etc)

And it only appears as undefined.

Hi @ceifa ,
The issue is resolved. It was caused by a  breaking change in newer version of lexical.


refactor(theme-common): mark package as ESM

Motivation
To allow Jest to understand imports from this package as ESM. #6520
Have you read the Contributing Guidelines on pull requests?
Yes

Fair, sounds like a future problem to worry about 😄


Capacity potentials

Check that capacity potentials take into account initial capacities correctly. Maybe start to use q_constrainedCapMultiUnit constraint and update it so that it takes into account existing capacity.

Changed some wind potentials, decreased solar potentials and distinguished between solar PV large and residential.
Added "new" units for battery investments (with limited maxUnitCount) as well as groups and constrainedCapMultiplier for proper storage investment.
Added "new" units for RES investments (Nofossil alternative)
Added "new" units for thermal unit investments (Base-roe alternative)
Added some maxUnitcount for nuclear based on announced national targets or educated guesses

Updated wind and solar potentials. Wind potentials now also include low CF areas and PV is separated into rooftop (residential) and ground (large). Initial capacities have now been subtracted from the potentials when calculating maxUnitCount.
Investments not allowed in hydro or fossil units.
If we include biofuel potentials, they should reflect the potential in the power production sector.
If other units should have investment limits (batteries, nuclear, etc.), then we need to update those.
Updated also the alternatives.

ToDo: PL/BT/BK offshore cf missing. Where to get?

Created VREunlimited alternative for unlimited wind and solar investments.

Update min wind/PV investments in Germany:
According to the German renewable energy act, we should have much higher capacities to be conform with a fully renewable electricity system. For 2040, they are defined as follows (see https://www.gesetze-im-internet.de/eeg_2014/__4.html):

PV: 400 GW by 2040 (compared to ~200 GW in the BB results)
Wind Onshore: 160 GW by 2040 (compared to ~110 GW in the BB results)
Wind Offshore: 70 GW by 2040 (compared to ~30 GW in the BB results)

German wind and solar capacity limits now updated as follows (GW)
~60-65% RES:
PV min 200, max 400
Onshore min 110, max 160
Offshore min 30.048, max 70
~85% RES:
same limits as above
~95% RES:
PV min 400, max 400
Onshore min 160, max 160
Offshore min 70, max 70

NL solar and wind capacity limits updated


WP3.7.5 Modify the sky-subtraction framework in the pipeline

Change Request C025 resulted in a change of title, focus and due date.
Modify the sky-subtraction framework in the pipeline by:
(a) exploring changes to the pipeline parameters
(b) developing bespoke algorithms for sky-subtraction, and optimizing them to work with the pipeline and for
speed.
The algorithms will take a holistic approach and aim to perform sky subtraction in a way that preserves LSB flux across all types of LSB structures

16/SEP/21

breaking this down into more manageable steps. Started   developing software for test and validation.  Many more similar  iterations with testing on smaller images before testing on HSC  data.
Unofficial milestone - the algorithm will be written up into a  paper co-authored by Robert Lupton and rest of DM team.  Hope for June'22 publication


Server crash after launching space station

Server crashed after launching my space station resulting in the loss of the said station heres the crash log
https://hastebin.com/jisudoqoxo.sql
was a server first launch so that might be the issue ill try a second time sometime tommarow
LibVulpes-1.11.2-0.2.3b-universal
AdvancedRocketry-1.11.2-1.2.0f
Thanks again in advance,
Dryxen

Looks like the world is somehow null and this should not be able to occur in vanilla forge.  Try to see if this occurs without sponge?

this one didnt crash on the initial send but gave the same server crash when i tried to send the player rocket to the station on the forge server i might also mention sending new stations on the sponge server have stopped at around y400 and never leave dim0
https://hastebin.com/yesesukowe.sql
if theres any other helpful info i could provide let me know xD sry for keeping you busy

What if you use forge 1.11.2-<IP_ADDRESS>8?

ill set it up real fast what dim its the station trying to build in btw? maybe a dim conflict on our end for the first error i didnt even think about that

By default the dim is -2, though the server normally crashes on load in the case of a dimension conflict, so I find that unlikely.  It appears to be the world object itself that's non-existant though, even though the dimension itself appears to exist.

nope fresh copy of 1.11.2-<IP_ADDRESS>8 let it gen all the configs outside of the planetdefs i added the ones over with the earth destination, still crashing on launching player to the newly formed station
https://hastebin.com/adulewabul.sql

That second crash still reports it's using forge 1.11.2-<IP_ADDRESS>8 rather than 1.11.2-<IP_ADDRESS>8, which even betwee those two the method sig may have changed.

hmm wasnt even paying attention to the version i was using before i just downloaded the version you requested xD which version are you wanting me to check it against ill download it in the morning and check if its crashing.

I think you may have just gotten a number switched somewhere, you used the correct number when replying to the issue, though it appears you just downloaded the wrong file.  But still version 1.11.2-<IP_ADDRESS>8

I was apparently to tired to read last night xD heres your 2228 crash log xD sry about that.
https://hastebin.com/ojukumogab.sql

I've been there XD.  Thanks.  I'll take a look into this now.

I've tried to replicate the issue in singleplayer, multiplayer, and 2 of the three versions of forge that you've had issues with, I simply cannot replicate the issue, the station builds fine every time I send a ship up.  I'm wondering if it's possible that your copy of the mod is corrupt.  If if you don't mind, can you try first deleting the advanced rocketry mod file on the server(esp if the server is a windows computer), then replacing it with a fresh copy from curse and then testing again?

sure ill do that now, also did the xml look ok since i needed to manually add earth?

Yeah, the XML looks ok.  Not to mention, this issue appears unrelated to planet configuration anyway.
This is a really bizarre issue...
Does it occur with these builds (Also, I'd suggest operating on a backup with these, unless you do not plan on going back to 1.2.0f)?
http://<IP_ADDRESS>:8080/job/LibVulpes/job/libVulpes/job/MC1.11/
http://<IP_ADDRESS>:8080/job/Advanced Rocketry/job/AdvancedRocketry/job/MC1.11/

its a testing environment entirely to testing this bug so it doesnt matter =P

even better

appears advanced rocketry didnt register as a mod this go around libvulpes did though

Did you download the sources jar by mistake for AR?

AdvancedRocketry-1.11.2-1.2.1-13.jar nope lemme try to redownload it as it may have been corrupted can unzip it either

aye first download was corrupted sry bout that ;P testing now

Alright, if check the MD5 of the file ( https://md5file.com/calculator, linked for convenience.  Any MD5 calculator should work): it should be f326d9ee7988d477344493052a80fb93

space hates me xD https://hastebin.com/yuveqiyiko.sql

I'm totally at a loss... would it be possible to upload the entire server directory, or at least forge related files, the mod/config folders so I can try to attach a debugger to it?

yea gimme a sec ill compress them and find a upload site

Thank you, I appreciate it

never used this upload site so no idea how long it will take to download =/ https://ufile.io/xz6j5

The free download is still pretty fast: 340KBps, so less than 5 minutes

Alright, the mystery deepens, so I've run the server from the zip you sent over and constructed and launched a spacestation without any problem (only change I made to the server was change the dir in mod_list.json to point to the equivilent location on my disk, but mercurius wasn't found in either of our setups according to the output logs).  I've also double checked that I'm using the same java build
So the issue does not appear to lie with either the mods or forge but rather something system specific as far as I can tell.
Also have you tested to see if this occurs in single player as well?

havent test lemme load it up rq

If that works then it's likely some kind of sided mishap.  Though i still don't know why it would be different... I'll look into some way of getting around that line of code entirely.

ya seems to work in sp i think i messed up the landing process though

Try this build on the server and see if it still crashes
AdvancedRocketry-1.11.2-1.2.1.jar.zip
Should run with the newest libvulpes from the jenkins.
Also what do you mean by you think you messed up the landing process?

oh that ive never been to space successfully so i saw the press space to decend and immeadiatly hopped out of the rocket to make sure i would stay there xD rocket disapeared ended up about 1k blocks away from the station with the rocket being on top of it

Oh, I'll have to make sure to mention not to do that when I get around to writing the guide for space travel.

that seemed to have fixed the issue i was able to land without a problem

Alright, good!  Thanks for your help!


Can I run this without Docker?

Great project.
Possible to run this without docker and just as a python script?
THX!

Hi @seanvree,
Indeed, using Docker is not mandatory. You can install Cabot as a PIP package. We have an ansible role to do this kind of installation − even if you cannot/don’t want to use Ansible, it will outline the steps you need for a production deploy of Cabot.

got it @JeanFred  - thanks!
i'm trying to run this on windows....so can I just clone the repo and run one of the .py files?

@JeanFred  - Hey, am I missing something when you say " it will outline the steps you need for a production deploy of Cabot."  I'm not seeing those instructions anywhere, where should I be looking
https://github.com/cabotapp/ansible-cabot

@seanvree Right, that was a bit unclean :). The tasks in the Ansible role do install Cabot, so one could roughly follow their intent manually to get there.
However they make sense on Linux, not on Windows (there is no Systemd scripts on Windows for example). We do not support deployment on Windows. If you really cannot use a Unix system, then Docker will probably be your best bet :)

Got it! Thanks!

Someone can help me please? I'm trying to run cabot, but I have no idea how to do it without Docker. I have an OpenVZ VPS, but Docker is not supported because of the 2.6 kernel. I installed cabot with pip install cabot then I tried to run # cabot but could not find it. Thank you in advance!

Apparently the author of cabot decided the only way to run his app will be through docker-compose. Even vagrant doc disappeared from the repo. This is sad. Not everyone wants to be part of the docker hype. I would probably waste less time implementing my own solution than trying to run this outside of docker.

There's an ansible playbook cabot here: https://github.com/cabotapp/ansible-cabot
Or you can install the dependencies manually with apt-get install gcc musl-dev libffi-dev libldap2-dev libsasl2-dev and then simply pip install cabot (with pip<10 atm)


Typo in blogpost URL

Q
A

Bug fix?
yes?

What's in this Pull Request?
There was a typo in blog post URL in readme.md
(re-targeting to DEV branch)
Guidance
There was a typo in blog-post URL in line 42

Hi @panjkov, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no Contribution License Agreement is required at this point. Real humans will now evaluate your PR.

TTYL, MSBOT;

thx Dragan, appreciated.


shadow_map.py error

Hi, i tried to run shadow_map.py with a trained checkpoint with the exact same suggested parameter set up in the readme(stage1 -> baking -> stage 2).
It is giving this error:
Traceback (most recent call last):
File "shadow_map.py", line 555, in
loop=args.loop,
File "/home/by12/.conda/envs/GS-IR/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "shadow_map.py", line 459, in launch
argmax_depth=argmax_depth,
TypeError: render() got an unexpected keyword argument 'argmax_depth'
I see that render() doesn't take in argmax_depth, is there a code mismatch here?

You should just comment this line:
# argmax_depth=argmax_depth,

It seems to have worked for me.


Cow: Watch co-yomi.com

Cow requests the watch of the watch_keyword co-yomi\.com. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

co-yomi\.com has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Self-rejected by Cow in Charcoal HQ. Reason: 'this is not spam as deemed by SU mods'


script doesn't work for Lua version < 5.2

I get this error when running mpv
[mpv_twitch_chat]
[mpv_twitch_chat] stack traceback:
[mpv_twitch_chat]       [C]: ?
[mpv_twitch_chat]       [C]: ?
[mpv_twitch_chat] Lua error: /home/maelstrom/.config/mpv/scripts/mpv-twitch-chat.lua:141: '=' expected near 'continue'

Distro details:
                                                                                maelstrom@Legion
    ⠀⠀⢀⣤⣴⣶⣶⣶⣦⣤⡀⠀⣀⣠⣤⣴⣶⣶⣶⣶⣶⣶⣶⣶⣤⣤⣀⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀   ----------------
    ⠀⣰⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣶⣤⡀⠀⠀⠀⠀⠀⠀⠀⠀   OS: Nobara Linux 37 (KDE Plasma) x86_64
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣷⣄⠀⠀⠀⠀⠀⠀   Host: 82B1 Lenovo Legion 5 15ARH05H
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣷⣄⠀⠀⠀⠀   Kernel: 6.1.14-201.fsync.fc37.x86_64
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣧⠀⠀⠀   Uptime: 22 mins
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠟⠋⠉⠁⠀⠀⠉⠉⠛⠿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣧⠀⠀   Packages: 2449 (rpm)
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠟⠁⠀⠀⠀⢀⣀⣀⡀⠀⠀⠀⠈⢻⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡇⠀   Shell: bash 5.2.15
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡏⠀⠀⠀⢠⣾⣿⣿⣿⣿⣷⡄⠀⠀⠀⠻⠿⢿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀   Resolution: 1920x1080
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠁⠀⠀⠀⣿⣿⣿⣿⣿⣿⣿⡇⠀⠀⠀⠀⠀⣀⣀⣬⣽⣿⣿⣿⣿⣿⣿⠀   DE: Plasma 5.27.2
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀⠀⠀⠀⠈⠻⢿⣿⣿⡿⠟⠁⠀⠀⠀⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀   WM: kwin
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀   Theme: [Plasma], Breeze [GTK2/3]
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣷⣤⣤⣄⣀⡀⠀⠀⠀⠀⠀⠀⠀⠀⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀   Icons: [Plasma], breeze-dark [GTK2/3]
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣷⣄⠀⠀⠀⠀⠀⠀⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀   Terminal: konsole
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣇⠀⠀⠀⠀⠀⢸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀   CPU: AMD Ryzen 7 4800H with Radeon Graphics (16) @ 2.900GHz
    ⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡿⠟⠛⠉⠉⠛⠛⢿⣿⣿⠀⠀⠀⠀⠀⠸⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡿⠀   GPU: NVIDIA GeForce RTX 2060 Mobile
    ⠀⠘⢿⣿⣿⣿⣿⣿⣿⣿⡿⠋⠀⠀⠀⠀⠀⠀⠀⠀⠈⢿⠀⠀⠀⠀⠀⠀⠙⢿⣿⣿⣿⣿⣿⣿⣿⠟⠁⠀   Memory: 3709MiB / 15862MiB
    ⠀⠀⠀⠈⠙⠛⠛⠛⠋⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠉⠛⠛⠛⠛⠉⠁

mpv version:
mpv 0.35.0 Copyright © 2000-2022 mpv/MPlayer/mplayer2 projects
 built on UNKNOWN
FFmpeg library versions:
   libavutil       57.28.100
   libavcodec      59.37.100
   libavformat     59.27.100
   libswscale      6.7.100
   libavfilter     8.44.100
   libswresample   4.7.100
FFmpeg version: 5.1.2

Isn't Lua embedded in mpv? It looks like mpv's Lua is guaranteed >= 5.2, so why do you put "< 5.2" in the title?
There is no "continue" around line 141 in the script. Maybe your installation went wrong?

Isn't Lua embedded in mpv? It looks like mpv's Lua is guaranteed >= 5.2, so why do you put "< 5.2" in the title?

I had a similar issue where i got this kind of error in another script, so I assumed it is the same one of this script. I maybe be totally wrong tho sorry.
https://github.com/vc-01/metadata-osd/issues/11

There is no "continue" around line 141 in the script. Maybe your installation went wrong?

oh sorry i didn't update to the latest version of this script. this is the new error log.
[mpv_twitch_chat]
[mpv_twitch_chat] stack traceback:
[mpv_twitch_chat]       [C]: ?
[mpv_twitch_chat]       [C]: ?
[mpv_twitch_chat] Lua error: /home/maelstrom/.config/mpv/scripts/mpv-twitch-chat.lua:205: unexpected symbol near ':'

The script seems to be working!!

I was wrong above that mpv still supports both Lua 5.1 and 5.2. It seems you built mpv yourself, and your Lua dependency is probably 5.1. Some people (including myself) use prebuilt versions such as shinchiro's which already embeds Lua 5.2.
Anyway, that "continue" label is obsolete so I removed it. You should be fine now.

Ah actually I didn't, I am using nobara-appstream's version of mpv, which is patched for that particular distro to reduce headaches for ppl, not so sure why they build it with 5.1 .
https://nobaraproject.org/ (see below)
TYSM for fixing it!! Really appreciate it! Have a pleasant day / night.

not so sure why they build it with 5.1

Probably because it's the minimum requirement from mpv, and some project tend to be on the more conservative side. If you use other scripts that requires 5.2 features, you probably should start talking to Nobara folks. Otherwise, enjoy.


Maybe change TextInputValidationFunc signature?

Right now we have this signature for TextInputValidationFunc:
func(newInputText string) (bool, *string)

Is there a reason to avoid a more idiomatic form like the following?
func(newInputText string) (string, bool)

Looking at the code, I think this could make sense:
result, replacement := t.validationFunc(t.inputText)
if !result {
	if replacement != nil {
		t.inputText = *replacement
	} else {
		t.inputText = t.lastInputText
	}
}

// =>

replacement, ok := t.validationFunc(t.inputText)
if !ok {
	if replacement != t.inputText {
		t.inputText = replacement
	} else {
		t.inputText = t.lastInputText
	}
}

However, it also requires a convention that returning s parameter with ok=false means this fallback behavior.
But even better, can we avoid updating inputText before it passes the validation?
replacement, ok := t.validationFunc(t.newText)
if !ok {
	t.inputText = replacement
} else {
	t.inputText = t.newText
}

It would make the API easier to use and understand in my opinion.


Working with Enum

Hi,
Is there a way to work with enums ?
Thank you !
EDIT :
I found a way, using TypeConverter. Implementing a easier way in the API would be an enhancement ?
Sorry for my poor english =)

Adding it in 2.1.0!

Hi just wondering if this is mentioned in document because I thought Enum is not supported so i made a lot of changes, until I saw this.....

Still implemented in 4.0? I don't know where it is in the docs...


CameraException(error, register failed, the sensor listeners size has exceeded the maximum limit 128)

While opening and closing the camera many times in the same day I'm getting the following error:
CameraException(error, register failed, the sensor listeners size has exceeded the maximum limit 128)
Stack trace:
#0 CameraController.initialize (package:camera/camera.dart:289)
#1 Subject.sink (package:rxdart/src/subjects/subject.dart)
#2 _CameraState.initState.. (package:camera_camera/page/camera.dart:57)
Device parameters:
brand: samsung
device: a31
hardware: mt6768
isPhysicalDevice: true
manufacturer: samsung
model: SM-A315G
versionCodename: REL
versionRelease: 10
versionSdk: 29
versionSecurityPatch: 2020-07-01

Maybe the camera is not unsubscribing its listeners when is closed, as described in
https://stackoverflow.com/questions/6919577/sensoreventlistener-doesnt-get-unregistered-with-unregisterlistener-method

Maybe the camera is not unsubscribing its listeners when is closed, as described in
https://stackoverflow.com/questions/6919577/sensoreventlistener-doesnt-get-unregistered-with-unregisterlistener-method


Problems when adding subscriber

I have added a subscriber to the demo code in this repo but it continuously reboots.  I started with a working version of the code in this repo and then merged in changes from the file:
https://github.com/micro-ROS/micro-ROS-demos/blob/foxy/rclc/int32_publisher_subscriber/main.c
My code builds and flashes without any problem.
I start the pre-built docker agent: microros/micro-ros-agent:foxy, and run the installed code on my ESP32.  When start the monitor (idf.py monitor) I get see these error messages:
...
I (1536) wifi station: connected to ap SSID:TP-LINK_AP_8670
Guru Meditation Error: Core  0 panic'ed (LoadProhibited). Exception was unhandled.
Core 0 register dump:
PC      : 0x400d5671  PS      : 0x00060730  A0      : 0x800d7030  A1      : 0x3ffca300
0x400d5671: rcl_node_init at ??:?

A2      : 0x3ffcb044  A3      : 0x3f402ea0  A4      : 0x3f402774  A5      : 0x3ffcb000
A6      : 0x3ffcafa0  A7      : 0x3ffcafa4  A8      : 0x800d5652  A9      : 0xa5a5a5a5
A10     : 0x00000000  A11     : 0x3ffca320  A12     : 0x3ffca320  A13     : 0x400d7228
0x400d7228: __default_zero_allocate at allocator.c:?

A14     : 0x00000000  A15     : 0x3ffcab74  SAR     : 0x00000008  EXCCAUSE: 0x0000001c
EXCVADDR: 0xa5a5a5b9  LBEG    : 0x400014fd  LEND    : 0x4000150d  LCOUNT  : 0xfffffffe

ELF file SHA256: daeebb271d1632ea

Backtrace: 0x400d566e:0x3ffca300 0x400d702d:0x3ffcaba0 0x400d516a:0x3ffcafe0 0x40087205:0x3ffcb0f0
0x400d566e: rcl_node_init at ??:?

0x400d702d: rclc_node_init_default at ??:?

0x400d516a: appMain at /home/build/micro_ros_idf/build/../main/rclc_int32_publisher_subscriber.c:52 (discriminator 3)

0x40087205: vPortTaskWrapper at /opt/esp/idf/components/freertos/port.c:143

Rebooting...

My code is here: https://github.com/AndyBlightLeeds/micro_ros_espidf_component/tree/publisher-1.
Any idea what I'm doing wrong?

Hello @AndyBlightLeeds, have you configured the RMW in the colcon.meta file?
Check here and rebuild: https://github.com/AndyBlightLeeds/micro_ros_espidf_component/blob/d08716642e7ecd6c14391c1423f519624ab78c14/components/micro-ROS/colcon.meta#L36
If you want to know more about the static memory pools in the micro-ROS middleware you can read this: https://micro-ros.github.io/docs/tutorials/core/microxrcedds_rmw_configuration/
Let me know if this solves your problem!

Hi @pablogs9,
Thanks for the quick reply.
I have modified the colcon.meta file by changing "-DRMW_UXRCE_MAX_SUBSCRIPTIONS=0", to
"-DRMW_UXRCE_MAX_SUBSCRIPTIONS=1", rebuilt and flashed but I had exactly the same result.  I then put some rubbish characters in the colcon.meta file to generate an error message and rebuilt but the changes were not picked up.  How do I force the build system to rebuild the colcon.meta file?

I guess that we have work to do here because files that are generated inside the component/micro-ROS folder are not cleaned properly.
Could you test this in a brand new fork?

I'll test on a new clone of the repo and let you know how I get on.

I have created a new clone using the foxy branch with no changes, did menuconfig, build and flash.  All OK.
I then added rubbish characters to the the colcon.meta file and rebuilt.  The build succeeded so the file isn't being picked up.
Just to find out what how the build fails when colcon.meta won't compile, I started again with a new clone, did menuconfig, added the characters AJB on a new line and rebuilt. The build failed as expected with the message:
...
Starting >>> rmw_microxrcedds
--- stderr: rmw_microxrcedds
CMake Error: The source directory "/home/build/micro_ros_idf/components/micro-ROS/micro_ros_src/build/rmw_microxrcedds/AJB"-DRMW_UXRCE_MAX_SUBSCRIPTIONS=0"" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
...

So the colcon.meta file does not get checked on the second rebuild.
Let me know if there is anything else you want me to test.

Ok, so I will keep this issue open in order to remind us that this should be taken into account in further builds of the project.
Regarding your test, are you able to use the micro-ROS subscription by modifying the colcon.meta before building the project? Is the Guru Meditation Error solved?

I took another look at the error messages, I took a guess that the (discriminator 3) text means the 3rd parameter to the function
rclc_node_init_default.  I reviewed my code changes again and realised that the support structure was not initialised when being used by rclc_node_init_default.  Having fixed the code and used the colcon.meta with your suggested changes, I now have a working publisher and subscriber. Yay!
The working code is on my branch publisher-1.
Thanks for your help and support.

BTW, when cleaning the repo to force a rebuild of the colcon.meta file, I used the following commands:
cd <repo_root>
git reset --hard
git clean -dfx
rm -rf components/micro-ROS/micro_ros_dev/ components/micro-ROS/micro_ros_src

These commands reset the sdkconfig.defaults file so I keep a modified copy outside the repo with my WiFi SSID and  password and the IP address of my server and copy it over the file in the repo root.  After that, it is the usual sequence of commands to rebuild and run.

Can you add subscriber example? Repository link?

Can you add subscriber example? Repository link?

@zhivko My personal repo has an example of the subscriber, see:
https://github.com/AndyBlightLeeds/micro_ros_espidf_component/blob/publisher-1/main/rclc_int32_publisher_subscriber.c
Hope this helps!

@zhivko My personal repo has an example of the subscriber, see:
https://github.com/AndyBlightLeeds/micro_ros_espidf_component/blob/publisher-1/main/rclc_int32_publisher_subscriber.c
Hope this helps!

Hello @AndyBlightLeeds it would be super nice if you create a PR with your code to our repo. This way everyone will have access to it.
Hope that you have some time for this. Thanks!

Hello @AndyBlightLeeds it would be super nice if you create a PR with your code to our repo. This way everyone will have access to it.
Hope that you have some time for this. Thanks!

@pablogs9 Good idea!  I'm busy with other work at the moment but I should be able to create a PR in a few days time.

@pablogs9 Good idea!  I'm busy with other work at the moment but I should be able to create a PR in a few days time.

@pablogs9 Finally found time to add my example code to PR #56. Please let me know if anything else needs to be done.


fix: perform basic form validation before address verification

The type of this PR is: Fix
This PR solves EMI-1316
Description
Basic form validation (e.g. required fields) should be performed before the address verification (e.g. calling Smarty API for suggestions). This changes the order to perform basic form validation first.
Before and after
Before

After

Merging this in preparation of the QA! 💪


Automate release

We should find out if its possible to automate our release procedure (creating the package, uploading it to PyPi, etc.)
Example: https://github.com/Deltares/hatyan/blob/main/.github/workflows/pypi-upload.yml

Usefull information: https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/

I created a GitHub workflow that should be able to automate the release steps as listed under https://deltares.github.io/HYDROLIB-core/0.4.1/guides/contributing/#releasing
Once this workflow is merged into main, a new action should be available under the GitHub Actions page. When clicked, you should be able to provide it with a major, minor and patch version for the new release.
The workflow should then:

checkout the code
install python, poetry and all required dependencies
generate the changelog
bump the version
push (tag) to git
build the package
upload the package to PyPI
create a new GitHub release for the new version including the correct name, version and changelog.

Once the workflow has been reviewed, we can merge it back to main. We can then test this workflow when we are going to release a new version for hydrolib-core in the near future and make changes if needed.


Using a local S3 server

I'm looking into the possibilities for testing my application using a mock S3 server.
For example, https://github.com/findify/s3mock this can run on a local port.
Is there currently any way to configure the S3Client to use such a local S3 server? I have not found it.

Yes it is possible. You can use Region::Costom for it. I've used local S3 with Minio this way and it worked just fine.
See https://docs.rs/rusoto_core/0.40.0/rusoto_core/region/enum.Region.html#aws-compatible-services
On September 9, 2019 9:36:19 AM UTC, Erlend Langseth<EMAIL_ADDRESS>wrote:

I'm looking into the possibilities for testing my application using a
mock S3 server.
For example, https://github.com/findify/s3mock this can run on a local
port.
Is there currently any way to configure the S3Client to use such a
local S3 server? I have not found it.

Nice, thanks!


Menu section with items and section

I am from @sergio-ocon team working in consumption. We need a new tab inside Cloud Intelligence, that will be a section with 4-5 items inside but we can't create a section with items inside.
The template does not allow mixing 2 & 3 levels under one top-level section
app/views/layouts/_vertical_navbar.html.haml
I changed  app/helpers/application_helper
  def secondary_nav_class(item)
    item.items.collect(&:id).include?(@layout) ? 'active' : nil
  end

to
  def secondary_nav_class(item)
    Array(item.try(:items)).collect(&:id).include?(@layout) ? 'active' : nil
  end

because that throw error undefined method `items' for #Menu::Item:0x007ffb43d41138 but @martinpovolny told me that we need make changes in the template

@skateman, @epwinchell : the problem is that in this case @aljesusg mixes Items and Sections on the same level in one top level Section.
I believe that this should not be a problem on the menu definition side, but it's not supported by the markup in app/views/layouts/_vertical_navbar.html.haml.

It would be great to have this


Settings are not working at export

Hi,
I've tried some settings for the export, and appart from "Print Profile" they seem not working.
Some I've tested : "Show page numbers"/ "Each Scene on new page" / "Footer" /  "Header" / "Watermark".

I'll take a look at it this weekend.

Thanks. By the way, I'll try to contribute during my August vacations.

Sorry, haven't been able to get to this yet.  Been in the middle of a big move... Thought I could do it sooner.  Will undertake to verify asap.

I've just run a test against this and found all of the settings under "Custom" section to be functional.  Have you seen any errors?  Any other information that could be related to producing this behavior?  The most likely way I could see this not rendering properly is if the config json isn't being properly written during a render... but I would expect some kind of write failure from the js code.  A file permission issue is always a possibility (for instance if the config file isn't owned by the current os user for some reason).

I'm having the same issue. No settings change anything in the PDF or previews.

@thegreentimtam @tillix4 Can we get some more information about your environments?  OS, OS version, Atom version, and Fountain version?  We have to figure out a way to reproduce this to fix it.

Windows 10 v1803; Atom 1.30.0; Fountain 0.8.0.

Hi, sorry for taking so long. I managed some tests with outline-view-tests.fountain and tried to change settings like Scene Numbers, Font Family, Double space between scenes, and many other in random. I also tried in user and dev modes for each OS test:
Windows 10 v1803; Atom 1.30.0; Fountain 0.8.0 : Nothing changes.
OSX 10.11.6; Atom 1.30.0; Fountain 0.8.0 : Nothing changes.
KDE Neon 5.13; Atom 1.30.0; Fountain 0.8.0 : Nothing changes.
I compared exported PDF files with changed settings for every test, they are perfectly identicals. Not even the page format (a4 or us-letter) worked.
Hope it'll help.

This is a confirmed bug, possibly having to do with: https://github.com/superlou/fountain-atom/blob/3480f98d111a9c6bf2560fd2db7c9568935923ab/lib/fountain-pdf-config-generator.coffee#L12-L13.
We will look into this immediately.

Wonderful. It works for me ! Thanks a lot.


ARROW-2479: [C++] Add ThreadPool class

A ThreadPool class with future-returning task submission, and the ability to change number of worker threads on-the-fly
Tests for the ThreadPool class, including stress tests
A singleton thread pool for cpu-bound tasks, configured based on hardware capacity
A public API to change global thread pool capacity
Migrated the Arrow codebase to using the global thread pool (except APIs taking a nthreads, see below)

Remaining open question:

[ ] what do we do with APIs that take a user-facing nthreads argument? (the Pandas conversion routines, which are able to convert/copy different columns in parallel)

It could be that the symbol depends on the version of the STL implementation. We can add that symbol to cpp/src/arrow/symbols.map to hide it always. If it's not there, it is silently ignored.

There's a crash in the manylinux job, unfortunately it seems the core dump isn't found by our Travis-CI script.

I'm not sure I understand how the Docker container works. Apparently there are two copies of Arrow? One in /arrow/ and one in /io/arrow/. If I modify something under /io/arrow/cpp, nothing changes it seems.

The copy in /io/arrow is not relevant, only the one that is in /arrow is the one that is used. Once you change something in the sources, you must run the docker build command again. (This is sadly not incremental)

Also, it seems libplasma.so is built only during the Python step?

That is ok, the first build of Arrow is solely for building parquet-cpp once and not per Python version.

So, I have the gdb backtrace:
(gdb) bt
#0  0x00007fca37db7377 in void std::__once_call_impl<std::_Bind_simple<std::_Mem_fn<void (std::__future_base::_State_base::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()()>&, bool&)> ()(std::__future_base::_State_base*, std::reference_wrapper<std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()()> >, std::reference_wrapper<bool>)> >() () from /io/arrow/python/pyarrow/libarrow.so.0
#1  0x00007fca410dc1d3 in pthread_once () from /lib64/libpthread.so.0
#2  0x00007fca372e6d94 in std::__future_base::_Task_state<std::_Bind<void (*()(unsigned char*, unsigned long, unsigned long*))(unsigned char const*, long, unsigned long*)>, std::allocator<int>, void ()()>::_M_run() () from /io/arrow/python/pyarrow/libplasma.so.0
#3  0x00007fca37dc39fe in arrow::internal::ThreadPool::WorkerLoop(std::_List_iterator<std::thread>) () from /io/arrow/python/pyarrow/libarrow.so.0
#4  0x00007fca37f5b560 in execute_native_thread_routine () from /io/arrow/python/pyarrow/libarrow.so.0
#5  0x00007fca410d683d in start_thread () from /lib64/libpthread.so.0
#6  0x00007fca407c1fdd in clone () from /lib64/libc.so.6

(there are other threads)

As it's in __once_call_impl could you check if it still happens if you remove __once_proxy from the symbols.map file again? (and comment out the check)

It looks worse in that case:
#0  0x0000000000000000 in ?? ()
#1  0x00007fe1712781d3 in pthread_once () from /lib64/libpthread.so.0
#2  0x00007fe1674825c4 in std::__future_base::_Task_state<std::_Bind<void (*()(unsigned char*, unsigned long, unsigned long*))(unsigned char const*, long, unsigned long*)>, std::allocator<int>, void ()()>::_M_run() () from /io/arrow/python/pyarrow/libplasma.so.0
#3  0x00007fe167f5fa3e in arrow::internal::ThreadPool::WorkerLoop(std::_List_iterator<std::thread>) () from /io/arrow/python/pyarrow/libarrow.so.0
#4  0x00007fe1680f75a0 in execute_native_thread_routine () from /io/arrow/python/pyarrow/libarrow.so.0
#5  0x00007fe17127283d in start_thread () from /lib64/libpthread.so.0
#6  0x00007fe17095dfdd in clone () from /lib64/libc.so.6

By the way, I'm curious about this comment in symbols.map:
      # devtoolset or -static-libstdc++ - the Red Hat devtoolset statically
      # links c++11 symbols into binaries so that the result may be executed on
      # a system with an older libstdc++ which doesn't include the necessary
      # c++11 symbols.

It looks like libarrow.so still links to libstdc++ dynamically?

Yes, we still link dynamically and we probably never run on systems that are older than CentOS 5.

Does it mean the comment above about devtoolset and static linking is obsolete?
Also, how do ensure the libstdc++ ABI is ok, if we compile against the devtoolset's header file but dynamically link with the system-global libstdc++?

Does it mean the comment above about devtoolset and static linking is obsolete?

yes

Also, how do ensure the libstdc++ ABI is ok, if we compile against the devtoolset's header file but dynamically link with the system-global libstdc++?

We hope that it works. Sadly this is not always the case. Thus my yet failed attempt to fix this: https://github.com/apache/arrow/pull/1464

Ok, for the record, the devtoolset doesn't automatically link statically, but it has a linker script to fetch missing symbols from the statically-compiled version:
# cat /opt/rh/devtoolset-2/root/usr/lib/gcc/x86_64-CentOS-linux/4.8.2/libstdc++.so
/* GNU ld script
   Use the shared library, but some functions are only in
   the static library, so try that secondarily.  */
OUTPUT_FORMAT(elf64-x86-64)
INPUT ( /usr/lib64/libstdc++.so.6 -lstdc++_nonshared )

And the thread-local "once_call" symbols (used by the std::call_once implementation) are only in the static library:
# nm -C --defined-only /opt/rh/devtoolset-2/root/usr/lib/gcc/x86_64-CentOS-linux/4.8.2/libstdc++_nonshared.a | grep once_call
0000000000000000 W void std::__once_call_impl<std::_Bind_simple<std::_Mem_fn<void (std::thread::*)()> (std::reference_wrapper<std::thread>)> >()
0000000000000000 B std::__once_call
0000000000000000 B std::__once_callable

Here's the std::call_once implementation from the devtoolset:
https://gist.github.com/pitrou/b8df9ffbcfcdf413b4b10b66f477b1a8

If I look back at gdb traceback in https://github.com/apache/arrow/pull/1953#issuecomment-386034595, my impression is the following:

in frame 3, libplasma.so calls std::call_once: this sets up a thread-local __once_callable with the actual callable, then calls pthread_once with a helper __once_call_impl
in frame 2,  pthread_once which calls an internal helper __once_proxy inside libstdc++ which probably redirects to __once_call_impl
in frame 1, __once_call_impl calls the thread-local __once_callable

Since frame 1 executes inside libarrow.so, and the *once_call* symbols have been statically linked (as they don't exist on the system libstdc++.so), it sounds plausible that frame 1 doesn't get the right function pointer and crashes.

Some debug prints inside the call_once implementation seem to confirm that hypothesis:
-- call_once: callable = 0x7f13f347f050
-- once_call_impl: callable = (nil)

(by the way, this is because libstdc++ implements setting a future's result using std::call_once)

The remaining question here is about the APIs that take an explicit nthreads argument. What should we do with them? @wesm

Regarding the nthreads= argument, I would change it to:

Make the ThreadPool accessible from Python, so that we can set the number of Threads there.
Deprecate nthreads and add in some places a new argument use_threads. For example in the Parquet reader, one can only use threads if the underyling file handle is thread-safe. In these cases, the user should explicitly enable threading.

It seems that by default we will start with a constant of 4 threads in the threadpool (looking into MakeThreadPool). It would be nice to use some other indication on how much threads to start. Often the environment variable OMP_NUM_THREADS is used for this.

It seems that by default we will start with a constant of 4 threads in the threadpool

It uses the number returned by std::thread::hardware_concurrency, which should (IMHO) never fail on "normal" setups. 4 threads is only a conservative fallback in case the former fails.

It would be nice to use some other indication on how much threads to start. Often the environment variable OMP_NUM_THREADS is used for this.

Yeah, that's something we'll have to do. There's some interesting discussion here about which heuristic to use exactly:
https://bugs.python.org/issue32986

Sorry I missed the ping here; I was traveling and had my GitHub notifications muted

I will do a round of code review; if you want to make any changes then this can go in a new patch

Feel free to incorporate any changes into the latest PR

This PR seems to have introduced https://issues.apache.org/jira/browse/ARROW-2657.

You have to import TensorFlow first. TensorFlow is not respecting the manylinux1 standard and is using newer compilers. We've seen bugs related to libstdc++ reported elsewhere

It does look like there's an issue related to that pthread symbol though that we should see if can be fixed


⚠ Samsung has degraded performance

In dbfc59f, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 2328 ms

Resolved: Samsung performance has improved in 82ab586.


tes3merge not working

Heres the log, i have no idea whats happening.
TES3Merge.log

im not using openmw but i am using mwse and mgex


Update tau.cpp

Fixing order of includes. The ifdef check has to happen after config.h has been included, or else the perfstubs header file won't get included. Also, adding an argument name for the end_tau() function.

@khuck, if you can confirm that the you are able to successfully run with the TAU logger with these changes, then I will go ahead and merge this PR.

THanks - I haven't tried running yet, I'll do that now...

@upsj, we cannot do that anymore as batch-develop and develop have diverged and batch-develop cannot be rebased on develop anymore. But we should create a PR to have these changes in develop as well.

Sure we can, the patch applies just fine on develop.
@khuck could you please reset your branch onto develop, cherry-pick 2b16c0f162a5d16cc5834adcbd384e95cff61e64 and push again? Then we can change the base of the PR to our development branch.

I thought we just need two pr: one for batch-develop and another for develop.
after this can be merged, create another pr for develop

batch-develop was last updated 2.5 months ago, I think we should focus on our active development branch.

@khuck , thank you again for the PR. Please create a PR for our main development branch, develop as well, as these changes are necessary there as well.


Change the name to RO-Crate?

We have to settle on the name and it's spelling of our spec. Is it ROLite, RO Lite, RO-Lite, Research Object Lite?
In discussion with @eocarragain and @ptsefton we are not sure if "Lite" gives the right message for anyone outside the RO community (who therefore also does not know what is "RO").
One suggestion is that as we are intending to build on both Research Objects and DataCrate the name could reflect both origins.
How about RO-Crate? Some properties of a crate is that it is not closed, you can see through it's edges, you can put anything inside it, even something large.
(I'm afraid I would want the - in there to avoid sounding like the ROC-rate - which would upset the economists)

This name is growing on me - I say yes, unless there's a complete rejection of DataCrate as a basis for encoding for manifests.

RO-Crate works for me! Also, RO part could also been understood as read-only , transmitting persistence

I prefer RO-Lite, but agree inclusion of 'Crate' better acknowledges the origins, so would settle for RO-Crate

+1 to RO-Crate . Like "RO Bundle" it also emphasizes the packaging/bundling emphasis of this spec compared with Research Objects in the broader sense.

We agreed to this change on the telco
@stain, can you update repo name etc. & close this ticket? cheers

Changed the name to "RO-Lite" everywhere, and renamed GitHub repository. Redirect should be in place both for GitHub repo and GitHub pages.
New URL: https://researchobject.github.io/ro-crate/


Fix nextEnvironment value for staging

This fixes the value the specifies the next environment to promote a release to. Staging promotes to production, otherwise it'd try to update the image tag for the same environment.

Haha whoops 😅 Good catch and thanks for fixing!