Auto populate the container version based on a "primary rpm" version

After opening [0] in koji-containerbuild it seems that atomic-reactor is a better place to implement this feature.
After briefly looking at the pre_bump_release plugin, I think the best way would be to create a similar plugin.
My inital thoughts are :

Dockerfile to have a new label that stores the primary rpm used to populate the version of the container for example "rpm.version". (we might be able to reuse the component label but having a new label brings flexibility)

"rpm.version" to be added to the LABEL_TYPE_VERSION so a Dockerfile with only "rpm.version" will be valid. If this label is not used then we still need to provide the Version label.

Add a new plugin in atomic-reactor (pre-build)

plugin to read the "rpm.version" label content
plugin to query koji with getLatestBuild ---> This API requires a koji tag ( need to investigate how we can find out which tag to use)
plugin to use the result of koji getLatestBuild to replace the value of "rpm.version"

[0] - https://github.com/release-engineering/koji-containerbuild/issues/95

plugin to query koji with getLatestBuild ---> This API requires a koji tag ( need to investigate how we can find out which tag to use)

This should be from either the koji build or destination tag of the given koji target.

Continuing discussion over here.
With post plugin, you would be 100% sure on the version. It's possible that querying koji could result into getting a different package than koji predicted? Also what about images which don't utilize koji for getting packages? When they get packages from dedicated repositories.
The implementation as a post build plugin could look like this:

version=$(docker run $image rpm -q --qf "%{version}" $package)
docker create --name working-container $image
docker commit -c "LABEL version=$version" working-container $image

@TomasTomecek, the Dockerfile at /root/build-info would also need updating. The internal state of atomic-reactor must also be updated for tag_and_push plugin, for example. There could be other oddities too, which makes me shy away from this approach.
I wanna say that the safest way would be to do as a prebuild plugin, then verify it was correctly set, say in all_rpm_packages. But it would be great if we didn't tie this to koji at all.
I think postbuild would make the feature more versatile as it wouldn't require koji. I'm just concerned that implementation might not be as straight forward. Maybe it's worth giving that a try and see how it lands?

Oh! It has to be done as a prebuild plugin if you want to leverage bump_release.

Maybe it's worth giving that a try and see how it lands?

+1

I see how the postbuild gives more flexibility and assurance that we have the correct version. It seems indeed a better solution that the prebuild.
I ll look at some postbuild plugin and try to see what I can do :)

How would this work for container-first container images, where there may not be any RPMs at all?

Closing due to lack of interest and activity. If you're still interested feel free to reopen it.


Actually report errors when hitting blacklisted errnos.

Motivation:
It's not all that useful to print the value of the pointer from sterror.
Instead we should aim to print the string. While we're here, let's avoid
repeating too much code.
Modifications:

Centralised the handling of blacklisted errnos.
Correctly printed the error string when a blacklisted errno is hit.
Updated the output of the assertion to include the function failing.

Result:
Better diagnostics when blacklisted errnos are hit.

haha


Handy script to analyse the project dependencies

Hi all,
I have been working on a script to complement the information stack dot produces under the form of a Graphviz graph (which @domenkozar already showed me). This script enumerates all the dependencies in cardano (by piggybacking on stack list-dependencies --test --bench and also tracks the direct & indirect dependencies (mostly relying on ghc-pkg field). It finally produces a table like this (which we can customise later):

I have deliberately filtered the cardano-* project family from the reverse dep list (thus some packages has a 0 in the rev deps section) so actually focus a bit better (in the future it's easy to add an option like --exclude-cardano and by default show everything). The game would be to start from the top (the table is sorted by number of direct deps) and for those check the packages which have the minor number of reverse dependencies, which means that, if we eliminate those, we can effectively get rid of the package from the codebase. For example, the screenshot shows the case of store (which in other branches has been already removed): It has as reverse deps timewarp and kademlia, so dropping store from both will allow us to remove store (disclaimer: it's basically what actually happened in other wip-branches).
To run the script, first build the project as usual, and then:
./scripts/haskell/dependencies.sh

I hope we can use this as a starting point to optimise our CI times and the project in general. Let me know what you think! cc @georgeee @neongreen

The list (with the cardano deps filtered)
deps_no_cardano.txt

Nice work! Could we also figure out how many dependencies will store remove?

Yep, having a field like “unique direct deps” would be very helpful

Nice work! Could we also figure out how many dependencies will store remove?
Yep, having a field like “unique direct deps” would be very helpful.

Aye, that would be my next step, but didn't get to that as I wanted to run this first pass for your scrutiny to make sure this piece of work was actually useful 😉

Ok, I have updated the script with the counters you asked:
deps_excluding_cardano_updated.txt
Assuming my code is correct (a review of uniqueDependenciesFor is surely appreciated) I can see how turtle, for example, it's used only in cardano but it's introducing 6 unique deps to the project. Maybe switching to something like shelly (which I have used professionally in the past with success) might be better.

For reasons I'm struggling to understand, omitting --test and --bench from stack list-dependencies wouldn't change a thing, and criterion (as well as hspec) would still show up. I have manually filtered those out of the initial list to produce the following deps list:
deps_no_cardano_lib_only.txt

Okay, here are my thoughts on dependencies:

Removing turtle, yes.
We can probably copy HasLens into CSL and get rid of the ether dependency.
I've heard that we can get rid of scotty (and we haven't done that yet only because nobody cared enough).
I would kinda like to throw away benchmarks and get rid of criterion and all its deps.
We use stm-containers in precisely one place. I don't know whether switching to a Map in a TVar there would lead to a performance degradation or not (for all I know, that map might be updated only once per second or something, and then using stm-containers is like bringing a gun to a knife-fight).
We don't use witherable and can get rid of it right now.
string-qq can be removed in favor of neat-interpolation.
optparse-simple and optparse-text can be removed too (they don't give that much over optparse-applicative).
serokell-util should probably lose a bunch of dependencies (like cereal-vector, aeson-extra and binary-orphans).

I tend to subscribe with you on pretty much the whole line. I would also add that it might be total overkill but it should be in principle possible to get rid of lens as well: from a first skim through the codebase looks like the only reason why we cannot switch to microlens (from which we depend already) is the absence of makePrisms from the latter (to be precise from microlens-th), but looks like migrating that is a bit of a pita, so not worth it.
On the scotty topic I have experience with both scotty and snap so I should be in the position to migrate everything to snap, provided that we can implement the same thing scotty provides us with only snap-core and snap-server (the only 2 dependencies pulled in from ekg. Depending from snap is even worse than depending from scotty 😛  ).

AFAIK we want everything to use wai+warp+servant and get rid of snap as well :)

AFAIK we want everything to use wai+warp+servant and get rid of snap as well :)

Alas ekg has a direct dependency on snap-server and snap-core, so not sure how many tricks we can play there (apart, of course, we don't need ekg for the library linking, only for the executable 😉 )

I wonder whether @tibbe might be open to adding a flag to ekg to use warp instead of snap. (Or even switching to warp completely.)

As interim, we could have a crack at http://hackage.haskell.org/package/ekg-wai (I never used it, but Hackage revealed with whilst searching for ekg). Might be a starting point!

from a first skim through the codebase looks like the only reason why we cannot switch to microlens (from which we depend already) is the absence of makePrisms from the latter (to be precise from microlens-th), but looks like migrating that is a bit of a pita, so not worth it.

I can easily write microlens-pro with prisms and isomorphisms. I don't think it will be worth it because lots of packages (swagger and wreq, for instance) depend on lens, and retting rid of them as well (or forking them) wouldn't be easy. It can be done, but I hope that Hydra will come sooner than our predicament becomes dire enough to justify the cost of the lens-switch :)

@neongreen can you prioritize your list by affection on build time (or our expectations)? Is it doable?

I don't think it will be worth it

Yea, it was my idea as well, but I wanted to be sure by getting a second opinion 😉
@neongreen @georgeee I haven't mentioned it explicitly, but needless to say I'm happy to take the burden at going through Artyom's list and reduce the deps load.
Only question I have for you is: can we merge this handy script in the meantime, and move the discussion somewhere else? 😀

can you prioritize your list by affection on build time (or our expectations)? Is it doable?

Btw, using something like sort -k3 -r should do the trick (ymmv, I did this on my local machine earlier this week but I couldn't find the exact parameters for sort, but it's definitely doable) 😉

/cc @23Skidoo who now co-maintains ekg.
I've long wanted to make ekg use a lighter-weight (meaning fewer dependencies/faster to compile) HTTP server. It only needs very basic functionality. Here's how I see the ekg ecosystem:
Existing packages:

ekg-core: defines the various types of metrics.
ekg-json: defines the JSON protocol used both by the HTML UI and other libraries/executables that need programmatic access to the metrics.
ekg: uses the above to create a "one liner" drop in monitoring for any Haskell executable. Will use whatever HTTP server works (i.e. it should be transparent to the user). Right now this is snap, could be something lighter-weight.
ekg-statsd: pushes metrics to statsd. You could imagine other metrics "pushers", such as ekg-graphite, etc.

Packages that should exist:

ekg-warp/ekg-snap/ekg-yesod: a way to expose ekg metrics through a specific web server/framework. Could integrate with authentication provided for that server/framework, add server/framework-specific metrics (e.g. "snap.requests", "yesod.requests.404"), etc. The ekg package could use one of these as an implementation detail.
ekg-ui: it would be nice to package up the ekg HTML UI so it could be reused by any server/framework as well.

I don't have time to work on this but if someone has please do!

Thanks @tibbe.
@adinapoli since we're primarily using ekg-statsd, it sounds like we can adjust things to depend only on ekg-core and ekg-statsd. Perhaps optionally in dev mode we can use the full ekg so that developers can see the html ui of individual nodes for convenience, whereas the devops team will always use the statsd route.

@dcoutts Sounds good to me! ;)


some typos revision

all modification is base on the vscode-angular2-snippets.  please check the changes.

thanks!


(URGENT) gem release of private gems failed due to broken rake CSS generation task

All private gem pushes to GitHub packages are failing due to this:
https://github.com/metanorma/metanorma-nist/runs/6454462062?check_suite_focus=true
git add ~
lib/isodoc/nist/html/_coverpage.css
lib/isodoc/nist/html/htmlstyle.css
lib/isodoc/nist/html/nist.css
lib/isodoc/nist/html/nist_cswp.css
lib/isodoc/nist/html/wordstyle.css
lib/isodoc/nist/html/wordstyle_cswp.css
fatal: /home/runner: '/home/runner' is outside repository at '/home/runner/work/metanorma-nist/metanorma-nist'
rake aborted!
Command failed with status (128): [git add ~...]
/home/runner/work/metanorma-nist/metanorma-nist/vendor/bundle/ruby/2.7.0/gems/isodoc-2.1.1/lib/isodoc/gem_tasks.rb:56:in `block in git_cache_compiled_files'
/home/runner/work/metanorma-nist/metanorma-nist/vendor/bundle/ruby/2.7.0/gems/isodoc-2.1.1/lib/isodoc/gem_tasks.rb:55:in `git_cache_compiled_files'
/home/runner/work/metanorma-nist/metanorma-nist/vendor/bundle/ruby/2.7.0/gems/isodoc-2.1.1/lib/isodoc/gem_tasks.rb:34:in `block in install'
/home/runner/work/metanorma-nist/metanorma-nist/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/opt/hostedtoolcache/Ruby/2.7.6/x64/bin/bundle:23:in `load'
/opt/hostedtoolcache/Ruby/2.7.6/x64/bin/bundle:23:in `<main>'
Tasks: TOP => build => build_scss
(See full trace by running task with --trace)

Notice that the command is trying to git add the entire home directory, which causes the failure.
This code is the generation of CSS from SCSS located here:
https://github.com/metanorma/isodoc/blob/main/lib/isodoc/gem_tasks.rb#L54-L64

Clearly the CLEAN constant is probably not meant to be used this way. I suspect this entire file should be rewritten.
This code SHOULD NOT directly call sh "git ...".
There is a typo for notify_borken_compilation should have been notify_broken_compilation.

Given the information on this post:
https://stackoverflow.com/questions/27303575/what-is-the-right-way-to-use-clean-rake-task
We know that CLEAN is a Rake FileList. It is meant to store the files/paths that are to be cleaned, NOT retain the CSS file paths.
If we wanted to add the generated CSS paths, use another FileList or just an Array at this line:
https://github.com/metanorma/isodoc/blob/d95a3f9a1f5d42cb476d2f931e7e125522718760/lib/isodoc/gem_tasks.rb#L132

https://github.com/metanorma/isodoc/pull/406 - merged


Push the game to Heroku

Well, it seems to be about that time. Oh boy the bugs we're going to find....

The app is now available at http://betrayal.seanharger.com
6970dae95c5f5fead243a37cbf12dd2066fbda48


separate system-tests to full and reduced set of tests

With growing count of tests will not possible to execute full set of tests in master. For this reason should be system-tests separated to full and reduced set of tests. Full sort of tests will executed separately.

I think we need to split the tests into different packages first. Then, we must configure gradle tasks for each set. Example 47.10 in https://docs.gradle.org/current/userguide/java_plugin.html shows an example

this is done, tests were separated to SmokeTests and others
-> close


[Bug] Identity is not populated with Signed in user upon workflow termination

Describe the bug
Identity is not populated with Signed in user upon workflow termination
To Reproduce
Steps to reproduce the behavior:

Go to 'Temporal Web' using SSO option.
Click on 'workflow'
Terminate it. In the pop up mention the reason for termination.

Expected behavior
I was hoping identity will be mine based upon SSO. But, instead it came up as empty.
Screenshots
If applicable, add screenshots to help explain your problem. You can also use Loom to do short video bug reports.
Desktop (please complete the following information where relevant):

OS: iOS
Browser: Chrome
Temporal Web Version ?
Temporal SaaS

hey Sunny,

we discussed this today and consider it a feature request rather than bug
the system doesnt make decisions based on identity field, this reason for termination field is purely for audit purposes
we are not clear what you mean when you talk about "identity" - when you say "instead it came up as empty", what exactly do you mean? maybe a screenshot would help.


error switching from 20.0.3 to 20.0.4 singularity

fmriprep version 20.0.4 (via singularity)
When switching from fMRIPrep 20.0.3 to 20.0.4 I get the below error message. This subject completed successfully using singularity container 20.0.3 and the only thing changed in the code/data was pointing to the new singularity image for 20.0.4

	1: [WARN] The recommended file /README is missing. See Section 03 (Modality agnostic files) of the BIDS specification. (code: 101 - README_FILE_MISSING)

	Please visit https://neurostars.org/search?q=README_FILE_MISSING for existing conversations about this issue.

        Summary:                Available Tasks:        Available Modalities:
        24 Files, 3.52GB        task-gaze               T1w
        1 - Subject             task-jovi               bold
        1 - Session             task-rest               fieldmap

	If you have any questions, please post on https://neurostars.org/tags/bids.

Making sure the input data is BIDS compliant (warnings can be ignored in most cases).
200324-14:16:49,110 nipype.workflow IMPORTANT:

    Running fMRIPREP version 20.0.4:
      * BIDS dataset path: /tmp/workflow_1009/BIDS.
      * Participant list: ['1009'].
      * Run identifier: 20200324-141648_8645153e-701d-4291-99b1-db39f1124b9d.

    Spatial References: anat, fsLR:den-32k, MNI152NLin6Asym:res-2

200324-14:16:49,950 nipype.utils INFO:
	 No new version available.
200324-14:16:52,482 nipype.workflow IMPORTANT:
	 Creating bold processing workflow for "/tmp/workflow_1009/BIDS/sub-1009/func/sub-1009_task-gaze_run-01_bold.nii" (0.28 GB / 275 TRs). Memory resampled/largemem=1.13/1.91 GB.
200324-14:16:52,490 nipype.workflow IMPORTANT:
	 No single-band-reference found for sub-1009_task-gaze_run-01_bold.nii
200324-14:16:53,78 nipype.interface INFO:
	 We advise you to upgrade DIPY version. This upgrade will open access to more function
200324-14:16:53,82 nipype.interface INFO:
	 We advise you to upgrade DIPY version. This upgrade will open access to more function
200324-14:16:53,82 nipype.interface INFO:
	 We advise you to upgrade DIPY version. This upgrade will open access to more models
200324-14:16:53,988 nipype.workflow IMPORTANT:
	 Creating BOLD surface-sampling workflow.
Process Process-2:
Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/local/miniconda/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/cli/run.py", line 693, in build_workflow
    bids_filters=bids_filters,
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/workflows/base.py", line 274, in init_fmriprep_wf
    bids_filters=bids_filters,
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/workflows/base.py", line 638, in init_single_subject_wf
    use_syn=use_syn,
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/workflows/bold/base.py", line 946, in init_func_preproc_wf
    repetition_time=metadata['RepetitionTime'])
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/workflows/bold/resampling.py", line 743, in init_bold_grayords_wf
    for hemi in 'LR'
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py", line 425, in validate
    value = super(MultiObject, self).validate(objekt, name, newvalue)
  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_types.py", line 2336, in validate
    return TraitListObject( self, object, name, value )
  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py", line 2313, in __init__
    raise excp
  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py", line 2305, in __init__
    value = [ validate( object, name, val ) for val in value ]
  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py", line 2305, in <listcomp>
    value = [ validate( object, name, val ) for val in value ]
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py", line 329, in validate
    value = super(File, self).validate(objekt, name, value, return_pathlike=True)
  File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py", line 134, in validate
    self.error(objekt, name, str(value))
  File "/usr/local/miniconda/lib/python3.7/site-packages/traits/trait_handlers.py", line 172, in error
    value )
traits.trait_errors.TraitError: Each element of the 'current_sphere' trait of a DynamicTraitedSpec instance must be a pathlike object or string representing an existing file, but a value of '[]' <class 'str'> was specified.

This is my code to run singularity:

FMRIPREP=/sw/examples/singularity/fmriprep-20.0.4.simg
SURF_LICENSE=${SCRIPTDIR}/data/fs_license.txt

BIDS_DIR=/tmp/workflow_${SUB}/BIDS
OUTPUT_DIR=/tmp/workflow_${SUB}/derivatives
WORK_DIR=/tmp/workflow_${SUB}/work

singularity run --cleanenv \
$FMRIPREP $BIDS_DIR $OUTPUT_DIR participant \
--nthreads ${NTHREADS} \
--mem-mb ${MEMMB} \
--fs-license-file=$SURF_LICENSE \
--participant-label=sub-${SUB} \
-w $WORK_DIR \
--ignore slicetiming \
--cifti-output \
--use-aroma \
--aroma-melodic-dimensionality -100 \
--output-spaces anat fsLR:den-32k MNI152NLin6Asym:res-2

since there were changes to the templateflow templates in 20.0.4, you may need to clear your local template cache (~/.cache/templateflow by default) to update fetching.

@saigerutherford did clearing your cache do the trick? If not, we have recently made changes to templateflow that should help avoid this down the line. You'll have to do these manually for now, but it will be shipped within the 20.1.0 release.

Download templateflow locally

pip install --upgrade templateflow==0.6.0rc2

Force the update of the templates skeleton

python -c "import templateflow as tf; tf.update(local=True, overwrite=False)"

this should be automatically fixed when using 20.0.7+, reopen if you are still having issues after upgrading.


tests/functional: Use the ansible reboot module

Ansible 2.7 introduces the reboot module so we don't need to use the
shell/reboot + wait_for tasks.
https://docs.ansible.com/ansible/latest/modules/reboot_module.html
Signed-off-by: Dimitri Savineau<EMAIL_ADDRESS>

We saw two failures of this module in the past 24 hours.  The VMs either failed to go down or rebooted successfully and ansible just couldn't reconnect.
https://2.jenkins.ceph.com/job/ceph-ansible-prs-centos-container-all_daemons/3274/consoleFull#1762410514574abcf-086e-4053-9702-6f4b34086ae8
https://2.jenkins.ceph.com/job/ceph-ansible-prs-centos-non_container-all_daemons/2948/consoleFull#-6395142924574abcf-086e-4053-9702-6f4b34086ae8
Objections to reverting this PR?


CONFIGURE_OPTS not honored when building RBX

The README states that "CONFIGURE_OPTS lets you pass additional options to configure", and "RUBY_CONFIGURE_OPTS... allow[s] you to specify options for building MRI".
However, when building RBX, only RUBY_CONFIGURE_OPTS is actually used, not CONFIGURE_OPTS. This seems wrong in 2 ways: first, according to the docs, RUBY_CONFIGURE_OPTS was supposed to be for MRI, not for RBX. And second, CONFIGURE_OPTS is not handled the way the docs say it should be.
Either the behavior is wrong, or the documentation is wrong, or both. I'm not sure what your intention was, or I would send a PR.

Thanks for reporting. Yes, the "MRI" bit is wrong. It should say that RUBY_CONFIGURE_OPTS is used to configure the Ruby install but installs for any of the dependencies like libyaml or openssl.

How about CONFIGURE_OPTS? Is it only passed to configure when building dependencies?

No, it's passed to all ./configure operations, including Ruby

We gave-up to support rbx build now.


Ability to reuse classes that are generated as items for each NPC

Could you maybe explain what you mean with this?

Basically it's because of stuff like this.. I get a fresh one of these for
every generated NPC and I'm generating an entire city worth of NPC's
[image: image.png]
On Mon, 2 Nov 2020 at 15:51, ardittristan<EMAIL_ADDRESS>wrote:

Could you maybe explain what you mean with this?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ardittristan/VTTNPCGen/issues/6#issuecomment-720556142,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACHF4XGSGG3BXPGPTQR2OTSN3IRFANCNFSM4TGZ4HSA
.

Thanks for the explanation! Could you maybe send the image directly through github or use an url instead? Images don't seem to work when coming from mail.


Feat: Add a bot protection form

Currently we are having a lot of bots and spammers joining our slack.
We attempted to make this google form to hold them off: https://forms.gle/fB9EZj7aEdTftqS98
Google forms do not allow you to show a success message if you get everything right though :( So right now I have the slack link showing even if they put the answers in wrong
Maybe there is a way to make our own form for this? Then only if they get the answers correct do they get let into the slack

we can use a captcha: https://scripts.nuxt.com/scripts. if anyone wants to work on it I'd be happy to help!


ElectronTeamID to submit to mas

Is this right pace to mention ElectronTeamID ??

@GregorBiswanger , do you know where can I set the value electrontemaid, so it shows up in info.plist once the app is built?


Adjust some level 40 agi/int crit mults

Added level 40 agi and int crit mults for Warrior, Rogue, Druid, Hunter and Shaman. Druid is first-hand data, others are what my WA sent me from guild members, but should be correct too.
Also removed the global int crit scaling from mana.go (was remnant WotLK code). Moved it to warlock/pet.go to no alter behavior there, as that was the only place that probably still needs that stat dependency. Added a TODO comment there because in any case the value will not be correct.

Guess I need to update the tests for new values too?

Guess I need to update the tests for new values too?

You need to merge latest master in your branch and then update tests due to recent changes


OP_REQUIRES failed at constant_op.cc

There is a mistake,
In Python, I use tf.while_loop to send LSTM the initial state. The following error occurred when freezing the model to call C language.

2020-10-29 17:19:01.449470: E tensorflow/core/framework/tensor.cc:555] Could not decode variant with type_name: "tensorflow::TensorList".  Perhaps you forgot to register a decoder via REGISTER_UNARY_VARIANT_DECODE_FUNCTION?
2020-10-29 17:19:01.451491: W tensorflow/core/framework/op_kernel.cc:1744] OP_REQUIRES failed at constant_op.cc:82 : Invalid argument: Cannot parse tensor from tensor_proto.
2020-10-29 17:19:02.134350: E tensorflow/core/framework/tensor.cc:555] Could not decode variant with type_name: "tensorflow::TensorList".  Perhaps you forgot to register a decoder via REGISTER_UNARY_VARIANT_DECODE_FUNCTION?
2020-10-29 17:19:02.194184: W tensorflow/core/framework/op_kernel.cc:1744] OP_REQUIRES failed at constant_op.cc:82 : Invalid argument: Cannot parse tensor from proto: dtype: DT_VARIANT

Is it necessary to compile source code to support it? Or is my usage incorrect?

@Z-yq,
In order to reproduce the issue reported here, could you please provide the TensorFlow version, the complete code and the dataset you are using. Thanks!

@amahendrakar，
Thank you for your help.
The specific code is in my project. https://github.com/Z-yq/TensorflowASR
Here, I pick the important part.
Okay, I implemented the Transducer structure for speech recognition using the Python version of Tensorflow (2.2.0).
When doing inference, I used tf.while_loop to speed up decoding.
lstm init code:
        self.lstm_cells = []
        # lstms units must equal (for using beam search)
        for i in range(num_lstms):
            lstm = tf.keras.layers.LSTMCell(units=lstm_units,
                                            )
            self.lstm_cells.append(lstm)
        self.decoder_lstms = tf.keras.layers.RNN(
            self.lstm_cells, return_sequences=True, return_state=True)

And lstm's call function:
    def call(self,
             inputs,
             training=False,
             p_memory_states=None,
             **kwargs):

        outputs = self.embed(inputs, training=training)#Embedding
        outputs = self.do(outputs, training=training)#dropout
        if p_memory_states is None:  # Zeros mean no initial_state
            p_memory_states = self.get_initial_state(outputs)

        outputs = self.decoder_lstms(outputs, training=training,initial_state=p_memory_states)
        new_memory_states =outputs[1:]
        outputs = outputs[0]

        return outputs  , new_memory_states

Previously, in the main body of the loop, the LSTM state at the last prediction is passed in each time to optimize decoding speed.It can run successfully in Python.
body like this:

 def _body(enc, i, decoded, h,T):
            hi = tf.reshape(enc[i], [1, 1, -1])  # [1, 1, E]
            y ,h_= self.predict_net(
                inputs=tf.reshape(decoded[-1], [1, 1]),  # [1, 1]
                p_memory_states=h,
                training=False
            )
            ytu = tf.nn.log_softmax(self.joint_net([hi, y], training=False))
            ytu = tf.squeeze(ytu, axis=None)  # [1, 1, 1, V] => [V]
            n_predict = tf.argmax(ytu, axis=-1, output_type=tf.int32)  # => argmax []
            n_predict = tf.reshape(n_predict, [1])

            def return_no_blank():

                return [tf.concat([decoded, n_predict], axis=0),h_]

            decoded,h= tf.cond(
                n_predict != self.text_featurizer.blank and n_predict != 0,
                true_fn=return_no_blank,
                false_fn=lambda: [decoded,h]
            )

            return enc, i + 1, decoded,h, T

So I freeze the model to PB file and make use of Python's function calls.It works well.
The following error occurred when making a call using TensorflowC (2.3.0) that I downloaded the compiled library from the official.
2020-10-29 17:19:01.449470: E tensorflow/core/framework/tensor.cc:555] Could not decode variant with type_name: "tensorflow::TensorList".  Perhaps you forgot to register a decoder via REGISTER_UNARY_VARIANT_DECODE_FUNCTION?
2020-10-29 17:19:01.451491: W tensorflow/core/framework/op_kernel.cc:1744] OP_REQUIRES failed at constant_op.cc:82 : Invalid argument: Cannot parse tensor from tensor_proto.
2020-10-29 17:19:02.134350: E tensorflow/core/framework/tensor.cc:555] Could not decode variant with type_name: "tensorflow::TensorList".  Perhaps you forgot to register a decoder via REGISTER_UNARY_VARIANT_DECODE_FUNCTION?
2020-10-29 17:19:02.194184: W tensorflow/core/framework/op_kernel.cc:1744] OP_REQUIRES failed at constant_op.cc:82 : Invalid argument: Cannot parse tensor from proto: dtype: DT_VARIANT
tensor_shape {
}
variant_val {
  type_name: "tensorflow::TensorList"
  metadata: "\001\000\001\377\377\377\377\377\377\377\377\377\001\022\002\010\001\022\003\010\300\002"
}

Cannot parse tensor from proto: dtype: DT_VARIANT
tensor_shape {
}
variant_val {
  type_name: "tensorflow::TensorList"
  metadata: "\001\000\001\377\377\377\377\377\377\377\377\377\001\022\002\010\001\022\003\010\300\002"
}

terminate called after throwing an instance of 'std::runtime_error'
  what():  Cannot parse tensor from proto: dtype: DT_VARIANT
tensor_shape {
}
variant_val {
  type_name: "tensorflow::TensorList"
  metadata: "\001\000\001\377\377\377\377\377\377\377\377\377\001\022\002\010\001\022\003\010\300\002"
}

         [[{{node StatefulPartitionedCall/StatefulPartitionedCall/while/body/_1143/StatefulPartitionedCall/while/body/_4035/ConformerTransducer_prediction/rnn/TensorArrayV2_1/_0__cf__0}}]]

Then I tried some ways to solve this problem.
A feasible method：
```
def _body(enc, i, decoded, T):
hi = tf.reshape(enc[i], [1, 1, -1])  # [1, 1, E]
y = self.predict_net(
inputs=tf.reshape(decoded, [1, -1]),  # [1, 1]
p_memory_states=None,
training=False
)
y = y[:, -1:]
# [1, 1, P], [1, P], [1, P]
# [1, 1, E] + [1, 1, P] => [1, 1, 1, V]
ytu = tf.nn.log_softmax(self.joint_net([hi, y], training=False))# self.joint_net is a Dense
ytu = tf.squeeze(ytu, axis=None)  # [1, 1, 1, V] => [V]
n_predict = tf.argmax(ytu, axis=-1, output_type=tf.int32)  # => argmax []
n_predict = tf.reshape(n_predict, [1])
        def return_no_blank():
            return tf.concat([decoded, n_predict], axis=0)

        decoded = tf.cond(
            n_predict != self.text_featurizer.blank and n_predict != 0,
            true_fn=return_no_blank,
            false_fn=lambda: decoded
        )

        return enc, i + 1, decoded, T

Cancel the input and return of the LSTM's states, and input the whole decoded value .But in this case, it will be slow when there are many decoded values.
I still want to use the way of input and return LSTM's states, so I come here for help.

Could not decode variant with type_name: "tensorflow::TensorList".  Perhaps you forgot to register a decoder via REGISTER_UNARY_VARIANT_DECODE_FUNCTION?
This looks like something is missing/not supported

Currently I can only find operations about LIST in RNN's initial_state ,and I'm a little confused：
In the case of no input and output states, the state of LSTM is also obtained by assignment in the built-in function.
But when this state is entered and returned as a parameter, it will fail.Even if I use tf.stack to make states as a tensor, that's the same.
So I think there may be a problem with the operation of the state of lstm.Or there are problems with the freeze graph API.

@Z-yq,

The specific code is in my project. https://github.com/Z-yq/TensorflowASR

The example you've provided is fairly complex and it would be difficult for us to pinpoint the issue. Can you remove the dependencies and get the example down to the simplest possible repro? That will allow us to determine the source of the error easily. Thanks!

@mihaimaruseac ，
Thanks for your help,
I've streamlined the code and there should be only tensorflow dependencies.  here:https://github.com/Z-yq/rnn-debug
the python/C script will recongnize test.wav . And python needs a librosa dependency only for read wav file.Using wave library is also possible.
In the python script, the example is called and converted to a pb file(The work has been completed in with_state/without_state ).
In CppInference folder,Two versions have been compiled：cppinference_with_state/cppinference_without_state, and cppinference_without_state will success.

I'm facing the same problem.
2020-11-13 11:19:55.600020: E tensorflow/core/framework/tensor.cc:555] Could not decode variant with type_name: "tensorflow::TensorList".  Perhaps you forgot to register a decoder via REGISTER_UNARY_VARIANT_DECODE_FUNCTION?
2020-11-13 11:19:55.600058: W tensorflow/core/framework/op_kernel.cc:1744] OP_REQUIRES failed at constant_op.cc:82 : Invalid argument: Cannot parse tensor from tensor_proto.
2020-11-13 11:19:55.616901: E tensorflow/core/framework/tensor.cc:555] Could not decode variant with type_name: "tensorflow::TensorList".  Perhaps you forgot to register a decoder via REGISTER_UNARY_VARIANT_DECODE_FUNCTION?
2020-11-13 11:19:55.617324: W tensorflow/core/framework/op_kernel.cc:1744] OP_REQUIRES failed at constant_op.cc:82 : Invalid argument: Cannot parse tensor from proto: dtype: DT_VARIANT
tensor_shape {
}
variant_val {
  type_name: "tensorflow::TensorList"
  metadata: "\000\004\377\377\377\377\377\377\377\377\377\001\022\013\010\377\377\377\377\377\377\377\377\377\001\022\003\010\264\001\022\003\010\264\001\022\002\010\003"
}

err=Run(): Cannot parse tensor from proto: dtype: DT_VARIANT
tensor_shape {
}
variant_val {
  type_name: "tensorflow::TensorList"
  metadata: "\000\004\377\377\377\377\377\377\377\377\377\001\022\013\010\377\377\377\377\377\377\377\377\377\001\022\003\010\264\001\022\003\010\264\001\022\002\010\003"
}

	 [[{{node StatefulPartitionedCall/StatefulPartitionedCall/TensorArrayV2/_0__cf__0}}]]

With the help of git bisect I was able to isolate breaking commit (I've found that it was broken somewhere between v2.1.1 and v2.2.0-rc0).
It has shown that the problem was caused by changes made in https://github.com/tensorflow/tensorflow/commit/de37b1eaca05431822223e5c996bc08245cf523b.
Moreover, I was able to work it around by partially reverting the commit (I've undone changed made in variant_op_registry.h and variant_op_registry.cc): https://github.com/bayandin/tensorflow/commit/2c64044fafde13490903eb6775088f0fb1986af9.
I've applied it on the top on v2.4.0-rc1 and it works fine for me.
I'm not familiar enough with this part of the codebase and not completely sure if these won't cause any regressions in other parts, but if it's ok I'll be happy to submit a PR with the fix.

here's my git bisect log, if you're interested.
$ git bisect log
git bisect start
# bad: [3c1e8c03419266bb6ba379d303d3e03a380617a8] Merge pull request #37486 from tensorflow/mm-r2.2-debug-win-build
git bisect bad 3c1e8c03419266bb6ba379d303d3e03a380617a8
# good: [54daf3c5700897a6062313983933ca28e92c410d] Pin scipy to 1.4.1.
git bisect good 54daf3c5700897a6062313983933ca28e92c410d
# bad: [579287b839d410b859efce68cb093995f27ef13c] Try to fix spacing/formatting in returns section of Keras dataset docs.
git bisect bad 579287b839d410b859efce68cb093995f27ef13c
# good: [085db02feb6065d5de4b426c8d530b4178b50061] Merge pull request #35686 from samikama:GBBPFix
git bisect good 085db02feb6065d5de4b426c8d530b4178b50061
# bad: [ca87dcae5568d1b44016be46ebfd8b4972a6959b] Go: Update generated wrapper functions for TensorFlow ops.
git bisect bad ca87dcae5568d1b44016be46ebfd8b4972a6959b
# bad: [6ab77f3be330714746054eb678c9c4116f300692] Add complex number support for tf.extract_image_patches
git bisect bad 6ab77f3be330714746054eb678c9c4116f300692
# bad: [335b20de311d9aeb3a2aaf13527c65f0fab5540e] Add a missing ProtobufStringToString.
git bisect bad 335b20de311d9aeb3a2aaf13527c65f0fab5540e
# good: [7e17fdff2868df65139f5f0eb79bca40b6bbc041] server_lib.create_local_server() should create TF server with job_name=localhost
git bisect good 7e17fdff2868df65139f5f0eb79bca40b6bbc041
# good: [0fd0b290824ce4d1dad23c3e050f2d859d2b845b] Adds an example showing a fault-tolerant custom training loop with MultiWorkerMirroredStrategy.
git bisect good 0fd0b290824ce4d1dad23c3e050f2d859d2b845b
# good: [0fb8ffc796fe6ce191eacb6654ab394449bec065] Add quantized types support for Arg min/max in TF Lite dialect
git bisect good 0fb8ffc796fe6ce191eacb6654ab394449bec065
# bad: [de37b1eaca05431822223e5c996bc08245cf523b] Refactor the mobile/portable/android/ios targets to use filegroups where possible.
git bisect bad de37b1eaca05431822223e5c996bc08245cf523b
# good: [20d5c6a05070d0ca20eb1f38877b91cc95b841f1] Add new Adafruit board to supported devices
git bisect good 20d5c6a05070d0ca20eb1f38877b91cc95b841f1
# good: [fcef1fc494fc9410362c3659b62928b88079bef1] [MLIR:TF/XLA] Handle function argument aliasing in side-effect analysis.
git bisect good fcef1fc494fc9410362c3659b62928b88079bef1
# good: [fd2cd3e10e799c1b99018ff82cc5fa32016726af] Add an XSpace to TraceEvents converter and unittest
git bisect good fd2cd3e10e799c1b99018ff82cc5fa32016726af
# first bad commit: [de37b1eaca05431822223e5c996bc08245cf523b] Refactor the mobile/portable/android/ios targets to use filegroups where possible.

I'm facing the same problem, but I can't solve it making the RNN stateless anyway. When I load the model and I try to feed it, I get the very same error
2020-11-14 20:59:37.164271: E tensorflow/core/framework/tensor.cc:555] Could not decode variant with type_name: "tensorflow::TensorList".  Perhaps you forgot to register a decoder via REGISTER_UNARY_VARIANT_DECODE_FUNCTION?
2020-11-14 20:59:37.164517: W tensorflow/core/framework/op_kernel.cc:1744] OP_REQUIRES failed at constant_op.cc:81 : Invalid argument: Cannot parse tensor from proto: dtype: DT_VARIANT
tensor_shape {
}
variant_val {
  type_name: "tensorflow::TensorList"
  metadata: "\000\003\377\377\377\377\377\377\377\377\377\001"
}

panic: {{function_node __forward_standard_gru_2677_specialized_for_StatefulPartitionedCall_1_StatefulPartitionedCall_sequential_gru_PartitionedCall_at_tf_graph}} {{function_node __forward_standard_gru_2677_specialized_for_StatefulPartitionedCall_1_StatefulPartitionedCall_sequential_gru_PartitionedCall_at_tf_graph}} Cannot parse tensor from proto: dtype: DT_VARIANT
tensor_shape {
}
variant_val {
  type_name: "tensorflow::TensorList"
  metadata: "\000\003\377\377\377\377\377\377\377\377\377\001"
}

         [[{{node while/Placeholder_0/accumulator/_0__cf__0}}]]
         [[StatefulPartitionedCall_1/StatefulPartitionedCall/sequential/gru/PartitionedCall]]

goroutine 1 [running]:
github.com/galeone/tfgo.(*Model).Exec(0xc000010030, 0xc00015e4f0, 0x1, 0x1, 0xc00015e520, 0x0, 0x80, 0x80)
        /home/paolo/projects/go/src/github.com/galeone/tfgo/model.go:73 +0xce
main.learn(0xc000010030, 0xc000cc30c0, 0xc000cc30e0, 0x0)
        /home/paolo/projects/galeone.github.io/code/all-in-graph/wat.go:85 +0x2e7
main.Learn(0xc000010030, 0x21, 0xc000c95fd8, 0x7, 0x1, 0x0, 0x2cb7d702cef0, 0x84f780, 0x414ae382bf31d3c7, 0x3f01030e, ...)
        /home/paolo/projects/galeone.github.io/code/all-in-graph/wat.go:115 +0x274
main.main()
        /home/paolo/projects/galeone.github.io/code/all-in-graph/wat.go:238 +0x6f0

@bayandin have you compiled the C version for Linux or macOS? with or without GPU support? It would be helpful if you can share your built library (I'm looking for the Linux version without GPU support)

Moreover, I was able to work it around by partially reverting the commit (I've undone changed made in variant_op_registry.h and variant_op_registry.cc): bayandin@2c64044.

@Z-yq,
Please take a look at @bayandin's comment and let us know if you are still facing the same issue. Thanks!

@bayandin have you compiled the C version for Linux or macOS? with or without GPU support? It would be helpful if you can share your built library (I'm looking for the Linux version without GPU support)

Initially, I tested my patch on macOS (CPU) and it worked fine.
But it turns out that the patch didn't fix the issue on my Linux setup (neither on CPU nor on GPU).

@bayandin,Thank you for your help.
@amahendrakar ,

Moreover, I was able to work it around by partially reverting the commit (I've undone changed made in variant_op_registry.h and variant_op_registry.cc): bayandin@2c64044.

I tried the method, but still failed.
My environment is ：
Ubuntu 18.04
gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0

And CPU info：
processor       : 7
vendor_id       : GenuineIntel
cpu family      : 6
model           : 142
model name      : Intel(R) Core(TM) i7-10610U CPU @ 1.80GHz
stepping        : 12
cpu MHz         : 2304.000

Is there any news? This bug is making the deployment of RNNs impossible...

Please check if this patch against 2.4.1 helps, there are 2 bugs with static/dynamic loading of TF libraries and this works it around
diff --git a/tensorflow/core/framework/type_index.h b/tensorflow/core/framework/type_index.h
index 7986904dd7a..5519df474d0 100644
--- a/tensorflow/core/framework/type_index.h
+++ b/tensorflow/core/framework/type_index.h
@@ -24,9 +24,10 @@ limitations under the License.

 #include "tensorflow/core/platform/types.h"

-#if defined(MACOS) || defined(TARGET_OS_MAC)
+#define USE_NAME_BASED_RTTI_ONLY
+#if defined(MACOS) || defined(TARGET_OS_MAC) || defined(USE_NAME_BASED_RTTI_ONLY)
 #include "tensorflow/core/platform/hash.h"
-#endif  // defined(MACOS) || defined(TARGET_OS_MAC)
+#endif  // defined(MACOS) || defined(TARGET_OS_MAC) || defined(USE_NAME_BASED_RTTI_ONLY)

 namespace tensorflow {

@@ -62,7 +63,7 @@ class TypeIndex {

 #if defined(__GXX_RTTI) || defined(_CPPRTTI)

-#if defined(MACOS) || defined(TARGET_OS_MAC)
+#if defined(MACOS) || defined(TARGET_OS_MAC) || defined(USE_NAME_BASED_RTTI_ONLY)
     // Use a hash based on the type name to avoid issues due to RTLD_LOCAL on
     // MacOS (b/156979412).
     return TypeIndex(Hash64(typeid(T).name()), typeid(T).name());
diff --git a/tensorflow/core/framework/variant_op_registry.cc b/tensorflow/core/framework/variant_op_registry.cc
index aa3bdeab5e2..069542d784a 100644
--- a/tensorflow/core/framework/variant_op_registry.cc
+++ b/tensorflow/core/framework/variant_op_registry.cc
@@ -26,6 +26,16 @@ limitations under the License.

 namespace tensorflow {

+// Get a pointer to a global UnaryVariantOpRegistry object
+UnaryVariantOpRegistry* UnaryVariantOpRegistryGlobal() {
+  static UnaryVariantOpRegistry* global_unary_variant_op_registry = NULL;
+
+  if (global_unary_variant_op_registry == NULL) {
+    global_unary_variant_op_registry = new UnaryVariantOpRegistry;
+  }
+  return global_unary_variant_op_registry;
+}
+
 std::unordered_set<string>* UnaryVariantOpRegistry::PersistentStringStorage() {
   static std::unordered_set<string>* string_storage =
       new std::unordered_set<string>();
@@ -55,6 +65,13 @@ bool DecodeUnaryVariant(Variant* variant) {
   if (variant->TypeName().empty()) {
     VariantTensorDataProto* t = variant->get<VariantTensorDataProto>();
     if (t == nullptr || !t->metadata().empty() || !t->tensors().empty()) {
+      LOG(ERROR) << __PRETTY_FUNCTION__ << ": empty typename, malformed variant: t == nullptr: " << (t == nullptr) << ": " << variant->DebugString() << std::endl;
+      if (t != nullptr) {
+         LOG(ERROR) << __PRETTY_FUNCTION__ << ": empty typename, malformed variant: t != nullptr: !t->metadata().empty(): " << !t->metadata().empty()
+		 << ", !t->tensors().empty(): " << !t->tensors().empty()
+		 << ": " << variant->DebugString()
+		 << std::endl;
+      }
       // Malformed variant.
       return false;
     } else {
@@ -66,11 +83,15 @@ bool DecodeUnaryVariant(Variant* variant) {
   UnaryVariantOpRegistry::VariantDecodeFn* decode_fn =
       UnaryVariantOpRegistry::Global()->GetDecodeFn(variant->TypeName());
   if (decode_fn == nullptr) {
+    LOG(ERROR) << __PRETTY_FUNCTION__ << variant->TypeName() << ": decode_fn == nullptr: " << variant->DebugString() << std::endl;
     return false;
   }
   const string type_name = variant->TypeName();
   bool decoded = (*decode_fn)(variant);
-  if (!decoded) return false;
+  if (!decoded) {
+    LOG(ERROR) << type_name << " ->" << variant->TypeName() << ": not decoded:" << variant->DebugString() << std::endl;
+    return false;
+  }
   if (variant->TypeName() != type_name) {
     LOG(ERROR) << "DecodeUnaryVariant: Variant type_name before decoding was: "
                << type_name
diff --git a/tensorflow/core/framework/variant_op_registry.h b/tensorflow/core/framework/variant_op_registry.h
index edfb9c544c0..ed8bec6cc6e 100644
--- a/tensorflow/core/framework/variant_op_registry.h
+++ b/tensorflow/core/framework/variant_op_registry.h
@@ -56,6 +56,9 @@ enum VariantDeviceCopyDirection {
   DEVICE_TO_DEVICE = 3,
 };

+class UnaryVariantOpRegistry;
+extern UnaryVariantOpRegistry* UnaryVariantOpRegistryGlobal();
+
 class UnaryVariantOpRegistry {
  public:
   typedef std::function<bool(Variant*)> VariantDecodeFn;
@@ -170,9 +173,7 @@ class UnaryVariantOpRegistry {

   // Get a pointer to a global UnaryVariantOpRegistry object
   static UnaryVariantOpRegistry* Global() {
-    static UnaryVariantOpRegistry* global_unary_variant_op_registry =
-        new UnaryVariantOpRegistry;
-    return global_unary_variant_op_registry;
+    return UnaryVariantOpRegistryGlobal();
   }

   // Get a pointer to a global persistent string storage object.

Please check if this patch against 2.4.1 helps, there are 2 bugs with static/dynamic loading of TF libraries and this works it around
diff --git a/tensorflow/core/framework/type_index.h b/tensorflow/core/framework/type_index.h
index 7986904dd7a..5519df474d0 100644
--- a/tensorflow/core/framework/type_index.h
+++ b/tensorflow/core/framework/type_index.h
@@ -24,9 +24,10 @@ limitations under the License.

 #include "tensorflow/core/platform/types.h"

-#if defined(MACOS) || defined(TARGET_OS_MAC)
+#define USE_NAME_BASED_RTTI_ONLY
+#if defined(MACOS) || defined(TARGET_OS_MAC) || defined(USE_NAME_BASED_RTTI_ONLY)
 #include "tensorflow/core/platform/hash.h"
-#endif  // defined(MACOS) || defined(TARGET_OS_MAC)
+#endif  // defined(MACOS) || defined(TARGET_OS_MAC) || defined(USE_NAME_BASED_RTTI_ONLY)

 namespace tensorflow {

@@ -62,7 +63,7 @@ class TypeIndex {

 #if defined(__GXX_RTTI) || defined(_CPPRTTI)

-#if defined(MACOS) || defined(TARGET_OS_MAC)
+#if defined(MACOS) || defined(TARGET_OS_MAC) || defined(USE_NAME_BASED_RTTI_ONLY)
     // Use a hash based on the type name to avoid issues due to RTLD_LOCAL on
     // MacOS (b/156979412).
     return TypeIndex(Hash64(typeid(T).name()), typeid(T).name());
diff --git a/tensorflow/core/framework/variant_op_registry.cc b/tensorflow/core/framework/variant_op_registry.cc
index aa3bdeab5e2..069542d784a 100644
--- a/tensorflow/core/framework/variant_op_registry.cc
+++ b/tensorflow/core/framework/variant_op_registry.cc
@@ -26,6 +26,16 @@ limitations under the License.

 namespace tensorflow {

+// Get a pointer to a global UnaryVariantOpRegistry object
+UnaryVariantOpRegistry* UnaryVariantOpRegistryGlobal() {
+  static UnaryVariantOpRegistry* global_unary_variant_op_registry = NULL;
+
+  if (global_unary_variant_op_registry == NULL) {
+    global_unary_variant_op_registry = new UnaryVariantOpRegistry;
+  }
+  return global_unary_variant_op_registry;
+}
+
 std::unordered_set<string>* UnaryVariantOpRegistry::PersistentStringStorage() {
   static std::unordered_set<string>* string_storage =
       new std::unordered_set<string>();
@@ -55,6 +65,13 @@ bool DecodeUnaryVariant(Variant* variant) {
   if (variant->TypeName().empty()) {
     VariantTensorDataProto* t = variant->get<VariantTensorDataProto>();
     if (t == nullptr || !t->metadata().empty() || !t->tensors().empty()) {
+      LOG(ERROR) << __PRETTY_FUNCTION__ << ": empty typename, malformed variant: t == nullptr: " << (t == nullptr) << ": " << variant->DebugString() << std::endl;
+      if (t != nullptr) {
+         LOG(ERROR) << __PRETTY_FUNCTION__ << ": empty typename, malformed variant: t != nullptr: !t->metadata().empty(): " << !t->metadata().empty()
+		 << ", !t->tensors().empty(): " << !t->tensors().empty()
+		 << ": " << variant->DebugString()
+		 << std::endl;
+      }
       // Malformed variant.
       return false;
     } else {
@@ -66,11 +83,15 @@ bool DecodeUnaryVariant(Variant* variant) {
   UnaryVariantOpRegistry::VariantDecodeFn* decode_fn =
       UnaryVariantOpRegistry::Global()->GetDecodeFn(variant->TypeName());
   if (decode_fn == nullptr) {
+    LOG(ERROR) << __PRETTY_FUNCTION__ << variant->TypeName() << ": decode_fn == nullptr: " << variant->DebugString() << std::endl;
     return false;
   }
   const string type_name = variant->TypeName();
   bool decoded = (*decode_fn)(variant);
-  if (!decoded) return false;
+  if (!decoded) {
+    LOG(ERROR) << type_name << " ->" << variant->TypeName() << ": not decoded:" << variant->DebugString() << std::endl;
+    return false;
+  }
   if (variant->TypeName() != type_name) {
     LOG(ERROR) << "DecodeUnaryVariant: Variant type_name before decoding was: "
                << type_name
diff --git a/tensorflow/core/framework/variant_op_registry.h b/tensorflow/core/framework/variant_op_registry.h
index edfb9c544c0..ed8bec6cc6e 100644
--- a/tensorflow/core/framework/variant_op_registry.h
+++ b/tensorflow/core/framework/variant_op_registry.h
@@ -56,6 +56,9 @@ enum VariantDeviceCopyDirection {
   DEVICE_TO_DEVICE = 3,
 };

+class UnaryVariantOpRegistry;
+extern UnaryVariantOpRegistry* UnaryVariantOpRegistryGlobal();
+
 class UnaryVariantOpRegistry {
  public:
   typedef std::function<bool(Variant*)> VariantDecodeFn;
@@ -170,9 +173,7 @@ class UnaryVariantOpRegistry {

   // Get a pointer to a global UnaryVariantOpRegistry object
   static UnaryVariantOpRegistry* Global() {
-    static UnaryVariantOpRegistry* global_unary_variant_op_registry =
-        new UnaryVariantOpRegistry;
-    return global_unary_variant_op_registry;
+    return UnaryVariantOpRegistryGlobal();
   }

   // Get a pointer to a global persistent string storage object.

Btw, assigning TF 2.3 tag shoud presumably imply that it has been fixed in 2.4 or it is not relevant anymore, but no, this bug is still present in 2.4.1.
In fact, there are 2 bugs.
For some inference models (maybe when there are 2 or more saved models in one global inference context) some TF libraries got loaded twice and some global variables got overwritten.
The second bug was introduced in https://github.com/tensorflow/tensorflow/commit/de37b1eaca05431822223e5c996bc08245cf523b as found by Alexander Bayandin above - TF statically loads a bunch of TF_VARIANT decoders, but then it dynamically loads the same (or some) libraries and overwrites global lists, this bug actually existed forever, but no one uses TF_VARIANT to store bool or int32, so those got overwritten (actually, lost), but no one cares. But variant decoder for TensorList got lost and reverting de37b1eaca05431822223e5c996bc08245cf523b should have fixed that. I have fixed this bug by explicitly calling single static function defined not in a header, but in C code.
But there is the first bug introduced in 15275d3a14c77e2244ae1155f93243256f08e3ed - again, because of the "second" library loading RTTI changes for every class for every new dynamic library loading, thus TensorList got different RTTI ID and decoder in variant->get<VariantTensorDataProto>() refuses to decode protobuf. You have fixed this for MacOS by reverting to the old behaviour, now my patch forces this for everyone else. This first bug ends up with the debug message printed above.
Or these two bugs can be different issue with TF loading some of its modules/libraries/anything multiple time for the inference environment with multiple saved_model in them.
This patch fixes problem for me with different models on CPU and GPU on linux. I will wait for some time for others to test and/or confirm whether it works or not and then will make a proper pull request.

Btw, assigning TF 2.3 tag shoud presumably imply that it has been fixed in 2.4 or it is not relevant anymore, but no, this bug is still present in 2.4.1.
In fact, there are 2 bugs.
For some inference models (maybe when there are 2 or more saved models in one global inference context) some TF libraries got loaded twice and some global variables got overwritten.
The second bug was introduced in https://github.com/tensorflow/tensorflow/commit/de37b1eaca05431822223e5c996bc08245cf523b as found by Alexander Bayandin above - TF statically loads a bunch of TF_VARIANT decoders, but then it dynamically loads the same (or some) libraries and overwrites global lists, this bug actually existed forever, but no one uses TF_VARIANT to store bool or int32, so those got overwritten (actually, lost), but no one cares. But variant decoder for TensorList got lost and reverting de37b1eaca05431822223e5c996bc08245cf523b should have fixed that. I have fixed this bug by explicitly calling single static function defined not in a header, but in C code.
But there is the first bug introduced in 15275d3a14c77e2244ae1155f93243256f08e3ed - again, because of the "second" library loading RTTI changes for every class for every new dynamic library loading, thus TensorList got different RTTI ID and decoder in variant->get<VariantTensorDataProto>() refuses to decode protobuf. You have fixed this for MacOS by reverting to the old behaviour, now my patch forces this for everyone else. This first bug ends up with the debug message printed above.
Or these two bugs can be different issue with TF loading some of its modules/libraries/anything multiple time for the inference environment with multiple saved_model in them.
This patch fixes problem for me with different models on CPU and GPU on linux. I will wait for some time for others to test and/or confirm whether it works or not and then will make a proper pull request.

This patch fixes problem for me with different models on CPU and GPU on linux. I will wait for some time for others to test and/or confirm whether it works or not and then will make a proper pull request.

The patch helped for my case on macOS CPU
Thanks!

This patch fixes problem for me with different models on CPU and GPU on linux. I will wait for some time for others to test and/or confirm whether it works or not and then will make a proper pull request.

The patch helped for my case on macOS CPU
Thanks!

This patch also fixed the same error for me.  Linux assorted (ancient) cpus, V100 + P6000 gpus.

I can see there's a PR with this patch: https://github.com/tensorflow/tensorflow/pull/47072

Please check if this patch against 2.4.1 helps, there are 2 bugs with static/dynamic loading of TF libraries and this works it around
diff --git a/tensorflow/core/framework/type_index.h b/tensorflow/core/framework/type_index.h
index 7986904dd7a..5519df474d0 100644
--- a/tensorflow/core/framework/type_index.h
+++ b/tensorflow/core/framework/type_index.h
@@ -24,9 +24,10 @@ limitations under the License.

 #include "tensorflow/core/platform/types.h"

-#if defined(MACOS) || defined(TARGET_OS_MAC)
+#define USE_NAME_BASED_RTTI_ONLY
+#if defined(MACOS) || defined(TARGET_OS_MAC) || defined(USE_NAME_BASED_RTTI_ONLY)
 #include "tensorflow/core/platform/hash.h"
-#endif  // defined(MACOS) || defined(TARGET_OS_MAC)
+#endif  // defined(MACOS) || defined(TARGET_OS_MAC) || defined(USE_NAME_BASED_RTTI_ONLY)

 namespace tensorflow {

@@ -62,7 +63,7 @@ class TypeIndex {

 #if defined(__GXX_RTTI) || defined(_CPPRTTI)

-#if defined(MACOS) || defined(TARGET_OS_MAC)
+#if defined(MACOS) || defined(TARGET_OS_MAC) || defined(USE_NAME_BASED_RTTI_ONLY)
     // Use a hash based on the type name to avoid issues due to RTLD_LOCAL on
     // MacOS (b/156979412).
     return TypeIndex(Hash64(typeid(T).name()), typeid(T).name());
diff --git a/tensorflow/core/framework/variant_op_registry.cc b/tensorflow/core/framework/variant_op_registry.cc
index aa3bdeab5e2..069542d784a 100644
--- a/tensorflow/core/framework/variant_op_registry.cc
+++ b/tensorflow/core/framework/variant_op_registry.cc
@@ -26,6 +26,16 @@ limitations under the License.

 namespace tensorflow {

+// Get a pointer to a global UnaryVariantOpRegistry object
+UnaryVariantOpRegistry* UnaryVariantOpRegistryGlobal() {
+  static UnaryVariantOpRegistry* global_unary_variant_op_registry = NULL;
+
+  if (global_unary_variant_op_registry == NULL) {
+    global_unary_variant_op_registry = new UnaryVariantOpRegistry;
+  }
+  return global_unary_variant_op_registry;
+}
+
 std::unordered_set<string>* UnaryVariantOpRegistry::PersistentStringStorage() {
   static std::unordered_set<string>* string_storage =
       new std::unordered_set<string>();
@@ -55,6 +65,13 @@ bool DecodeUnaryVariant(Variant* variant) {
   if (variant->TypeName().empty()) {
     VariantTensorDataProto* t = variant->get<VariantTensorDataProto>();
     if (t == nullptr || !t->metadata().empty() || !t->tensors().empty()) {
+      LOG(ERROR) << __PRETTY_FUNCTION__ << ": empty typename, malformed variant: t == nullptr: " << (t == nullptr) << ": " << variant->DebugString() << std::endl;
+      if (t != nullptr) {
+         LOG(ERROR) << __PRETTY_FUNCTION__ << ": empty typename, malformed variant: t != nullptr: !t->metadata().empty(): " << !t->metadata().empty()
+		 << ", !t->tensors().empty(): " << !t->tensors().empty()
+		 << ": " << variant->DebugString()
+		 << std::endl;
+      }
       // Malformed variant.
       return false;
     } else {
@@ -66,11 +83,15 @@ bool DecodeUnaryVariant(Variant* variant) {
   UnaryVariantOpRegistry::VariantDecodeFn* decode_fn =
       UnaryVariantOpRegistry::Global()->GetDecodeFn(variant->TypeName());
   if (decode_fn == nullptr) {
+    LOG(ERROR) << __PRETTY_FUNCTION__ << variant->TypeName() << ": decode_fn == nullptr: " << variant->DebugString() << std::endl;
     return false;
   }
   const string type_name = variant->TypeName();
   bool decoded = (*decode_fn)(variant);
-  if (!decoded) return false;
+  if (!decoded) {
+    LOG(ERROR) << type_name << " ->" << variant->TypeName() << ": not decoded:" << variant->DebugString() << std::endl;
+    return false;
+  }
   if (variant->TypeName() != type_name) {
     LOG(ERROR) << "DecodeUnaryVariant: Variant type_name before decoding was: "
                << type_name
diff --git a/tensorflow/core/framework/variant_op_registry.h b/tensorflow/core/framework/variant_op_registry.h
index edfb9c544c0..ed8bec6cc6e 100644
--- a/tensorflow/core/framework/variant_op_registry.h
+++ b/tensorflow/core/framework/variant_op_registry.h
@@ -56,6 +56,9 @@ enum VariantDeviceCopyDirection {
   DEVICE_TO_DEVICE = 3,
 };

+class UnaryVariantOpRegistry;
+extern UnaryVariantOpRegistry* UnaryVariantOpRegistryGlobal();
+
 class UnaryVariantOpRegistry {
  public:
   typedef std::function<bool(Variant*)> VariantDecodeFn;
@@ -170,9 +173,7 @@ class UnaryVariantOpRegistry {

   // Get a pointer to a global UnaryVariantOpRegistry object
   static UnaryVariantOpRegistry* Global() {
-    static UnaryVariantOpRegistry* global_unary_variant_op_registry =
-        new UnaryVariantOpRegistry;
-    return global_unary_variant_op_registry;
+    return UnaryVariantOpRegistryGlobal();
   }

   // Get a pointer to a global persistent string storage object.

Hi @bioothod. How can I apply these patch fixes?  Can you give me some directions?

Hi @schramm , you need to use patch tool to apply this diff to 2.4 tensorflow sources, and then to rebuild tensorflow. These bugfixes are alsow in master branch for the upcoming 2.6 version.

I have built the master brunch for the 2.6 version and it is working
fine now.
Thank you!
Em qui., 3 de jun. de 2021 às 13:48, Evgeniy Polyakov <
@.***> escreveu:

Hi @schramm https://github.com/schramm , you need to use patch tool to
apply this diff to 2.4 tensorflow sources, and then to rebuild tensorflow.
These bugfixes are alsow in master branch for the upcoming 2.6 version.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/44428#issuecomment-853808894,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACEAZ5USXXUG73FTPC65C3TQ5T2NANCNFSM4TDPFF7Q
.

--
Dr Rodrigo Schramm

@mihaimaruseac what do you mean that fix is available in 2.4.1 and 2.5? This fix has not been committed neither into 2.4 nor into 2.5 tree although was submitted, those branches currently have this bug, bugfix has only been pushed into the current master tree (future 2.6).

Sorry, I misunderstood. You are right.
If we want to patch this in older versions, we need to open PRs against those branches, but given the complexity of the branch and the current timelines, I think it's better to just wait for 2.6 release in a few weeks.

I have a similar issue and tried everything suggested under this issue and it didn't work.
I created a GitHub issue
https://github.com/tensorflow/tensorflow/issues/51052

@mihaimaruseac, would it be conceivable to backport the fix to TF 2.4? Later versions of TensorFlow require updating CUDA from 11.0 to 11.2, which complicates deployments.

2.4 series will get EOLed in December, so probably will get at most one single patch release until then.
If you create a PR on the branch, we can try merging it in and running tests on it, but if there are failures we won't be able to fully patch this. Given the large refactoring that occurred since, there is a potential that even if our tests works the patch might break for other users.

Hi,
Thank you for opening this issue. Since this issue has been open for a long time, the code/debug information for this issue may not be relevant with the current state of the code base.
The Tensorflow team is constantly improving the framework by fixing bugs and adding new features. We suggest you try the latest TensorFlow version with the latest compatible hardware configuration which could potentially resolve the issue. If you are still facing the issue, please create a new GitHub issue with your latest findings, with all the debugging information which could help us investigate.
Please follow the release notes to stay up to date with the latest developments which are happening in the Tensorflow space.


Adding C++20 barrier

adding hpx/barrier.hpp that imports lcos::local::cpp20_barrier into namespace hpx

You've been busy, thanks! Do you know if we have any internal uses of barrier that should be replaced by the new one?

I wanted to go through our code in a second sweep to replace thing where possible.

This should be good to go now.


Build fails with [gcc 12.1.0], [OTP 24.3.2]

See https://github.com/emqx/emqx/pull/8340

It may fixed by #35.
Now I can successfully build using v0.3.5

@JimMoen, good. I'm closing this now.


Error lightclientUpdater.onFinalized

Describe the bug
Got this while syncing oonoonba network:
Jun-07 10:33:48.782 [CHAIN]           error: Error lightclientUpdater.onFinalized epoch=515 message=REGEN_ERROR_BLOCK_NOT_IN_FORKCHOICE
Error: REGEN_ERROR_BLOCK_NOT_IN_FORKCHOICE
    at StateRegenerator.getBlockSlotState (/Users/tuyennguyen/Projects/workshop/lodestar/packages/lodestar/src/chain/regen/regen.ts:92:13)
    at StateRegenerator.getCheckpointState (/Users/tuyennguyen/Projects/workshop/lodestar/packages/lodestar/src/chain/regen/regen.ts:86:23)
    at Object.job (/Users/tuyennguyen/Projects/workshop/lodestar/packages/lodestar/src/chain/regen/queued.ts:91:66)
    at Timeout._onTimeout (/Users/tuyennguyen/Projects/workshop/lodestar/packages/lodestar/src/util/queue/index.ts:106:32)
    at listOnTimeout (internal/timers.js:554:17)
    at processTimers (internal/timers.js:497:7)

Expected behavior
No error

another error
Jun-07 12:43:21.502 [CHAIN]           error: Error lightclientUpdater.onFinalized epoch=488 message=No block found for checkpoint 488 : 0xad0da9ee63bf4aca2ebccf8f21525e31f6b65659de12cc0d35909ccd94a33b88
Error: No block found for checkpoint 488 : 0xad0da9ee63bf4aca2ebccf8f21525e31f6b65659de12cc0d35909ccd94a33b88
    at BeaconChain.onFinalized (/root/workspace/node/lodestar/packages/lodestar/src/chain/eventHandlers.ts:185:15)
    at ChainEventEmitter.<anonymous> (/root/workspace/node/lodestar/packages/lodestar/src/chain/eventHandlers.ts:33:7)

maybe this is the root cause #2783

the onFinalized handler try to get block by skipped slot
Jul-07 03:32:49.057 [CHAIN]           error: Error lightclientUpdater.onFinalized epoch=4475 message=No block found for checkpoint 4475 : 0xbfd44f561487162039156ddf83bf69379908881c6d6d0126abba287481202a5c
Error: No block found for checkpoint 4475 : 0xbfd44f561487162039156ddf83bf69379908881c6d6d0126abba287481202a5c
    at BeaconChain.onFinalized (/usr/app/packages/lodestar/src/chain/eventHandlers.ts:179:15)
    at runMicrotasks (<anonymous>)
    at runNextTicks (internal/process/task_queues.js:58:5)
    at listOnTimeout (internal/timers.js:523:9)
    at processTimers (internal/timers.js:497:7)
    at ChainEventEmitter.<anonymous> (/usr/app/packages/lodestar/src/chain/eventHandlers.ts:33:7)

start slot of epoch 4475 is slot 143200, it shows in the validator log that it's a skipped slot
Jul-07 03:20:04.040 []                error: Error on runSyncCommitteeTasks slot=143200 message=Error producing SyncCommitteeMessage - Not Found: No block found for id '143200'
Error: Error producing SyncCommitteeMessage - Not Found: No block found for id '143200'
    at HttpClient.request (/usr/app/packages/api/src/client/utils/httpClient.ts:85:15)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at HttpClient.json (/usr/app/packages/api/src/client/utils/httpClient.ts:57:12)
    at Object.request [as getBlockRoot] (/usr/app/packages/api/src/client/utils/client.ts:71:19)
    at SyncCommitteeService.produceAndPublishSyncCommittees (/usr/app/packages/validator/src/services/syncCommittee.ts:93:29)
    at /usr/app/packages/validator/src/services/syncCommittee.ts:51:31
    at Clock.runAtMostEvery (/usr/app/packages/validator/src/util/clock.ts:68:7)

also this checkpoint onFinalized is conflict to forkChoiceFinalized event and both seems to happen at the same time
Jul-07 11:43:16.660 [CHORES]           info: Start processing finalized checkpoint epoch=1080
Jul-07 11:43:16.666 [CHAIN]           error: Error lightclientUpdater.onFinalized epoch=1080 message=No block found for checkpoint 1080 : 0x14823228ba8041d0576bf07f3c6ac2233fd4363be6ab3aa79c2f30dd36654f28
Error: No block found for checkpoint 1080 : 0x14823228ba8041d0576bf07f3c6ac2233fd4363be6ab3aa79c2f30dd36654f28
    at BeaconChain.onFinalized (/Users/tuyennguyen/Projects/workshop/lodestar/packages/lodestar/src/chain/eventHandlers.ts:183:15)
    at ChainEventEmitter.<anonymous> (/Users/tuyennguyen/Projects/workshop/lodestar/packages/lodestar/src/chain/eventHandlers.ts:33:7)
Jul-07 11:43:16.764 [CHORES]           info: Finish processing finalized checkpoint epoch=1080


Use "Smalltalk makefiles"

This PR changes makefiles to use (shared) "smalltalk makefiles" 1
The aim is to move as much as possible to common repository that can
be shared rather than copy & paste code to all "our" projects and repositories.

LGTM.
It is important to underline that this PR is about splitting-off a common part of the Makefile, and NOT about how to properly manage dependencies of which we now have two kinds (Metacello and non-Metacello).


Check for redis connection on init

ping/pong

Actually I already reset! right after creation so this would be redundant


Support for sampleHint

The proposal is to add the sampling pushdown to the protocol.
Context
The /query endpoint reduces the number of files that are exposed and fetched by a client to reduce data transfers and to speed up data loading. The mechanisms described in the protocol are two:

Predicate Hints: Allows filtering which files are needed taking advantage of the data layout, e.g., if data is partitioned, we may access only a few partitions.
Limit Hints: Thanks to the delta log, the backend knows how many records each file contains and can return only the files needed to satisfy the limit.

Both are hints, meaning they may not be satisfied, and the client may need to apply the actions in-memory too.
Proposal
Another way to reduce the number of files to be read is by introducing the sampling pushdown. The idea is that if the client wants to access a sample of data, the sampling predicate is sent to the backend, who then filters which files need to be accessed.
My proposal is to extend the protocol with an optional predicate on the query endpoint called "sampleHint" which would look like this:
{
  "predicateHints": [
    "date >= '2021-01-01'",
    "date <= '2021-01-31'"
  ],
  "limitHint": 1000,
  "sampleHint": 0.30,
  "version": 123
}

The server would try to select the minimum amount of files to satisfy the request. The sampleHint would be a double in the range of 0 to 1.0 as it is commonly done in the ecosystem.
Order of operations
sampleHint takes precedence, then predicateHints, and finally limitHint should be applied.

@polsm91 Sorry for the late response.
So could you share a specific use case where limitHint would not work and sampleHint could help? I feel sampleHint could satisfy most cases?


Added python global list to collection item definition

Checklist

[x] I have added tests

How come this one is marked as draft PR?

No idea. Seems to be undeterministic if a created pr will be draft or review ready by default?

Can't say I've ever seen that before, although Refined GitHub does always show separate buttons when opening a PR.

Strange. I've never seen that behaviour either

It happens to me from time to time. Now idea why

How do you open your pull requests?

Usually by pushing the branch first and then clicking pull request => new on GitHub


error/Nothing happened when call AcquireTokenSilentAsync method[Office365]

I converted this into ASP.NET, and when it called below code(AcquireTokenSilentAsync) nothing happened. can anyone please help me to fix this issue ASAP.
`internal static async Task EnsureOutlookServicesClientCreatedAsync(string capabilityName)
{
var signInUserId = ClaimsPrincipal.Current.FindFirst(ClaimTypes.NameIdentifier).Value;
var userObjectId = ClaimsPrincipal.Current.FindFirst("http://schemas.microsoft.com/identity/claims/objectidentifier").Value;
var tenantId = ClaimsPrincipal.Current.FindFirst("http://schemas.microsoft.com/identity/claims/tenantid").Value;
    AuthenticationContext authContext = new AuthenticationContext(string.Format("{0}/{1}", SettingsHelper.AuthorizationUri, tenantId), new ADALTokenCache(signInUserId));

    try
    {
        DiscoveryClient discClient = new DiscoveryClient(SettingsHelper.DiscoveryServiceEndpointUri,
            async () =>
            {
                var authResult = await authContext.AcquireTokenSilentAsync(SettingsHelper.DiscoveryServiceResourceId, new ClientCredential(SettingsHelper.ClientId, SettingsHelper.AppKey), new UserIdentifier(userObjectId, UserIdentifierType.UniqueId));

                return authResult.AccessToken;
            });

        var dcr = await discClient.DiscoverCapabilityAsync("Contacts");

        return new OutlookServicesClient(dcr.ServiceEndpointUri,
            async () =>
            {
                var authResult = await authContext.AcquireTokenSilentAsync(dcr.ServiceResourceId, new ClientCredential(SettingsHelper.ClientId, SettingsHelper.AppKey), new UserIdentifier(userObjectId, UserIdentifierType.UniqueId));

                return authResult.AccessToken;
            });
    }
    catch (AdalException exception)
    {
        //Handle token acquisition failure
        if (exception.ErrorCode == AdalError.FailedToAcquireTokenSilently)
        {
            authContext.TokenCache.Clear();
            throw exception;
        }
        return null;
    }
    catch (Exception ex)
    {

        throw;
    }
}`

Your sample attachment doesn't contain a .csproj or solution file, so I can't run it. Is the call throwing an exception? If so, could you provide the details (including the inner exception)? Could you also run your code with the debugger and check the values that are being passed to the AcquireTokenSilentAsync method? If the call is going out and generating an exception, could you provide a Fiddler trace as well?

I did, it shows me object reference error. this is website so no need for .csproj.
can you open as website and check, I tried with all ways but no success.

I fixed it. and working now.


Error during initialization: invalid type: found map, expected a string for key "default.datasource.0" in bbox.toml TOML file

Thank you for providing BBOX!
Trying to setup bbox as a featureserver, I get
Error during initialization: invalid type: found map, expected a string for key "default.datasource.0" in bbox.toml TOML file
Any hints on how to tackle this?
I tried to create a minimal config, but never get a working WFS...
Would it be possible to provide some more full working configs?
The config details in the docs are too rudimentary (at least for me :-) )
Or at least some general idea of what the toml should look like, or why.
Is there supposed to be some 'default datasource'?
Is the second part of 'datasource.postgis' in the examples a 'type', or an 'id'?
etc
[webserver]
server_addr = "<IP_ADDRESS>:8080"  # Default: <IP_ADDRESS>:8080

[featureserver]

[[datasource]]
name = "test"

[[datasource.postgis]]
# export PGPASSWORD=test
url = "postgres://test@localhost:5432/test"

[[collection]]
name = "lastweek"
title = "lastweek"
description = "lastweek"
[collection.postgis]
datasource = "test"
table = "myschema.mytable"

datasource.postgis is not an array, so has only single brackets:
[[datasource]]
name = "test"

[datasource.postgis]
url = "postgres://test@localhost:5432/test"

See also the example at https://www.bbox.earth/feature-server/configuration.html
The error message is not good, but not easy to improve.


Allow scaling an application automatically based on scaling policy

Instead of defining an instance count, define some criteria to scale an application automatically.
Different policies should be implemented, so that users can define the appropriate set needed for their application.
The first policy to implement should be based on the mesos slave count.
This policy select all slaves in the cluster, that match the defined constraints of an application and takes that as instance count. Whenever the slaves change, the instance count is changed accordingly.

Hi @aquamatthias! Can you make it pluggable from the beggining? :) Auto-scaling can be based on multiple signal sources: current utilization, history of utilization, quick changes in traffic, predictions etc. Making it pluggable will allow for more generic implementations.

@wendigo definitively. The plan is to create this as a pluggable interface.

pluggable interface， how to archive it? where is found the design doc?

Hey @xiaods look here: https://mesosphere.github.io/marathon/docs/plugin.html.
There is currently no plugin interface for auto scaling - it is part of that story.

is the API for this going to be pluggable as well? for example (assuming this is baked into the app API):
/*app*/ {
  "id": "log-shipper",
  "cmd": "run-fluentd.sh"
  ...
  "plugins": [
    {
      "kind": "policy/scale",
      "id": "log-shipper-scaling",
      "provider": "daemonScaler"
    }
  ]
}

/*app*/ {
  "id": "my-webapp",
  "cmd": "run-app.sh"
  ...
  "plugins": [
    {
      "kind": "policy/scale",
      "id": "webapp-scaling",
      "provider": "horizontalScaler",
      "spec": { /* spec parsed by horizontalScaler plugin */
        "targetUtilization": {
          "cpu": 80.0 /* percentage */
        }
      }
    }
  ]
}

Note: This issue has been migrated to https://jira.mesosphere.com/browse/MARATHON-3108. For more information see https://groups.google.com/forum/#!topic/marathon-framework/khtvf-ifnp8.

Note: This issue has been migrated to https://jira.mesosphere.com/browse/MARATHON-3108. For more information see https://groups.google.com/forum/#!topic/marathon-framework/khtvf-ifnp8.


When running code in the editor i am getting the following error in the concole

When running code in the editor i am getting the following error.
NameError: name 'runfile' is not defined
after running
set SPYDER_DEBUG=3
spyder --show-console
sys.exit(spyder.app.start.main())
File "C:\Program Files\Anaconda3\lib\site-packages\spyder\app\start.py", line 141, in main
mainwindow.main()
File "C:\Program Files\Anaconda3\lib\site-packages\spyder\app\mainwindow.py", line 3023, in main
or options.optimize)
File "C:\Program Files\Anaconda3\lib\site-packages\spyder\utils\windows.py", line 33, in set_attached_console_visible
return bool(ShowWindow(console_window_handle, flag[state]))
KeyError: 3

Spyder Version:3.21
Python Version:3.6
Qt Version:3.52
PyQt Version:3.5
Operating system:win10

set SPYDER_DEBUG=3 was already fixed and it'll be part of Spyder 3.2.2.
Sorry for the inconvenience in the meantime.

ran set SPYDER_DEBUG=3 still getting the same error

I know, the fix hasn't been released yet to a public version. But this bug was already fixed here, in Github.

sorry thanks