Remove the term "SACA" from code assets

Benefit/Result/Outcome
So that assets deployed by the system are clarified as having broader applicability than SACA and SCCA.
Description
SACA is Microsoft's implementation of SCCA. Having SACA terminology applied to Mission LZ code assets and deployed Azure resources may appear to limit the applicability of them to people who are not required to implement SCCA. The controls defined by SCCA are sensible for anyone who is establishing a secure network enclave in Azure, but SACA and SCCA may not apply to everyone who wants network security.
Acceptance Criteria

SACA and SCCA terms are removed from code and deployed assets.

Out of Scope
We do not want to remove SCCA and SACA from the documentation. Mission LZ continues to focus on meeting the requirements of SCCA.

Need some primary clarification on this.
SACA terminology is present in the following files:
src/scripts/config/generate_config_file.sh:8
src/scripts/config/generate_config_file.sh:39
src/scripts/terraform/create_tfvars_from_config.sh:60
src/scripts/terraform/create_tfvars_from_config.sh:61
src/docs/management-groups.md:7
src/docs/management-groups.md:24
src/build/README.md:18
src/build/README.md:24
src/build/README.md:60
src/build/README.md:61

SCCA is in the following:
src/docs/scca.md:1
src/docs/scca.md:5
src/docs/scca.md:7
src/docs/scca.md:19

What would be an acceptable replacement for this term and which files should be exempt from the proposed change?


Fixes for clang

Checked without CUDA backend with:

clang version 5.0.1-4
clang version 6.0.0-1ubuntu2

Have we decided which versions of clang to support?

Note that it still fails to build with Apple clang. This conversion (to the return type defined by the signature) seems invalid. Could you take a look? https://github.com/chainer/chainer/blob/7f539ef53c3b2ff22e4203ea7f08250889e29545/chainerx_cc/chainerx/native/native_device/memory.cc#L18

Sorry forgot this comment. Could you address it? It LGTM otherwise. https://github.com/chainer/chainer/pull/5744#pullrequestreview-181162212

LGTM


Pin chef-ingredient to ~> 1.0

Description
chef-ingredient v2.0.0 introduces breaking changes, so this pins us to chef-ingredient ~> 1.0.
Check List

[ ] All tests pass. See https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/TESTING.MD
[x] New functionality includes testing.
[x] New functionality has been documented in the README if applicable
[x] All commits have been signed for the Developer Certificate of Origin. See https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/CONTRIBUTING.MD

Looks like tests, even on master, haven’t passed for some time. However, one of the tests passed, so that’s something, I guess?

I missed this PR, but thanks for putting it together. I've pined to ~1.1 on master now to ensure Chef 13 compatibility as well


Add Entity Slicing using Entity Manifests to Cedar

Description of changes
This PR allows users of Cedar to get an entity slice of an existing Entities store using the entity manifest.
Mainly this is useful for testing entity manifests. However, it may also be useful for users to load all entities in to memory, but only send a slice over the network to a server.
Issue #, if available
Checklist for requesting a review
The change in this PR is (choose one, and delete the other options):

[ ] A breaking change requiring a major version bump to cedar-policy (e.g., changes to the signature of an existing API).
[ ] A backwards-compatible change requiring a minor version bump to cedar-policy (e.g., addition of a new API).
[ ] A bug fix or other functionality change requiring a patch to cedar-policy.
[ ] A change "invisible" to users (e.g., documentation, changes to "internal" crates like cedar-policy-core, cedar-validator, etc.)
[x] A change (breaking or otherwise) that only impacts unreleased or experimental code.

I confirm that this PR (choose one, and delete the other options):

[x] Updates the "Unreleased" section of the CHANGELOG with a description of my change (required for major/minor version bumps).
[ ] Does not update the CHANGELOG because my change does not significantly impact released code.

I confirm that cedar-spec (choose one, and delete the other options):

[ ] Does not require updates because my change does not impact the Cedar formal model or DRT infrastructure.
[ ] Requires updates, and I have made / will make these updates myself. (Please include in your description a timeline or link to the relevant PR in cedar-spec, and how you have tested that your updates are correct.)
[x] Requires updates, but I do not plan to make them in the near future. (Make sure that your changes are hidden behind a feature flag to mark them as experimental.)
[ ] I'm not sure how my change impacts cedar-spec. (Post your PR anyways, and we'll discuss in the comments.)

This PR stacks on top of #1102
It looks like I can't select that PR as a base branch since it's from my fork of Cedar.

Looks like changing the API for cedar testing broke downstream dependencies build

Closed in favor of a single PR.


Can't import torchmaxflowcpp, DLL load failed while importing

Hi masadcv, great work with torch support on this algo. I do pip install and try run the demo, it shows 'DLL load failed while importing torchmaxflowcpp'.
I see there is a 'torchmaxflow.cpp' file, but i don't know how to connect python with cpp.

Hi @rayLemond ,
Many thanks for your interest.
The library should be automatically linked with the DLLs during installation. So it is likely that something is going wrong on your setup during installation.
Can you please fill out the following bug report template and provide any additional detail that you think could be relevant for checking what is going wrong here:
Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. iOS]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Open the demo project in PyCharm, then at the start of file './torchmaxflow/_ init _.py', the line 'import torchmaxflowcpp' reports bug 'No module name 'torchmaxflowcpp''.
Then i run 'setup.py', and the above bug is gone, and i can see the code of 'torchmaxflowcpp.py' file.
Then i run 'demo_maxflow.py' and got:
Traceback (most recent call last):
  File "C:\Codes\torchmaxflow\demo_maxflow.py", line 8, in <module>
    import torchmaxflow
  File "C:\Codes\torchmaxflow\torchmaxflow\__init__.py", line 59, in <module>
    import torchmaxflowcpp
ImportError: DLL load failed while importing torchmaxflowcpp:

OS: Windows 10 Professional (21H1)

Many thanks for providing further details.
I have looked into this and the cause is that current pre-compiled wheels for torchmaxflow are compiled using pytorch 1.10
You have pytorch 1.13 which is causing DLL link errors.
I will look into resolving this from package side.. however for now there are two workarounds that you could use to continue your work:

Downgrade to pytorch v1.10 using conda install pytorch==1.10.1 cpuonly -c pytorch - you can also change to GPU version if you plan to train neural networks on GPU... I added cpuonly because I see you already have cpuonly installation of pytorch.
Install by forcing package to build from source, you can do this using: pip install --no-binary torchmaxflow torchmaxflow --no-build-isolation

Please do let me know if you still experience issues with the proposed workarounds.
I will get back with a PR to fix the issue within pypi package manager for long term solution.... however currently I am on vacation for Christmas, so this will be sometime in new year...

Thank you for the comprehensive response, i do get the solutions.
Sorry for interrupting the holiday, no hurry at all, it is just a trial run.
Have fun, cheers!


SSZ encoding of list of booleans

The encoding (serialisation) fo list of booleans seem to produce

a unit32 with the size of the list
a sequence of Bytes each one encoding a bit in the list.

This is according to the implementation
https://github.com/apache/incubator-tuweni/blob/03c08eeeba108670938252fef1fee209bdf435fe/ssz/src/main/java/org/apache/tuweni/ssz/SSZ.java#L1221
and test
https://github.com/apache/incubator-tuweni/blob/03c08eeeba108670938252fef1fee209bdf435fe/ssz/src/test/java/org/apache/tuweni/ssz/BytesSSZWriterTest.java#L415
If this is correct, it does not seem to correctly implement the format of the Eth2.0 spec https://github.com/ethereum/eth2.0-specs/blob/dev/ssz/simple-serialize.md#bitlistn
which specifies that the bits should be put together in Bytes (which seems to a good idea as produces more compact encodings).
Is that going to be problem when communicating with clients that do implement the spec?

The current SSZ implementation fell behind the spec. It needs to update to the latest eth2 tests and adapt.
I recently added the eth2 spec tests as a submodule to allow for this, but haven't pushed for running the tests yet.

Thanks @atoulme
I am interested in getting the test results when it is set up to run.

I think this might be covered by https://github.com/apache/incubator-tuweni/pull/487 - closing. Please reopen if more work is needed.


Remove existing accessory from cache

Great template. But there is no function that removes existing accessories from the cache when they are no longer in the setup. Currently, there are only two options: get from cache or add new ones.

Here is example: https://github.com/mkz212/homebridge-virtual-device/blob/master/src/platform.ts

Hey @mkz212,
Will this work for you?
https://github.com/uswong/homebridge-plugin-template/blob/latest/src/platform.ts
The lines I have added to remove those existing accessories not restored from cache are as follow:
https://github.com/uswong/homebridge-plugin-template/commit/8358c8cb7f454a759569959782121171d3d4ade6

Hey @mkz212,
Will this work for you? https://github.com/uswong/homebridge-plugin-template/blob/latest/src/platform.ts
The lines I have added to remove those existing accessories not restored from cache are as follow: uswong@8358c8c

For me it seems to be ok, but @bwp91 should take a look, he maintain this and knows it best.

i just updated the template with some example code of doing this


telemetry_int early 2023.2 release by Oct 2023

2023.2 is expected to finish by December 2023.2, but telemetry_int needs to be released individually by Oct 2023 at the latest, so I'll capture on this issue just so it shows up in the epic items explicitly.

dev/demo branch is available for demo usage. As of now intra and inter EVPLs can be enabled/disabled and data plane is working as expected. EPLs should work too, as long as table 2 on your switch support the matches.
A release won't be published in Oct since there's still many major parts to be delivered on epic_telemetry_int_v1 but branch dev/demo at least can be used in the meantime. I'll go ahead and close this issue.


Add additional tests to DEV

Problem
Closes #184
Solution
Added additional tests in regards to encoding/decoding of different transaction types. (DEV branch does not have the econdig/decoding problem of the master branch: #179, #173 )

This PR adds additional tests and also adds additional features. It would be better to split them up, so that each individual problem can be focussed on and merged separately. Let's first add additional tests, then merge, then we can add each functionality in separate merges.

Travis is still not working correctly.

I think you need to merge the latest dev branch

Strange, the merge seems to be fine, but Travis still complaint.

You overwrote the base58 checkDecode and CheckEncode in in the dev branch with the one in your branch. It it one of those functions that is giving an error. Although I think travis keeps failing for Pull Request builds nonetheless.
Is the CheckDecode and CheckEncode in dev different to yours in functionality? I believe that they were not passing tests, which is why they were initially commented out

I just moved the methods CheckDecode and CheckEncode from the master to the dev to make my tests pass. To be honest I didn't look at their functionality. Let me replace them with yours to see if my tests are still passing. If the tests pass I will commit them. If not I will investigate further.

@decentralisedkev ,
Are you sure that the latest dev branch has the above mentioned methods?(CheckDecode and CheckEncode). I don't find them here

So sorry, I made a mistake when I copied it. Can you pull again?

Codecov Report

Merging #183 into dev will decrease coverage by 1.04%.
The diff coverage is 0%.

@@            Coverage Diff            @@
##              dev    #183      +/-   ##
=========================================
- Coverage   56.15%   55.1%   -1.05%
=========================================
  Files          56      56
  Lines        2014    2016       +2
=========================================
- Hits         1131    1111      -20
- Misses        802     828      +26
+ Partials       81      77       -4

Impacted Files
Coverage Δ

pkg/wire/util/uint160.go
46.34% <0%> (-2.38%)
:arrow_down:

pkg/peer/peerhandshake.go
38.66% <0%> (-9.34%)
:arrow_down:

pkg/peer/peer.go
26.76% <0%> (-7.58%)
:arrow_down:

pkg/crypto/publickey/publickey.go
47.36% <0%> (-2.64%)
:arrow_down:

pkg/wire/payload/transaction/invocation.go
86.66% <0%> (+13.33%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update de31371...6ca6362. Read the comment docs.

Woohh! @decentralisedkev! I see you did some magic! :smile: . Nice job!

@decentralisedkev ,
Shall we close this PR as it does not add so much value?

Yep, you can open up another issue, if you feel like something should be added from this PR


FRAME_END: pprof interpretation

I am running pprof's heap profiler. I see the following:
Total: 86.1 MB
66.1  76.7%  76.7%     86.1 100.0% __FRAME_END__
2.8   3.3%  80.0%      2.8   3.3% std::vector::_M_realloc_insert
.....
What does '__FRAME_END__' mean?
Help would be much appreciated.

We are using whatever symbol name addr2line will give us. If your compiler or compiler tool toolchain somehow produces such "odd" symbol name infos then we may end up with something like this. I don't really know what FRAME_END is.


Add 120hz

Just a simple feature request please if you don't have time point me where i can add it

There is already a option that works for that feature.
set maximumRefreshRate option to 120.


Adds Explore CC Menu (CC Global Header)

Fixes

Fixes #68  by @adidevs
Description

This PR adds Explore CC Menu (CC Global Header) to the  CC Resource Archive
Technical details

The code follows the below mentioned specifications:

The Global Header has been implemented using Vocabulary as visual reference from CC Open Source Website
The script for toggling the menu has been added to the _layouts pages to keep the page source clean
The CSS for the Global Header has been marked with comments and is distinguishable from the original code for maintainability
CC Logo and CC-Heart icons have been taken from the CC Downloads page and added to a new assets folder inside the images folder
The Donate button on the CC Open Source site is inactive so the link has been taken from the one used in CC GitHub Repositories (https://creativecommons.org/donate)
The CC License Chooser link now redirects directly to Chooser Beta as the prompt requests on Chooser

Screenshots

https://user-images.githubusercontent.com/99022811/228337274-f1e8d380-1802-47b0-ab06-4e0ae8d2b90d.mp4
Checklist

[x] My pull request has a descriptive title (not a vague title like Update index.md).
[x] My pull request targets the default branch of the repository (main or master).
[x] My commit messages follow best practices.
[x] My code follows the established code style of the repository.
[x] I added or updated tests for the changes I made (if applicable).
[x] I added or updated documentation (if applicable).
[x] I tried running the project locally and verified that there are no
visible errors.

Developer Certificate of Origin

For the purposes of this DCO, "license" is equivalent to "license or public domain dedication," and "open source license" is equivalent to "open content license or public domain dedication."

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

@adidevs Nice work 🚀

@adidevs This works, we might want to eventually upgrade it match the newer header on search.creativecommons.org. But this is closer to current, so I'm gonna say Approved.


usage of identity in cross-validation

I am getting the following error by setting the aggregator option to opt.cross_validation.identity
---------------------------------------------------------------------------
     33 # Define Parameter Tuning
---> 34 optimal_pars_clf_sgd, _, _ = opt.maximize(clf_sgd_cv, num_evals=n_hyperparams_evals, alpha=[0.001, .1], l1_ratio=[0., 1.])
     35
     36 # Train model on the Inner Training Set with Tuned Hyperparameters

../local/lib/python2.7/site-packages/optunity/api.pyc in maximize(f, num_evals, solver_name, pmap, **kwargs)
    179     solver = make_solver(**suggestion)
    180     solution, details = optimize(solver, f, maximize=True, max_evals=num_evals,
--> 181                                  pmap=pmap)
    182     return solution, details, suggestion
    183

../local/lib/python2.7/site-packages/optunity/api.pyc in optimize(solver, func, maximize, max_evals, pmap)
    243     time = timeit.default_timer()
    244     try:
--> 245         solution, report = solver.optimize(f, maximize, pmap=pmap)
    246     except fun.MaximumEvaluationsException:
    247         # early stopping because maximum number of evaluations is reached

../local/lib/python2.7/site-packages/optunity/solvers/ParticleSwarm.pyc in optimize(self, f, maximize, pmap)
    257             fitnesses = pmap(evaluate, list(map(self.particle2dict, pop)))
    258             for part, fitness in zip(pop, fitnesses):
--> 259                 part.fitness = fit*fitness
    260                 if not part.best or part.best_fitness < part.fitness:
    261                     part.best = part.position

TypeError: can't multiply sequence by non-int of type 'float'

Here is my code
import optunity as opt
from optunity.metrics import _recall, contingency_table
from sklearn.linear_model import SGDClassifier
import numpy as np

n_in = 1
k_in = 2
n_hyperparams_evals = 10

clf_sgd = SGDClassifier(
            penalty="elasticnet",
            shuffle=True,
            n_iter=500,
            fit_intercept=True,
            learning_rate="optimal")

# Define Inner CV
cv_decorator = opt.cross_validated(x=X, y=Y.values,
                                   num_folds=k_in, num_iter=n_in,
                                   strata=[Y[Y==1].index.values],
                                   regenerate_folds=True,
                                   aggregator=opt.cross_validation.identity)

def obj_fun_clf_sgd(x_train, y_train, x_test, y_test, alpha, l1_ratio):
    model = clf_sgd.set_params(l1_ratio=l1_ratio, alpha=alpha).fit(x_train, y_train)
    y_pred = model.predict(x_test)
    score = _recall(contingency_table(y_test,y_pred))
    return score

clf_sgd_cv = cv_decorator(obj_fun_clf_sgd)

# Define Parameter Tuning
optimal_pars_clf_sgd, _, _ = opt.maximize(clf_sgd_cv, num_evals=n_hyperparams_evals, alpha=[0.001, .1], l1_ratio=[0., 1.])

# Train model on the Inner Training Set with Tuned Hyperparameters
optimal_model_clf_sgd = clf_sgd.set_params(**optimal_pars_clf_sgd).fit(X, Y.values)

The objective is to keep track of all the scores from the various folds. Is it a bug? or am I using incorrectly the API ?
Thanks in advance

All solvers we use expect to get a single number for each evaluation of an objective function (which is constructed via cross-validation in your case). The identity aggregator constructs a list of all results, which causes this error.
The identity aggregator is mainly intended for other uses than you have now, for instance in the outer cross-validation procedure of a nested cross-validation setup.
At this point I am not entirely sure what your intentions are: when you are optimizing hyperparameters you really need to return a single number for every set of hyperparameters that is tried. Note that you already get a trace of all objective function evaluations (that is the overall cross-validation score) in the return values of maximize, minimize and optimize.
If you want to retain scores for each fold, you will need to modify optunity internally. I can make a quick code example if you like.

I'd like to keep track of the scores obtained during the model selection in order to estimate the variance of the scores as a function of the hyper-parameters values.
I suppose this could be done by building an ad-hoc scorer, and then use the agregator "list_mean"
However, even like that, I would get only num_evals data points, i.e. the number of iterations configured in the "optimizer". I'd like anyway to get at least the points for each CV iteration.
So, yes, if it is not a problem for you, some lines of code would be welcome.

You can now freely return all fold-wise information during cross-validation. Solvers now have an extra layer between objective function evaluations and actually using the return value. If your objective function returns an indexable, it's first element is used in optimization.
You can see an example at in bin/examples/python/advanced_cv.py, which returns the full cross-validation results. If this is not what you were looking for, please do let me know so I can reopen this issue.
PS: you will need to update optunity from git to get the new feature.

thanks for the update. Now it even works with a self-provided aggregator. Here is the one I have defined FYI. It computes mean,std,min and max for a list of list of scores, and provides as first element, the score to be used by the optimizer.
´´´
def score_aggregators(list_of_scores,list_of_agg_fun=[np.mean,np.std,np.min,np.max]):
try:
scores = zip(*list_of_scores)
agg_scores = list(map(agg_func,scores) for agg_func in list_of_agg_fun)
return (agg_scores[0][0],list(map(agg_func,scores) for agg_func in list_of_agg_fun))
except:
agg_scores =  [agg_func(list_of_scores) for agg_func in list_of_agg_fun]
return [agg_func(list_of_scores) for agg_func in list_of_agg_fun]
pass
´´´

Glad to hear it is working! The current solution allows pretty much every use case I can think of, but feel free to let us know if there are still any remaining issues.


WIP: auto download subtitles from providers like opensubtitles

This PR adds the possibility to download subtitles automatically from providers like opensubtitles.org.
I wanted to add this feature because I liked Kodi  subtitle auto download feature and found it very useful.
Currently the user can click on "auto download" and the code will download the 3 most popular subs based on the current playing file. The subs are stored in the same folder as the playing file, prefixed with "auto_downloaded". Every time a sub is downloaded, the controller add's it in the player.
If no subs are found, an error message is displayed, and for now the only provider implemented is opensubtitles.
The next steps would be to :

create an ui or dialog to allow the user to choose which subs he wants to download
when no subs are found, allow the user to enter other information as the imdbid for instance.
get user main language or allow him to choose it

What to you think about this feature ?
This is my first open source contribution so don't hesitate to tell me if I'm doing something wrong, I will appreciate it and improve myself  :).

I miss this feature a lot.

Hi @Diogo-Ferreira!
I added this feature on #677.
But the legality for this feature is not very clear and the team decided to stay legal, which I believe is the best way to move on (remember PopcornTime?).
My advice is to put effort in a plugin system, which will enable us to add this feature as an external plugin, which would be awesome ;)
I'm also looking forward to work on a plugin system soon.
Thanks for contributing! :)

I'll close this PR because of the same conclusion on #677.
And again, I really like this idea and would love to see it implemented as a plugin.
Cheers!

I'll close this PR because of the same conclusion on #677.
And again, I really like this idea and would love to see it implemented as a plugin.
Cheers!

Hi @codealchemist ,
I understand the problem, thank you for your time anyway.
I think too that a plugin system would be awesome, I look forward to seeing it, and to contribute to it if I can help :)
Cheers.


Fix pytest-notebook job

The ci-tests.yml and notebooks.yml Actions are currently not stable:
https://github.com/ComPWA/compwa-org/actions/runs/1881099598
The reason is that each notebook installs its own pinned dependencies, which somehow results in a ContextualVersionConflict. Ideally, these jobs should run over each notebook, install the dependencies, and run them without problems.

Error seems to originate specifically from phasespace. See for instance this log, where only the notebook that requires phasespace fails.


docs: update copyright header

Codecov Report

Merging #314 (8c83e58) into master (3ef41a6) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #314   +/-   ##
=======================================
  Coverage   26.45%   26.45%
=======================================
  Files          56       56
  Lines        3534     3534
  Branches      641      641
=======================================
  Hits          935      935
  Misses       2378     2378
  Partials      221      221

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3ef41a6...8c83e58. Read the comment docs.


web hot-restart bug report

Happened when I tried to hot-restart.
command
flutter run -d chrome
exception
ChromeDebugException: Unexpected error from chrome devtools:
text: Uncaught
exception:
description: ReferenceError: require is not defined
at :1:1
type: object
value: null
attempted JS eval: require('dart_sdk').dart.getLibraries();
#0      handleErrorIfPresent (package:dwds/src/services/chrome_proxy_service.dart:569:5)
#1      AppInspector._getLibraryRefs (package:dwds/src/debugging/inspector.dart:489:5)
<asynchronous suspension>
#2      AppInspector._initialize (package:dwds/src/debugging/inspector.dart:72:36)
<asynchronous suspension>
#3      AppInspector.initialize (package:dwds/src/debugging/inspector.dart:108:21)
#4      _AsyncAwaitCompleter.start (dart:async-patch/async_patch.dart:43:6)
#5      AppInspector.initialize (package:dwds/src/debugging/inspector.dart:84:41)
#6      ChromeProxyService.createIsolate (package:dwds/src/services/chrome_proxy_service.dart:112:37)
#7      _AsyncAwaitCompleter.start (dart:async-patch/async_patch.dart:43:6)
#8      ChromeProxyService.createIsolate (package:dwds/src/services/chrome_proxy_service.dart:106:29)
#9      DevHandler._handleConnection.<anonymous closure> (package:dwds/src/handlers/dev_handler.dart:259:15)
#10     _asyncThenWrapperHelper.<anonymous closure> (dart:async-patch/async_patch.dart:71:64)
#11     _rootRunUnary (dart:async/zone.dart:1132:38)
#12     _CustomZone.runUnary (dart:async/zone.dart:1029:19)
#13     _FutureListener.handleValue (dart:async/future_impl.dart:137:18)
#14     Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:678:45)
#15     Future._propagateToListeners (dart:async/future_impl.dart:707:32)
#16     Future._addListener.<anonymous closure> (dart:async/future_impl.dart:387:9)
#17     _rootRun (dart:async/zone.dart:1124:13)
#18     _CustomZone.run (dart:async/zone.dart:1021:19)
#19     _CustomZone.runGuarded (dart:async/zone.dart:923:7)
#20     _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:963:23)
#21     _microtaskLoop (dart:async/schedule_microtask.dart:41:21)
#22     _startMicrotaskLoop (dart:async/schedule_microtask.dart:50:5)
#23     _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:116:13)
#24     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:173:5)

flutter doctor
[✓] Flutter (Channel dev, v1.10.2, on Linux, locale en_US.UTF-8)
    • Flutter version 1.10.2 at /home/ehsan/FlutterSDK
    • Framework revision f5733f7a62 (4 days ago), 2019-09-11 21:03:42 -0700
    • Engine revision 7ea9884ab0
    • Dart version 2.5.0 (build 2.5.0-dev.4.0 be66176534)

[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
    • Android SDK at /home/ehsan/AndroidSDK
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-28, build-tools 28.0.3
    • ANDROID_HOME = /home/ehsan/AndroidSDK
    • Java binary at: /home/ehsan/.local/share/JetBrains/Toolbox/apps/AndroidStudio/ch-0/183.5522156/jre/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)
    • All Android licenses accepted.

[✓] Chrome - develop for the web
    • CHROME_EXECUTABLE = /snap/bin/chromium

[✓] Android Studio (version 3.5)
    • Android Studio at /home/ehsan/.local/share/JetBrains/Toolbox/apps/AndroidStudio/ch-0/183.5522156
    • Flutter plugin version 36.1.1
    • Dart plugin version 183.6270
    • Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)

[✓] VS Code (version 1.38.1)
    • VS Code at /usr/share/code
    • Flutter extension version 3.4.1

[✓] Connected device (2 available)
    • Chrome • chrome • web-javascript • Chromium 77.0.3865.75 snap
    • Server • web    • web-javascript • Flutter Tools

• No issues found!

Hello @a-h-mzd !
Thank you for posting an issue.
It seems like this behaviour was previously reported as well: #39246. When solution will be found - it will be posted on that issue.
As there is a report about the same problem, closing this thread for now.
Thank you.


Replace the Host Storage Manager

Not everything is complete, and there's no upgrade path from host -> host2. So it's going into staging instead of master.
We should merge the wallet branch into staging as well. Then we can all be working on the same base as we charge towards v1.1.

uh oh

Fixed - was just a problem with the staging branch being really out of date

As discussed, I would like to see the test files compressed and extracted at test time, or perhaps even generated at test time. If possible, their entropy should be low so that they compress well. Right now they are cluttering the diff (can't even load the table of contents)

No appveyor build?
But, should be good to go.  Hopefully forgivable as it's basically just spellchecking.

To get appveyor to build I think you just need to add it to the appveyor.yml. We should definitely do that because otherwise we have little means to test windows compatibility

I can't figure out why travis is stuck? The Makefile won't complete?

yeah I think that happens when you PR into a branch that isn't up to date with master. I updated the staging branch so it should work now.

the 300 files are still in the git history because they were added and deleted in separate commits. We should rebase this branch to remove the commit where they were added. I can handle this if you want, but if you have unpushed changes to this branch you should probably do it yourself

As long as the build passes I'm all set. I've screwed up the git here a couple of time, I'll let you handle pulling out the 300 files that we don't need.


Cannot compile after atom 1.7.x update

My platform is windows 10 with atom 1.7.2.
Platform: win32; texpath:
Processing file foo.tex (foo) in directory E:\documents\foo
texify builder
texify -b -p --engine=pdftex --tex-option="--synctex=1" "foo.tex"
Parsing E:\documents\foo\foo.log
Could not read log file!
The problem is that no 'foo.log' is generated. If I compile this tex file outside atom with miktex, I can get the foo.log file and build with latextools in atom will not complain about the log file. However, building with latextools will not update the log file even if the tex file is modified. Thus, no change for the corresponding pdf file.
Everything works fine in previous atom 1.6.2 or in miktex.

Is your E: a network drive or otherwise resolvable to a UNC path? This might be getting bit by the same thing as #81...

E: is on my local hard disk.
I guess it's the problem of atom itself. Another latex package atom-latex also breaks with the same issue after the update of atom.

When compiling outside of Atom, do you use the command above or another command? Does anything show up in any of the MiKTeX logs (usually in %AppData%\MiKTeX\2.9\logs)?

I have the same question as chenyuewu. But I compile *.tex with texlive. It mess me up, and I can't find the answer in Google.

I have reinstalled texlive(2016 replaced 2015), then the problem was solved...

@blizzardwj: that seems like it might've been an altogether different issue.

I have the same question.
My platform is ubuntu 16.04 with atom 1.11.2. texlive2016
Platform: linux; texpath: $PATH:/usr/texbin
Setting PATH = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:$PATH:/usr/texbin:$PATH:/usr/texbin:$PATH:/usr/texbin
Processing file test_en.tex (test_en) in directory /home/agamemnon/workspace
latexmk builder
latexmk -cd -e -f -pdf -interaction=nonstopmode -synctex=1 "test_en.tex"
Parsing /home/agamemnon/workspace/test_en.log
Could not read log file!
but i can compile the same file successfully out side atom.
besides, it seems that latexmk generates no log file within atom

@agamemnon314: Again that looks like a different issue. In particular, your texpath setting looks like it contains $PATH:/usr/texbin:$PATH:/usr/texbin:$PATH:/usr/texbin. $PATH isn't (currently) supported in the texpath setting. Try adjusting that and see if it fixes things.

I've had the same issue. Setting either the TeX environment in the path or the texpath in the package settings in atom solved it.
If you launch MikTeX's mo_admin.exe on Windows, you'll see a respective warning, if the path is not set correctly.

@elmewo and @ig0774, I have the same issue but in Ubuntu. Do you know how to make it works. You talked about changing the Path. Could you please show me more? Thank you.

@minhducpsc I was getting the same error and then realized I hadn't installed latexmk.  After installing latexmk it works fine without having changed the default path.


Change the Computing-Oviedo.xml name to something representative.

Change the name of Computing_Oviedo.xml to something representative like occ-java-eclipse-style.xml in order to improve the productivity of the repository when more languages were added.
Link to file: [standards/Computing_Oviedo.xml] (https://github.com/computer-engineering-uniovi/standards/blob/master/Computing_Oviedo.xml)

By Monday 19, naming system will be improved.


Cannot open Mvc.sln in visual studio 2019 community edition

Cloned https://github.com/aspnet/AspNetCore.git from github.com and when tried to run Restore from Command prompt, getting the following error
C:\AspNetCore.tools\msbuild\16.0.0-alpha\tools\MSBuild\Microsoft\VC\v160\Microsoft.Cpp.Default.props" was not found. Confirm that the path in the  declaration is correct, and that the file exists on disk
Tried to open Mvc.sln in visual studio 2019 community edition, but none of the projects in the solution are loaded.
I would appreciate if anyone knows how to resolve this issue.

@AVin9 have you had a look at the build from source instructions here: https://github.com/aspnet/AspNetCore/blob/master/docs/BuildFromSource.md?

Thanks for responding to this issue.
Followed the instruction in the link.
Everything worked fine until startvs.cmd command in the link (Restore.cmd worked fine without any problems)
But running startvs.cmd Mvc.Sln  in the command prompt didn't open the visual studio solution. Instead it opens another command prompt  with the physical path of Mvc.Sln as the title bar of the new command prompt window.
Tried to open Mvc.sln in visual studio 2019 community edition directly, the solution loaded without any error.
But building the solution gave the error  'The types in 'Microsoft.AspNetCore.Components.RenderTree' are not recommended for use outside of the Blazor framework. These  type definitions will change in future releases.
Does anyone know how to resolve this issue?

I am also trying to build the project. I cloned and when I tried to run .\restore.cmd, I got following error:
E:\OpenSource\AspNetCore\src\Servers\IIS\AspNetCoreModuleV2\AspNetCore\AspNetCore.vcxproj(31,3): error MSB4019: The imported project "E:\OpenSource\AspNetCore.tools\msbuild\16.3.0-alpha\tools\MSBuild\Microsoft\VC\v160\Microsoft.Cpp.Default.props" was not found. Confirm that the path in the  declaration is correct, and that the file exists on disk.
There were many, but all related to the same path ".tools\msbuild\16.3.0-alpha\tools\MSBuild\Microsoft\VC\v160". I checked this path in the downloaded repo, and it not there, so it's not downloaded by the restore command.
Did anyone resolve this issue yet?
.

I also ran into this issue

The error message usually indicates that you don't have the C++ workloads installed in VS. Portions of ASP.NET Core, such as MVC, Blazor, SignalR etc, do not require ANCM to build and their solutions can be opened without having the workloads installed \ ANCM built. However if you do want to get clean builds, the instructions https://github.com/aspnet/AspNetCore/blob/master/docs/BuildFromSource.md include a command for installing VS with the required workloads.

Thanks, @pranavkm , VS 2019 Community was already installed on my system, but I updated it using below command:
PS> ./eng/scripts/InstallVisualStudio.ps1 -Edition Community
And after that .\restore.cmd run successfully.


Get Flannel to build on windows and add placeholders

Description
As a first step towards adding windows support in Flannel, get it to build!
Also add placeholders for where we need Windows implementations
Commented out all backends except vxlan and hostgw
Todos

[ ] Need to run Tests to see if it works (fails) as expected

Aargh - did not mean to raise a PR on coreos just yet!


Take screenshot example AttachmentTypes is not clear where is applicable

Steps to reproduce
Given a user opens the documentation of screenpy_selenium
When the user tries to replicate the example
the_actor.attempts_to(
    SaveScreenshot.as_(filepath).and_attach_it_with(
        attachment_type=AttachmentTypes.PNG,
    ),
)

Expected result
The snippet works as expected
Actual Result
The test fails because AttachmentTypes.PNG does not exists
After opening the example screenpy_selenium/github I realized that the expected import is:
from allure_commons.types import AttachmentType

But is not clear in the given example that this only works if I'm using the screenpy_adapter_allure package

Oh! Thank you for this—this example is actually from before the extensions were split out. I'll move this example to ScreenPy Adapter: Allure's documentation!

The docs have been updated! Sorry for the two-month-long turnaround!


Type inference of the type union and intersection

TypeScript Version:  2.7.2
Search Terms: Type union, type intersection, type inference
Code
interface A {
    a: boolean
}

interface A1 extends A {
    a1: boolean
}

interface A2 extends A {
    a2: boolean
}

type A1andA2 = A1 & A2

type A1orA2 = A1 | A2

type A1orA2orA1andA2 = A1orA2 | A1andA2

// Expected
const test1: A1orA2 = {
    a: true,
    a1: true,
}

console.log(test1) // Type inference: A1

// Expected
const test2: A1orA2 = {
    a: true,
    a2: true,
}

console.log(test2) // Type inference: A2

// Expected
const test3: A1orA2orA1andA2 = {
    a: true,
    a1: true,
    a2: true,
}

console.log(test3) // Type inference: A1andA2 <<< good

// Expected => union accept the intersections
const test4: A1orA2 = {
    a: true,
    a1: true,
    a2: true,
}

console.log(test4) // Type inference: A1 | A2 <<< Should be A1 & A2, not good
console.log(test4.a1) // Error
console.log(test4.a2) // Error

It is not very clear in the documentation if the union contains the intersection, but both assumptions seems to have issues
Expected behavior:

Assumption: union contains the intersection
A1 | A2 equals A1 or A2 or A1 & A2: the type intefence of { a: true, a1: true, a2: true } should be A1 & A2
Assumption: union excludes the intersection
A1 | A2 equals A1 or A2 excluding A1 & A2: { a: true, a1: true, a2: true } should not be assignable to A1 | A2

Actual behavior:

Assumption: union contains the intersection
The type inference of { a: true, a1: true, a2: true } is A1 | A2 and we can not use the properties a1 and a2 without type guards
Assumption: union excludes the intersection
{ a: true, a1: true, a2: true } is still assignable to A1 | A2

Playground Link:
https://www.typescriptlang.org/play/index.html#src=interface A {
    a%3A boolean
}

interface A1 extends A {
    a1%3A boolean
}

interface A2 extends A {
    a2%3A boolean
}

type A1andA2 %3D A1 %26 A2

type A1orA2 %3D A1 | A2

type A1orA2orA1andA2 %3D A1orA2 | A1andA2

%2F%2F Expected
const test1%3A A1orA2 %3D {
    a%3A true%2C
    a1%3A true%2C
}

console.log(test1) %2F%2F Type inference%3A A1

%2F%2F Expected
const test2%3A A1orA2 %3D {
    a%3A true%2C
    a2%3A true%2C
}

console.log(test2) %2F%2F Type inference%3A A2

%2F%2F Expected
const test3%3A A1orA2orA1andA2 %3D {
    a%3A true%2C
    a1%3A true%2C
    a2%3A true%2C
}

console.log(test3) %2F%2F Type inference%3A A1andA2 <<< good

%2F%2F Expected %3D> union accept the intersections
const test4%3A A1orA2 %3D {
    a%3A true%2C
    a1%3A true%2C
    a2%3A true%2C
}

console.log(test4) %2F%2F Type inference%3A A1 | A2 <<< Should be A1 %26 A2%2C not good
console.log(test4.a1) %2F%2F Error
console.log(test4.a2) %2F%2F Error

Union includes the intersection, otherwise you would get weird behaviour such as 0 not being assignable to 0 | number. The operator behaves much the same as the set operation.
This issue seems to be similar to #20863.
I wouldn't expect the inferred type of test4 to be A1 & A2 because of the structural nature of TypeScript. Partitioning the inferred type into A1 and A2 is arbitrary and it would be equally valid to infer {a: boolean; a1: boolean; a2; boolean}. You get this behaviour for test3 because you include it explicitly in the union type.


Allow to pass into CMAKE flags when building BoringSSL

I would love to be able to use -DOPENSSL_C11_ATOMIC when building the vendored BoringSSL lib as it can make a lot of difference especially when you do a lot of handshakes. Basically I would love to do the same as we did in netty-tcnative:
https://github.com/netty/netty-tcnative/pull/568
I know I could just build my own BoringSSL lib and link against it but this sounds like something that could be easily supported without the need to build BoringSSL myself.

/cc @davidben @LPardue

This would also be super useful to set things like:
-DCMAKE_OSX_DEPLOYMENT_TARGET=10.9

For now I just define all of this via environment variables, which works but feels a bit hacky.

I think you can build BoringSSL alone, and set QUICHE_BSSL_PATH=/path/to/your/boringssl
Take a look my script
https://github.com/cntrump/build_curl/blob/main/build_quiche.sh

I know... It just felt a bit of overkill for such a thing.. Let me just close this for now as there is a way to archive it.


Restart of terminal causes release pipeline to fail

Agent Version and Platform
Agent 2.136.1
Windows 7 embedded
VSTS Type and Version
grovergaming.VisualStudio.com
What's not working?
As part of our release process for one of our on-premise devices deploying to, the final step is to restart the device.  On all previous versions of the deployment group agent (2.134.2 for example) this task (and the release as a whole) was successful.  When using agent version 2.136.1 the terminals would restart, but VSTS wouldn't receive the logs of either the restart task and many times most of the tasks from downloading build artifacts to restarting the terminal (about 5 tasks in total).  The release would eventually time out after about a total of 10 min with the error that VSTS lost connection with the agent.
All the tasks do indeed execute but the results are never reported to VSTS, thus giving false errors.
This does not occur using agent version 2.134.2
VSTS actual reported the following in the web view:

This is what i have been receiving and expected to receive:

the only difference is that the successful deployment was using agent 2.134.2 and the failed deployment used agent 2.136.1.  The successful deployment started working when i downgraded it from 2.136.1 to 2.134.2.
Agent and Worker's Diagnostic Logs
WorkerLog:
This shows that all the steps were completed (in contrast to the attached picture where the job didn't even initialize and download the tasks).
[2018-07-17 19:10:49Z INFO JobRunner] Job result after all job steps finish: SucceededWithIssues
[2018-07-17 19:10:49Z INFO JobRunner] Completing the job execution context.
[2018-07-17 19:10:49Z INFO JobRunner] Shutting down the job server queue.
[2018-07-17 19:10:49Z INFO JobServerQueue] Fire signal to shutdown all queues.
[2018-07-17 19:10:51Z INFO Worker] Cancellation/Shutdown message received.
[2018-07-17 19:10:51Z INFO HostContext] Agent will be shutdown for OperatingSystemShutdown
[2018-07-17 19:10:52Z INFO JobServerQueue] Try to append 1 batches web console lines for record 'fd56d6e8-edb0-4944-92de-17cfd6b5ee94', success rate: 1/1.
[2018-07-17 19:10:54Z ERR  VisualStudioServices] POST request to https://grovergaming.vsrm.visualstudio.com/_apis/CustomerIntelligence/Events failed. System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: A security error occurred
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Threading.Tasks.RendezvousAwaitable`1.GetResult()
at System.Net.Http.WinHttpHandler.d__105.MoveNext()
--- End of inner exception stack trace ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.d__4.MoveNext()
the finished with issues is okay as there were some previous powershell steps the failed in a non-critical manner that are to continue on error.
AgentLog:
[2018-07-17 19:11:25Z INFO RSAEncryptedFileKeyManager] Loading RSA key parameters from file C:\vstsagent\A1.credentials_rsaparams
[2018-07-17 19:11:31Z INFO VisualStudioServices] Finished operation Location.GetConnectionData
[2018-07-17 19:11:33Z ERR  VisualStudioServices] POST request to https://grovergaming.visualstudio.com/_apis/distributedtask/pools/20/sessions failed. HTTP Status: Conflict
[2018-07-17 19:11:33Z ERR  MessageListener] Catch exception during create session.
[2018-07-17 19:11:33Z ERR  MessageListener] Microsoft.TeamFoundation.DistributedTask.WebApi.TaskAgentSessionConflictException: The task agent HORSERACING-GT1 already has an active session for owner HORSERACING-GT1.
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.d__53.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.d__51.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.d__471.MoveNext() --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__281.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.VisualStudio.Services.Agent.Listener.MessageListener.d__11.MoveNext()
[2018-07-17 19:11:33Z INFO MessageListener] The session for this agent already exists.
[2018-07-17 19:11:33Z ERR  Terminal] WRITE ERROR: A session for this agent already exists.
[2018-07-17 19:11:33Z INFO MessageListener] The session conflict exception haven't reached retry limit.
[2018-07-17 19:11:33Z ERR  Terminal] WRITE ERROR: 2018-07-17 19:11:33Z: Agent connect error: The task agent HORSERACING-GT1 already has an active session for owner HORSERACING-GT1.. Retrying until reconnected.
[2018-07-17 19:11:33Z INFO MessageListener] Sleeping for 30 seconds before retrying.
[2018-07-17 19:12:03Z INFO MessageListener] Attempt to create session.
[2018-07-17 19:12:03Z INFO MessageListener] Connecting to the Agent Server...
[2018-07-17 19:12:06Z ERR  VisualStudioServices] POST request to https://grovergaming.visualstudio.com/_apis/distributedtask/pools/20/sessions failed. HTTP Status: Conflict
[2018-07-17 19:12:06Z ERR  MessageListener] Catch exception during create session.
[2018-07-17 19:12:06Z ERR  MessageListener] Microsoft.TeamFoundation.DistributedTask.WebApi.TaskAgentSessionConflictException: The task agent HORSERACING-GT1 already has an active session for owner HORSERACING-GT1.
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.d__53.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.d__51.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.d__471.MoveNext() --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__281.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.VisualStudio.Services.Agent.Listener.MessageListener.d__11.MoveNext()
[2018-07-17 19:12:06Z INFO MessageListener] The session for this agent already exists.
[2018-07-17 19:12:06Z ERR  Terminal] WRITE ERROR: A session for this agent already exists.
[2018-07-17 19:12:06Z INFO MessageListener] The session conflict exception haven't reached retry limit.
[2018-07-17 19:12:06Z INFO MessageListener] Sleeping for 30 seconds before retrying.
[2018-07-17 19:12:36Z INFO MessageListener] Attempt to create session.
[2018-07-17 19:12:36Z INFO MessageListener] Connecting to the Agent Server...
[2018-07-17 19:12:49Z INFO MessageListener] Session created.
[2018-07-17 19:12:49Z INFO Terminal] WRITE LINE: 2018-07-17 19:12:49Z: Agent reconnected.
[2018-07-17 19:12:49Z INFO Terminal] WRITE LINE: 2018-07-17 19:12:49Z: Listening for Jobs
[2018-07-17 19:12:49Z INFO JobDispatcher] Set agent/worker IPC timeout to 30 seconds.
Attached is from the pages directory showing what should be in the VSTS log but didn't make it:
bd27a351-9e2d-4d9d-969f-962309f8c105_1.log
Logs are located in the agent's _diag folder. The agent logs are prefixed with Agent_ and the worker logs are prefixed with Worker_. All sensitive information should already be masked out, please double check before pasting here.

@xtreampb the attached log shows it runs on 2.136.1 agent, so i am not sure whether it's the newer version agent causing the problem.
2018-07-17T19:10:33.0629352Z ##[section]Starting: Release
2018-07-17T19:10:33.0639352Z Current agent version: '2.136.1'

Can you also share more detail on what's 'Restart the device' means? you have a step to restart the machine?
@chshrikh FYI, in case of a Deployment Group related question.

@TingluoHuang correct, agent 2.136.1 which VSTS informed me to use this morning is causing the symptoms.  When i downgrade to 2.134.2, the symptoms go away.  The provided logs are from the failed release attempt (agent version 2.136.1).
By resarting the machine, i mean run the powershell inline task with the following command: Restart-Computer -Force

@xtreampb your release failed because of following error:
2018-07-17T19:10:48.0537926Z ##[command]. 'C:\Users\terminaluser\AppData\Local\Temp\0d313df4-323f-4218-87b2-65e8583e51c3.ps1'
2018-07-17T19:10:48.4148133Z ##[error]Rename-Item : Cannot rename because item at
'C:\grover\maestro\GoteMaestro.exe' does not exist.

have you check that error?

that powershell script task says to continue on error b/c i anticipated that

there isn't a reliable way to restart the agent and grantee the running job succeed, i don't think this is changed between 2.134 and 2.136, can you share agent log from 2.134 agent that indicate everything works?

The second image posted shows it working, and has always worked reliably until this last version.  I think the logs will be the same, I think it is more or less an issue where the agent isn't reporting before shutting down.
Successful Worker Log:
Worker_20180717-182534-utc.log
Successful Agent Log:
Agent_20180717-182443-utc.log

Attached are all the agent and worker logs i have for the failed terminal:  I had cleared all the logs before starting the job.
Agent_20180717-175639-utc.log
Agent_20180717-191109-utc.log
Agent_20180717-191310-utc.log
Worker_20180717-191027-utc.log

@xtreampb believe or not, i guess your machine is restart quicker for the failed case.
in succeed case:
agent receive shutdown signal from OS at [2018-07-17 18:25:58Z INFO Agent] Received Ctrl-Break signal from agent service host, this indicate the operating system is shutting down.
if you check the worker log, your powershell script to call restart finished at [2018-07-17 18:25:51Z INFO ProcessInvoker] Finished process with exit code 0, and elapsed time 00:00:00.3477566.
it toke 7 seconds for windows to response Restart command.
in failed case:
agent receive shutdown signal from OS at [2018-07-17 19:10:51Z INFO Agent] Received Ctrl-Break signal from agent service host, this indicate the operating system is shutting down.
the worker log shows your powershell script finished at [2018-07-17 19:10:49Z INFO ProcessInvokerWrapper] Finished process with exit code 0, and elapsed time 00:00:00.3635295.
it only toke 2 seconds for OS to handle Restart.
i believe the 2s vs 7s will make a huge different for your case.

Okay i understand what your saying, however, when I use the agent version 2.134.2 and I get the same failure on that powershell task, but the status is fully reported to VSTS release manager, not giving me a connection error.  These are the logs from the same machine and release, just downgraded to agent version 2.134.2:
Agent_20180718-141539-utc.log
Agent_20180718-141644-utc.log
Agent_20180718-142025-utc.log
Worker_20180718-141753-utc.log
and the resulting VSTS release view:

This picture is in contrast to the same release, but shown as a complete failure. (re attached below for convenience):

@xtreampb your entire release only runs for less than 30 seconds.

I am going to close this issue, the agent never support handle machine reboot, we do have a backlog work item to improve it.
#1505


shared_multi_heap: Rework framework

This PR is about entirely reworking the shared_multi_heap framework and making it useful again (tm).

Yeah I know, it didn't have much success. The shared_multi_heap framework is an extremely niche framework introduced by yours truly in #39071. It is basically used when the application or a driver needs some dynamic memory with certain characteristics. Think for example of some drivers that needs a certain amount of non-cacheable memory at run time, this framework can be used to allocate this memory from heap pools of non-cacheable memory.
Why are you reworking it?
Well, there are several problems with the current state of the framework:

It is a bit cumbersome to use given that it relies on reserved-memory DT nodes
It is using a standalone compatible in the DT that feels a bit weird to use for a framework / library
It doesn't really scale and it is not flexible enough

Why are you reworking it now?
Because with the #43119 in place we can now obtain all the information we need from the DT about the memory regions (and their attributes) without having to rely on any reserved-memory magic (that I plan to remove soon because it is heavily overlapping with the zephyr,memory-region mechanism). Please note though that the reworked version is now decoupled and doesn't strictly rely anymore on DT.
Ok, I'm sold. How does this work?
For a deeper analysis please refer to the official documentation updated in this PR, here a quick overview.
At first some platform code (possibly soc or board level) registers to the framework a set of memory regions to be used as heaps using an opaque attribute parameter that is carrying some meaning about the capability of the region (i.e. cacheable, non-cacheable, external, etc...)
struct shared_multi_heap_region cacheable = {
    .addr = region_cacheable_address,
    .size = region_cacheable_size,
    .attr = SMH_REG_ATTR_CACHEABLE,
};
shared_multi_heap_add(&region_cacheable, NULL);

struct shared_multi_heap_region noncacheable = {
    .addr = region_noncacheable_address,
    .size = region_noncacheable_size,
    .attr = SMH_REG_ATTR_NON_CACHEABLE,
};
shared_multi_heap_add(&region_noncacheable, NULL);

When a driver or application needs some dynamic memory with a certain capability, it can request the memory specifying its requirement using the opaque parameter. The framework will take care of selecting the correct heap (thus memory region) to carve memory from, based on the opaque parameter and the runtime state of the heaps (available memory, heap state, etc...).
void *buf;

// Allocate 4K of cacheable memory
buf = shared_multi_heap_alloc(SMH_REG_ATTR_CACHEABLE, 0x1000);

// Allocate 8K of non-cacheable memory
buf = shared_multi_heap_alloc(SMH_REG_ATTR_NON_CACHEABLE, 0x2000);

// Allocate 4K of non-cacheable aligned memory
buf = shared_multi_heap_aligned_alloc(SMH_REG_ATTR_NON_CACHEABLE, 4, 0x1000);

I'm dull. Can you show me a real example?
Sure, in this PR a test is provided showing how the framework is used on MMU-enabled plaftorms (qemu_cortex_a53) and MPU-enabled platforms (mps2_an521).
In both cases we use two overlay files to add a set of memory regions in the DT with different capabilities leveraging the zephyr,memory-region compatible and the zephyr,memory-region-mpu attribute so that we derive the opaque parameters (SMH_REG_ATTR_CACHEABLE and SMH_REG_ATTR_NON_CACHEABLE) from the devicetree. A test application then tries to allocate memory from the different set of heaps playing with attributes and sizes.

DNM until #43119 is merged.

@dcpleung and @andyross reviews would be very appreciated, thanks!

This needs to be rebased.

At first some platform code (possibly soc or board level) registers to the framework a set of memory regions to be used as heaps using an opaque attribute parameter that is carrying some meaning about the capability of the region (i.e. cacheable, non-cacheable, external, etc...)

Is it necessary to implement platform code? Why not add generic code to shared_multi_heap component which would register memory regions based on device tree?

Is it necessary to implement platform code? Why not add generic code to shared_multi_heap component which would register memory regions based on device tree?

Because different platforms can have different needs. The interface to register the memory regions is now generic, the information  can be retrieved from DT, but this is not mandatory anymore.
For example a platform would want to have one single big region in DT and allocate several heaps in this region according to some peculiar scheme, etc...
I also feel like cramming even more the DT with some software configuration parameters could be a bit too much.

This means we cannot track it stability status, which makes me unhappy whenever I see breaking changes :(

I can add this as experimental / unstable but in a different PR because I do not want to lose the precious Andy's ACK :)


[Enhancement]: Encryption of profile properties.

Is there an existing issue for this?

[X] I have searched the existing issues

Description of problem
During DNN Connect we discussed the option to add an option to allow a profile property to be saved encrypted to db.
Description of solution
During the summer break I will try to do the work in DNN. I have some questions and remarks I hope you all can give your oppinion on.
Description of alternatives considered

Add extra types? Or add option to encrypt on top. (I prefer the extra option)
Extra attention to basic properties e.g. for view vw_Users
Or only allow encryption on custom added properties?
Use same encryption like SMTPPassword in hostsettings
Think about changing encryption keys (via iis?)
Test if when property is deleted the values in userproperties are also deleted and not autolinked if re-created after delete.
Prevent removing encryption yes/no?
should there be a site setting to set what roles should be able to see the decrypted values? << this one seems out of scope. and has big consequenses on api.

Also we need to think about custom modules. When they use the api things will be allright, but if they use sql to get profile prop values (e.g. SP) they will get encrypted string.
Anything else?
No response
Do you be plan to contribute code for this enhancement?

[X] Yes

Would you be interested in sponsoring this enhancement?

[ ] Yes

Code of Conduct

[X] I agree to follow this project's Code of Conduct

The code is allmost ready for PR but I need some help.
Encryption of the property is ready as well as userprofile read/write.
My issue is when there are existing values in userprofile for property that is switched to encryption. atm I do not encrypt all existing values in db (it is an option offcourse)
so when you edit a user the userprop is not encrypted but the property setting is.
dnn does not update values unless they are marked as dirty. so as long as you do not change anything to the value it is not updated and thus not saved encrypted.
So the only thing I found was an ugly solution where I add an extra space to values received (only for properties marked as encrypted) so it is marked as Dirty and thus saved but now encrypted.
(spaces are trimmed when saved so nothing changes)
So maybe go back to the idea of encrypting all existing values when admin switches the property..
please any thoughts?

@sboshuis you can submit a PR of what you have so far as a "Work In Progress" which will help visualize potential solutions and make it easier to shuffle some ideas. My initial thought would be upon read if the "Secure" flag does not match the admin encryption preferences, it is resaved right there using that preference. When that setting changes, we could invalidate cache and in theory that logic of resaving should only happen once on initial direct data access.


druid in Ambari coordinator and overlord port not listen

Hello! Everyone.
I install druid with Ambari. Then,My metadata storage is mysql.
When I installed druid , everything seems like ok!
But I found coordinator and overlord port not listen.
In addition, I have check my firewall setting is ok.
following is my error:
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/DRUID/0.10.1/package/scripts/service_check.py", line 44, in
ServiceCheck().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 375, in execute
method(env)
File "/var/lib/ambari-agent/cache/common-services/DRUID/0.10.1/package/scripts/service_check.py", line 30, in service_check
self.checkComponent(params, "druid_coordinator", "druid-coordinator")
File "/var/lib/ambari-agent/cache/common-services/DRUID/0.10.1/package/scripts/service_check.py", line 40, in checkComponent
logoutput=True)
File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in init
self.env.run()
File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run
self.run_action(resource, action)
File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action
provider_action()
File "/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py", line 262, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 72, in inner
result = function(command, **kwargs)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 102, in checked_call
tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 303, in _call
raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'curl -s -o /dev/null -w'%{http_code}' --negotiate -u: -k ip-172-31-0-167.ap-northeast-1.compute.internal:8081/status | grep 200' returned 1.
Then, when I change metadata storage to derby that it can work!
Does anyone know this problem.
I don't know how to solve it !
Please someone can help me.

If changing the metadata storage to derby fixes the issue it is likely not a network issue but rather an issue with you metadata store. How was it installed?


Docs: hashbang.md: Don't use sudo for chmod

Presumably the user owns the script file in question, so there's no earthy reason why they should need to run chmod with root permissions.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Proposal for rewrite of encode module

Hi,
i need to encode videos before pushing them into after effects because some formats are rendered with issues under mac.
for this, i copied your module and changed the input path to a possible wildcard/regex.
now, any file that matches the wildcard will be encoded to the desired format.
in the process i added a preset gifs. check the commit log to see the implementation reference. i havent  tested it tbh. but this is what it could  look like.
also its now possible to pass in a ffmpeg path.
i added both to the docs/readme.
its not production code.
known issues:
if the surrounding folder of the ffmpeg or the asset destination path is missing, the process dies.
the original use for the module can not be fascilitated, sind the ouput parameter is not acutally used to name the output files. reason: its now potentially multiple files instead of one. easy fix:  check how many files where matched by the wildcard, and if its only one -> apply the name from output else the current pattern
best,
finn

Hello,
First thing I wanted to ask you, what was the motivation behind choosing to create a new action rather than updating the existing action-encode?
I had some of my own thoughts on what could be changed/improved in the action-encode as well. Like possibly switching back to a bundled version of FFmpeg inside of the binary edition rather than downloading. As well as a change that you've made with regards of providing a path to an existing FFmpeg binary installation on the system.
The file-matching feature you suggested actually brings me to an idea that those things should be extracted in a separate abstract action, that all other actions could use, and extend upon. At the same time, it would define a general rule set and a general interface with which all actions could be operated on. That, however, sounds like a bigger issue on its own, and something that should be included in a new major release.
What do you think?

i think the idea to extract the encoding of a file  into a seperate package that can be used by after render encoding as well as pre render encodning is a good a good one.
i ran into some issue trying to satisfy both usages in the same function. its not clean.
i dont know weather a major  version is  required for this. thats your judgement. I only know that i need this feature soon and will not wait for long. i would run my own implementation during the wait.

my motivation was that .mov files are rendered with sound issues on mac (weird).
there is a misalignment of audio and video when render is complete.

I see, in that case I would suggest you to just publish this action as it is, as a separate package on npm (or use local version of it). That way it will for you exactly as you need. You can take a look at here for more examples and info: https://github.com/inlife/nexrender#awesome-external-packages

ill do for the prerender encode feature.
what about the ffmpeg preset for gifs and the binary location parameter?

Those can be indeed incorporated, but I would not be able to merge them from this PR directly, since it contains too many changes. I can cherry-pick those the gif commit.
And regarding FFmpeg path, I would probably implement it manually, and based it off an environmental variable, since that is more of a system responsibility space rather than a job responsibility space.

i will publish my own module for encoding multiple files in prerender.

for my specific problem, it would be nice to have a post-download-hook. what do you think about that? we could either define it on the level of individual asset or in the actions.
the hook would receive the asset object and the local path reference.
i could use it to encode the downloaded file and update the reference inside the asset.
if we add the post-download-hook directly to the asset, i could pass parameters about the desired format of the file directly in the descriptkion of the asset. like this:

for example:
module.exports = function(job, settings) {
    settings.logger.log(`[${job.uid}] downloading assets...`)

    const promises = [].concat(
        download(job, settings, job.template),
      job.assets.map(asset => download(job, settings, asset).then(file => {
        asset.actions.postrender.map(action =>{
          return requireg(action.module)(job, settings, action, actionType, asset, file)
        })
      })),
    )

    return Promise.all(promises).then(_ => job);
}

Honestly, I don't think there is a need in post-download hook, since prerender technically is located in the exact place you are talking about:
(snippet from core/src/index/js)
        .then(job => state(job, settings, setup, 'setup'))
        .then(job => state(job, settings, download, 'download'))
        .then(job => state(job, settings, prerender, 'prerender'))
        .then(job => state(job, settings, script, 'script'))
        .then(job => state(job, settings, dorender, 'dorender'))
        .then(job => state(job, settings, postrender, 'postrender'))
        .then(job => state(job, settings, cleanup, 'cleanup'))

Added both gif preset from this PR and support for env var pointing to the exiting FFmpeg binary.
I believe this PR can be closed now.

the post download hook would be callend on an individual file basis

?

the post download hook would be callend on an individual file basis

pre-render action can be used to iterate on all assets, since it does have access to full job structure


I added a ToS and added some emojis to the README

I mean, the ToS might get rejected, but I think the emojis are really a 👍

I'm not sure if an open source library should have a ToS lol. Also the emojis look kinda unprofessional
Therefore no merge, sorry


Use more common expression for 24/7

Normally, it's 24/7, not 7/24 (ref: https://en.wikipedia.org/wiki/24/7_service)

Thank you @rassie 🚀


Fail to parse url

Url are not accepted by the html_tokenizer.
To reproduce, just replace line 28 of "test_html_parser.ml" from
  |> Markup__Html_tokenizer.tokenize Error.ignore_errors

to
  |> Markup__Html_tokenizer.tokenize report

and make test yields
File "test/support/test_support.ml", line 70, characters 1-1:
Error: markup.ml:220:html.parser.links (in the code).

foo
got "line 1, column 24: bad token '&image=' in attribute: unterminated entity reference followed by '='"

I submitted a PR #73 for that.

Hello
https://thisthat.dev/encode-uri-vs-encode-uri-component/ explains that there are 11 characters not encoded in Uri except when they are in the path or query values.
Percent encoding does not work for query. In my patch I took exactly the attributes which have an uri type.
Cheers
Christophe
Le 13 juillet 2023 03:30:52 GMT-10:00, Anton Bachin @.***> a écrit :

AFAIK, the current behavior of Markup.ml is correct. URLs, like any other strings, should be HTML attribute encoded before being inserted into attributes, so they should look like http://foo.org/?a=1&b=2. There is also nothing in the spec on parsing attribute values on special handling for URLs.
--
Reply to this email directly or view it on GitHub:
https://github.com/aantron/markup.ml/issues/72#issuecomment-1634250581
You are receiving this because you authored the thread.
Message ID: @.***>

https://datatracker.ietf.org/doc/html/rfc3986#page-11 here is the precise RFC. I could implement it exactly to verify uri?
Le 13 juillet 2023 03:30:52 GMT-10:00, Anton Bachin @.***> a écrit :

AFAIK, the current behavior of Markup.ml is correct. URLs, like any other strings, should be HTML attribute encoded before being inserted into attributes, so they should look like http://foo.org/?a=1&b=2. There is also nothing in the spec on parsing attribute values on special handling for URLs.
--
Reply to this email directly or view it on GitHub:
https://github.com/aantron/markup.ml/issues/72#issuecomment-1634250581
You are receiving this because you authored the thread.
Message ID: @.***>

I dig more that issue. First firefox and chromium accept non percent encoded delimiter in URI and delimiter are defined in RFC 3986, where it is clear that percent encoding is used for escape reserved characters in values.
Nevertheless, this document:
https://www.w3.org/TR/REC-html40/appendix/notes.html#non-ascii-chars agrees with you.
In the end I can live with both, feel free to close the issue and PR #73 or dig more the subject.

And by the way, I don't know how browser manage to accept both kind of url, this should mess up their code ?

And while digging, I learnt I could use, and it is recommended to use semicolumn for query ;-)

I'm not sure I understood you, but replying to what I think you mean.
The issue here is not percent encoding -- that is a separate step. After percent encoding or any other encoding applied to the URI, when the encoded URI is placed into an attribute value, it should be additionally encoded using HTML attribute encoding, which should, among other things, convert & to &. Not doing so can work for most URLs, because the error is recoverable by the HTML parser. But if you have a URI like http://foo.org/?a=1&amp=2 it will be misinterpreted if not HTML attribute encoded as http://foo.org/?a=1&amp=2.

And, as you saw here, the document also recommends escaping. It's not really even recommended, but practically required, though if you are manually writing HTML you can get by without it because error recovery will save you as long as a query parameter name does not match one of the HTML entity names.
So I think there is nothing to fix for this issue on the Markup.ml side.

Anton Bachin @.***> writes:

And, as you saw here, the document also recommends escaping. It's not really even recommended, but practically required, though if you are manually
writing HTML you can get by without it because error recovery will save you as long as a query parameter name does not match one of the HTML entity
names.
So I think there is nothing to fix for this issue on the Markup.ml side.

Yes I agree. took me some time to understant all this, sorry.

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.

--
Christophe Raffalli
tél: +689 87 23 11 48
web: http://raffalli.eu
Ce mail est signé avec pgp (Pièce jointe signature.asc, clef sur https://raffalli.eu/pgp)
This mail is signed with pgp (Attachment signature.asc, key  on  https://raffalli.eu/pgp)

Thanks! I'm going to close this issue, but feel free to discuss further if anything else comes up on it.

I read the html specification here and it says that the only restriction for & is about ambiguous ones, i.e. &toto;
where toto is not a defined character reference. Hence href="/path?x=1&y=2" is a perfectly valid attribute value as there is no semicolumn after the y:

Attribute values are a mixture of text and character references, except with the additional restriction that the text cannot contain an ambiguous ampersand.

The HTML specification specifies a particular behavior for the tokenizer, including when it emits errors. The section you are citing is under Writing HTML documents, rather than parsing them. The rules for writing HTML documents are much simpler.


chore(deps): update ratatui to 0.26.0

Requires an update to tui-react / crosstermion to work

Thanks a lot for the service!


Cannot find module '.bin/pub' on katoda streetlights demo

Hi,
Describe the bug
When i run command  in step 2:
mqtt pub -t 'light/measured' -h 'localhost' -m '{"id": 1, "lumens": 3, "sentAt": "2017-06-07T12:34:32.000Z"}'

then,  I get error in interactive console
Error: Cannot find module './bin/pub'
    at Function.Module._resolveFilename (module.js:548:15)
    at Function.Module._load (module.js:475:25)
    at Module.require (module.js:597:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/usr/lib/node_modules/mqtt/bin/mqtt.js:14:29)
    at Module._compile (module.js:653:30)
    at Object.Module._extensions..js (module.js:664:10)
    at Module.load (module.js:566:32)
    at tryModuleLoad (module.js:506:12)
    at Function.Module._load (module.js:498:3)

How to Reproduce
https://www.katacoda.com/asyncapi/scenarios/streetlight-tut
Go to Step 2 and run MQTT command

@michmzr thanks man for reporting. Looks like there is some bug in MQTT package -> https://github.com/mqttjs/MQTT.js/issues/1106
I'll update the tutorial shortly

@michmzr ok, change is published, I checked and works like a charm. Thanks again for taking time to report it!


Possibility of implementing YCCK color transform as available in some jpeg files?

For more info read:
http://halicery.com/Image Decoders/JPEG/JPEG notes/CMYK.html

After studying this issue, it appears that CMYK is functionally equivalent to RGBA, as K functions as an inverse alpha mask. The missing capability is CMY<-->RGB conversion, as YCC is merely YUV transform applied to CMY.

After studying this issue, it appears that CMYK is functionally equivalent to RGBA, as K functions as an inverse alpha mask. The missing capability is CMY<-->RGB conversion, as YCC is merely YUV transform applied to CMY.

Still debating whether to implement this internally. It is possible to transform YCCK/CMYK to YUV/RGB with an I/O callback:
YUV:
Y = 1 - ((1 - K) * Y + K)
U = -((1 - K) * C0)
V = -((1 - K) * C1)
RGB:
R = 1 - ((1 - K) * C + K)
G = 1 - ((1 - K) * M + K)
B = 1 - ((1 - K) * Y + K)
or more abstractly
Negative = (1 - K) * Negative_K + K * Black
Positive = Black - Negative
where Black = [1 0 0] for YCC and [1 1 1] for CMY

Still debating whether to implement this internally. It is possible to transform YCCK/CMYK to YUV/RGB with an I/O callback:
YUV:
Y = 1 - ((1 - K) * Y + K)
U = -((1 - K) * C0)
V = -((1 - K) * C1)
RGB:
R = 1 - ((1 - K) * C + K)
G = 1 - ((1 - K) * M + K)
B = 1 - ((1 - K) * Y + K)
or more abstractly
Negative = (1 - K) * Negative_K + K * Black
Positive = Black - Negative
where Black = [1 0 0] for YCC and [1 1 1] for CMY

what are C0 and C1 values?

what are C0 and C1 values?

First and second negative chroma channels. Per "YCC".

First and second negative chroma channels. Per "YCC".

And also PhotoYCC as was used in Photo CD, is used in JPEG 2000 and JPEG.
I described it here: https://en.wikipedia.org/wiki/Photo_CD#Encoding
Uses xvYCC transfer function (negative RGB values use XYZ matrix), not scaled BT.601 matrix and BT.709 color primaries. Uses different quantization (Y' is instead applied scaling in BT.601 matrix, 1.402). Implementation is here: https://github.com/DavidGriffith/finx/blob/17825bf28b2351f96867b34a7dc39f7997e2f9ff/nx-X11/programs/Xserver/XIE/mixie/process/mprgb.c#L1004
YCCK implementation is here: https://github.com/mozilla/mozjpeg/search?q=YCCK

And also PhotoYCC as was used in Photo CD, is used in JPEG 2000 and JPEG.
I described it here: https://en.wikipedia.org/wiki/Photo_CD#Encoding
Uses xvYCC transfer function (negative RGB values use XYZ matrix), not scaled BT.601 matrix and BT.709 color primaries. Uses different quantization (Y' is instead applied scaling in BT.601 matrix, 1.402). Implementation is here: https://github.com/DavidGriffith/finx/blob/17825bf28b2351f96867b34a7dc39f7997e2f9ff/nx-X11/programs/Xserver/XIE/mixie/process/mprgb.c#L1004
YCCK implementation is here: https://github.com/mozilla/mozjpeg/search?q=YCCK

@richardpl Can you provide a raw dump of YCCK or CMYK image data?

Relevant file can be downloaded from here:
http://trac.ffmpeg.org/ticket/9085

The FFmpeg tracker only has a JPEG file. If I open it in an image viewer, I only get the decoded RGB. Please upload the raw YCCK pixel data.

The JPEG is fine; the viewer is the problem. Unfortunately there are very few formats that hold CMYK.
djpeg produces the attached pnm, but I cannot verify whether it is correct. It is probably CMYK and not YCC still. I mean, you expect djpeg to give you RGB and not YCC, and it's the same here. If you want the guts, you need to get the library out.
Channel.pnm.zip


Battery status not showing in HA

What type of installation are you running?
Desktop
Which Linux OS are you using?
Linux Mint Mate
Which version of LNXLink has the issue?
2023.6.1
The problem
Hi, me again :-)
I got everything installed and running. The messages are all showing correct in MQTT Explorer. Most of the values are also shown and updated regularly in Home Assistant, except the one that is most important to me: Battery status.
MQTT Explorer shows for example this:
battery = {"status": 51.0, "SanyoDELL 5K1GW03G": 51.0, "Wacom HID 482D": 0.0}
The Battery icon is shown in HA, but when I click it, it shows:

Also: Would it be possible to add the CPU temperature to the outputs? I have seen the examples on the website, but my Python programming skills are a bit limited...
Logs
No response

I am not sure there is a way to distinguish the CPU temperature out of all the temperature sensors on your system, but I'll see what I can do.
You should check through MQTT Explorer if the state is sent in a regular interval. If it is not, then it automatically becomes unavailable.
You should also send me the logs. This is the command that will give you the logs:
journalctl --user -u lnxlink -n 300 --no-pager

It looks like the state of the battery is sent to HA only once (and it is retained) about 8 minutes after the tablet starts. But I cannot see any value sent.

This is the part that journalctl shows for today:
-- Boot 0bad024ec53244c29471402668995f66 --
Jun 25 10:30:36 hermann-Latitude-5175 systemd[941]: Started LNXlink.
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: LNXlink 2023.6.1 started: 3.10.6
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: Loaded addon: battery
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: Loaded addon: cpu
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: Loaded addon: idle
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: Loaded addon: screen_onoff
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: Loaded addon: bash
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: Loaded addon: boot_select
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: Loaded addon: brightness
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: Loaded addon: camera_used
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: Loaded addon: disk_usage
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: Loaded addon: keep_alive
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: Loaded addon: media
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: Loaded addon: memory
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: Loaded addon: microphone_used
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: Loaded addon: network
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: Loaded addon: notify
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: Loaded addon: required_restart
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: Loaded addon: restart
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: Loaded addon: send_keys
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: Loaded addon: shutdown
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: Loaded addon: suspend
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: Loaded addon: webcam
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: Loaded addon: xdg_open
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: Traceback (most recent call last):
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]:   File "/home/hermann/.local/bin/lnxlink", line 8, in <module>
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]:     sys.exit(main())
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]:   File "/home/hermann/.local/lib/python3.10/site-packages/lnxlink/__main__.py", line 335, in main
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]:     lnxlink = LNXlink(config_file)
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]:   File "/home/hermann/.local/lib/python3.10/site-packages/lnxlink/__main__.py", line 42, in __init__
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]:     self.setup_mqtt()
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]:   File "/home/hermann/.local/lib/python3.10/site-packages/lnxlink/__main__.py", line 102, in setup_mqtt
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]:     self.client.connect(self.config['mqtt']['server'], self.config['mqtt']['port'], 60)
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]:   File "/home/hermann/.local/lib/python3.10/site-packages/paho/mqtt/client.py", line 914, in connect
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]:     return self.reconnect()
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]:   File "/home/hermann/.local/lib/python3.10/site-packages/paho/mqtt/client.py", line 1044, in reconnect
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]:     sock = self._create_socket_connection()
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]:   File "/home/hermann/.local/lib/python3.10/site-packages/paho/mqtt/client.py", line 3685, in _create_socket_connection
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]:     return socket.create_connection(addr, timeout=self._connect_timeout, source_address=source)
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]:   File "/usr/lib/python3.10/socket.py", line 845, in create_connection
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]:     raise err
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]:   File "/usr/lib/python3.10/socket.py", line 833, in create_connection
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]:     sock.connect(sa)
Jun 25 10:30:37 hermann-Latitude-5175 lnxlink[950]: OSError: [Errno 101] Network is unreachable
Jun 25 10:30:37 hermann-Latitude-5175 systemd[941]: lnxlink.service: Main process exited, code=exited, status=1/FAILURE
Jun 25 10:30:37 hermann-Latitude-5175 systemd[941]: lnxlink.service: Failed with result 'exit-code'.
Jun 25 10:30:37 hermann-Latitude-5175 systemd[941]: lnxlink.service: Consumed 1.360s CPU time.
Jun 25 10:30:38 hermann-Latitude-5175 systemd[941]: lnxlink.service: Scheduled restart job, restart counter is at 1.
Jun 25 10:30:38 hermann-Latitude-5175 systemd[941]: Stopped LNXlink.
Jun 25 10:30:38 hermann-Latitude-5175 systemd[941]: lnxlink.service: Consumed 1.360s CPU time.
Jun 25 10:30:38 hermann-Latitude-5175 systemd[941]: Started LNXlink.

It seems that a lot of the logs after are missing.
Can you run it manually to get the error logs?
Follow these commands by replacing the config.yaml with your config file that you've used:
systemctl --user stop lnxlink.service
lnxlink -c config.yaml

When I run lnxlink from my home directory (where config.yaml is), I get this:
hermann@hermann-Latitude-5175:~$ lnxlink -c config.yaml
LNXlink 2023.6.1 started: 3.10.6
Loaded addon: battery
Loaded addon: cpu
Loaded addon: idle
Loaded addon: screen_onoff
Loaded addon: bash
Loaded addon: boot_select
Loaded addon: brightness
Loaded addon: camera_used
Loaded addon: disk_usage
Loaded addon: keep_alive
Loaded addon: media
Loaded addon: memory
Loaded addon: microphone_used
Loaded addon: network
Loaded addon: notify
Loaded addon: required_restart
Loaded addon: restart
Loaded addon: send_keys
Loaded addon: shutdown
Loaded addon: suspend
Loaded addon: webcam
Loaded addon: xdg_open
Connected to MQTT with code 0
Can't load addon: idle
Traceback (most recent call last):
  File "/home/hermann/.local/lib/python3.10/site-packages/lnxlink/__main__.py", line 77, in monitor_run
    pub_data = addon.getInfo()
  File "/home/hermann/.local/lib/python3.10/site-packages/lnxlink/modules/idle.py", line 16, in getInfo
    idle_ms = monitor.get_dbus_idle()
  File "/home/hermann/.local/lib/python3.10/site-packages/dbus_idle/__init__.py", line 105, in get_dbus_idle
    idle_sec = int(stdout.strip())
ValueError: invalid literal for int() with base 10: ''
Can't load addon: brightness
Traceback (most recent call last):
  File "/home/hermann/.local/lib/python3.10/site-packages/lnxlink/__main__.py", line 81, in monitor_run
    pub_data = addon.getControlInfo()
  File "/home/hermann/.local/lib/python3.10/site-packages/lnxlink/modules/brightness.py", line 12, in getControlInfo
    avg_brightness = sum(displays.values()) / len(displays.values())
ZeroDivisionError: division by zero
Can't load addon: idle
Traceback (most recent call last):
  File "/home/hermann/.local/lib/python3.10/site-packages/lnxlink/__main__.py", line 77, in monitor_run
    pub_data = addon.getInfo()
  File "/home/hermann/.local/lib/python3.10/site-packages/lnxlink/modules/idle.py", line 16, in getInfo
    idle_ms = monitor.get_dbus_idle()
  File "/home/hermann/.local/lib/python3.10/site-packages/dbus_idle/__init__.py", line 105, in get_dbus_idle
    idle_sec = int(stdout.strip())
ValueError: invalid literal for int() with base 10: ''
Can't load addon: brightness
Traceback (most recent call last):
  File "/home/hermann/.local/lib/python3.10/site-packages/lnxlink/__main__.py", line 81, in monitor_run
    pub_data = addon.getControlInfo()
  File "/home/hermann/.local/lib/python3.10/site-packages/lnxlink/modules/brightness.py", line 12, in getControlInfo
    avg_brightness = sum(displays.values()) / len(displays.values())
ZeroDivisionError: division by zero

The Traceback... lines are then repeated every few seconds. It is interesting that it says cannot load idle and brightness. When I run this as a service, I see the value of idle changing and I can set the brightness of the screen from HA.
Is there any other log I can check?
The config.yaml looks like this:

mqtt:
  prefix: lnxlink
  clientId: DESKTOP-Linux
  statsPrefix: monitor/stats
  server: <IP_ADDRESS>
  port: 1883
  auth:
    user:
    pass:
  discovery:
    enabled: true
  lwt:
    enabled: true
    qos: 1
    retain: true
    connectMsg: 'ON'
    disconnectMsg: 'OFF'
update_interval: 5
modules:
- battery
- cpu
- idle
- screen_onoff
- bash
- boot_select
- brightness
- camera_used
- disk_usage
- keep_alive
- media
- memory
- microphone_used
- network
- notify
- required_restart
- restart
- send_keys
- shutdown
- suspend
- webcam
- xdg_open

You can check if the individual commands work. Send me the results to see what's the problem.
For Brightness
xrandr --verbose --current | grep Brightness

For Idle
python3.10

from dbus_idle import IdleMonitor
float(IdleMonitor.get_monitor().get_dbus_idle())

Not sure what changed today, but
lnxlink -c config.yaml

did not give me any more errors.
hermann@hermann-Latitude-5175:~$ lnxlink -c config.yaml
LNXlink 2023.6.1 started: 3.10.6
Loaded addon: battery
Loaded addon: cpu
Loaded addon: idle
Loaded addon: screen_onoff
Loaded addon: bash
Loaded addon: boot_select
Loaded addon: brightness
Loaded addon: camera_used
Loaded addon: disk_usage
Loaded addon: keep_alive
Loaded addon: media
Loaded addon: memory
Loaded addon: microphone_used
Loaded addon: network
Loaded addon: notify
Loaded addon: required_restart
Loaded addon: restart
Loaded addon: send_keys
Loaded addon: shutdown
Loaded addon: suspend
Loaded addon: webcam
Loaded addon: xdg_open
Connected to MQTT with code 0

hermann@hermann-Latitude-5175:~$ xrandr --verbose --current | grep Brightness
	Brightness: 1.0

This changes to the value I set in HA.
hermann@hermann-Latitude-5175:~$ python3.10
Python 3.10.6 (main, May 29 2023, 11:10:38) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from dbus_idle import IdleMonitor
>>> float(IdleMonitor.get_monitor().get_dbus_idle())
6.0
>>> float(IdleMonitor.get_monitor().get_dbus_idle())
8.0
>>> float(IdleMonitor.get_monitor().get_dbus_idle())
7.0
>>>

The numbers keep changing every time I call it.

I worked around it by reading the MQTT values with node-red.


test_parallel_cache_files doesn't work with strict mode

https://github.com/aimclub/FEDOT/blob/master/test/integration/cache/test_parallel_cache_files.py#L25

Solved in https://github.com/aimclub/FEDOT/pull/987


add conditionally rendering components example

closes https://github.com/typescript-cheatsheets/react-typescript-cheatsheet/issues/147

@sami616 anything else to add?

Nothing from me! Looks great. Thanks so much


remove algoliasearch.helper from main project

For V3 we can remove the algoliasearch helper from the main library.

[ ] create a new github project for the helper
[ ] algoliasearch-helper in npmjs
[ ] algoliasearchHelper in UMD build (browserify -s algoliasearchHelper)
[ ] helper search function should follow algoliasearch API on callbacks and promises: we should be able to use a callback or a promise.
[ ] maybe some simple tests (for V3, in the end we would need test coverage) reusing the same toolset than the JS client
[ ] udpate V3 changelog stating it's a breaking change

@bobylito would be cool to get your help on this, I did a quick TODO but that's your doby now :)

@bobylito feel free to then open an issue on enhancing the helper. That's a step we will make after V3 release.


Fix colour picker current not updating on HSV changes when not present

Closes #4620.
This is incidentally the perfect counterexample for why blanket-including this.Scheduler.HasPendingTasks in the base Drawable.IsPresent is not a be-all and end-all of these types of issues.
Doing "the obvious" does not work with this component's structure. The child components are not the ones accessible outwardly, the picker itself is. And if someone is messing with the inner components, they are creating a problem entirely by themselves by misusing them.

Did not take that into account... that drops the idea of "making scheduler run while not present" completely.
Though (and unfortunately) I think the ColourPicker should also override IsPresent as it is the one that would be hidden?

Done.


overheating cause keyboard manager to stop working properly

Microsoft PowerToys version
0.66.0
Installation method
PowerToys auto-update
Running as admin
Yes
Area(s) with issue?
Keyboard Manager
Steps to reproduce
usually happen when I run multi thread processes in pycharm.
specifically I remapped capslock to backspace and while i type the computer will "forget" that the remapping is in effect will treat capslock as capslock for 1 click (meaning that capslock will turn on) but then continue to function correctly (as backspace) and i will need to open powertoys to change it back.
✔ Expected Behavior
capslock will continue to behave as backspace even when the computer is working hard
❌ Actual Behavior
the remapping fails sometimes while typing and capslock is being pressed incorrectly
Other Software
No response

I experience the same problem, though, I wasn't able to attribute it to a specific cause. Overheating is a possibility..

/dup https://github.com/microsoft/PowerToys/issues/27319