Introduce namespace exclusions for downgrading "pipeline" SA

Goal

Allow the admin to turn off anyuid in specific namespaces or all namespaces using a regex or "exact" string match

What

Added new API attribute called spec.namespaceExclusions in the Config CRD where the admin would be able to specify which namespaces are to have the pipeline SAs downgraded from anyuid to restricted by removing the rolebinding responsible for the same.

apiVersion: operator.tekton.dev/v1alpha1
kind: Config
spec:
  namespaceExclusions:
    - sbose2                         ## regex is supported too.
  targetNamespace: openshift-pipelines

Maybe support namespaceInclusions too?
What I've tested so far

Validate that the Config CR called cluster is auto-created ( no change )
Validate that a ClusterRole for "anyuid" is created.
Validate that all namespaces have pipeline sa bound to the above role using a rolebinding
Edit cluster to add specific namespaces to the list of namespaceExclusions
Validate removal of rolebindings seen in (2)
Empty the list of namespaceExclusions and see the rolebindings re-appear.

TODO

[x] Validate reconciliation of SAs, rolebindings, roles
[x] Tests

@sbose78 I would like to split this into two.
One for:

Downgraded controller sa from privileged to nonroot.
Granted anyuid to every pipeline sa. ( added a clusterrole and a rolebinding in every namespace )

A follow-up for

Added new API attribute called spec.namespaceExclusions in the Config CRD where the admin would be able to specify which namespaces are to have the pipeline SAs downgraded from anyuid to restricted by removing the rolebinding responsible for the same.

One reason for it is, so far, we do not put anything in the Config CR because it is automatically created without the user having anything to say about it (at install). I would like to experiment with http://openshift.github.io/openshift-origin-design/designs/administrator/olm/operand-creation-at-operator-install/ before putting some fields in the configuration.

@vdemeester Sounds good, have them in a single branch because I wanted to test it end-to-end  💇♂

I would like to experiment with http://openshift.github.io/openshift-origin-design/designs/administrator/olm/operand-creation-at-operator-install/ before putting some fields in the configuration.

Link is broken :( or maybe I don't have access to it.

Not sure why you don't have access though :stuck_out_tongue:

Magically the link works now ;)
I didn't get you though 🤔 .
I was hoping the admin would open the  cluster resource and edit it to exclude specific namespaces if she wants to. What is your proposed approach to specifying this information?
🚗

I was hoping the admin would open the cluster resource and edit it to exclude specific namespaces if she wants to. What is your proposed approach to specifying this information?

Yeah that's a possibility. I don't necessarly have any other alternative right now.

Removed code which has been incorporated in previous PRs.

/test 4.5-e2e

/test 4.5-e2e

/retest

/retest

Somehow 4.4-e2e fails
/retest

@sbose78 I would like to split this into two.
One for:
* Downgraded controller sa from `privileged` to `nonroot`.

hey @vdemeester  @sbose78 ... following up on this area after the holidays.
I know @vdemeester 's comment above was a ways back (Oct 2020). but given some of the user interaction just before the holidays (i.e. https://github.com/openshift/tektoncd-pipeline-operator/issues/545#issuecomment-749570146 for example) I'm curious if
a) we need to resuscitate this PR and get it completed
b) if we should take things a step further on @vdemeester 's point and downgrade the controller/webhook SA's to restricted
I just tried it today with the master branch of tektoncd/pipelines and things work (at least the basics) with restricted ... see https://github.com/tektoncd/pipeline/issues/3625#issuecomment-754093515 for details.
thanks

* Granted `anyuid` to every pipeline sa. ( added a clusterrole and a rolebinding in every namespace )

A follow-up for
* Added new API attribute called `spec.namespaceExclusions` in the `Config` CRD where the admin would be able to specify which namespaces are to have the `pipeline` SAs downgraded from `anyuid` to `restricted` by removing the rolebinding responsible for the same.

One reason for it is, so far, we do not put anything in the Config CR because it is automatically created without the user having anything to say about it (at install). I would like to experiment with http://openshift.github.io/openshift-origin-design/designs/administrator/olm/operand-creation-at-operator-install/ before putting some fields in the configuration.

@sbose78 I would like to split this into two.
One for:
* Downgraded controller sa from `privileged` to `nonroot`.

hey @vdemeester  @sbose78 ... following up on this area after the holidays.
I know @vdemeester 's comment above was a ways back (Oct 2020). but given some of the user interaction just before the holidays (i.e. https://github.com/openshift/tektoncd-pipeline-operator/issues/545#issuecomment-749570146 for example) I'm curious if
a) we need to resuscitate this PR and get it completed
b) if we should take things a step further on @vdemeester 's point and downgrade the controller/webhook SA's to restricted
I just tried it today with the master branch of tektoncd/pipelines and things work (at least the basics) with restricted ... see https://github.com/tektoncd/pipeline/issues/3625#issuecomment-754093515 for details.
thanks

* Granted `anyuid` to every pipeline sa. ( added a clusterrole and a rolebinding in every namespace )

A follow-up for
* Added new API attribute called `spec.namespaceExclusions` in the `Config` CRD where the admin would be able to specify which namespaces are to have the `pipeline` SAs downgraded from `anyuid` to `restricted` by removing the rolebinding responsible for the same.

One reason for it is, so far, we do not put anything in the Config CR because it is automatically created without the user having anything to say about it (at install). I would like to experiment with http://openshift.github.io/openshift-origin-design/designs/administrator/olm/operand-creation-at-operator-install/ before putting some fields in the configuration.

if we should take things a step further on @vdemeester 's point and downgrade the controller/webhook SA's to restricted

That sounds good to me. I was rather playing safe, but since you've verified, I'm happy to make it restricted. The pipeline sa anyway would have the higher privileges for buildah push

if we should take things a step further on @vdemeester 's point and downgrade the controller/webhook SA's to restricted

That sounds good to me. I was rather playing safe, but since you've verified, I'm happy to make it restricted. The pipeline sa anyway would have the higher privileges for buildah push

if we should take things a step further on @vdemeester 's point and downgrade the controller/webhook SA's to restricted

That sounds good to me. I was rather playing safe, but since you've verified, I'm happy to make it restricted.

Cool deal

The pipeline sa anyway would have the higher privileges for buildah push

Yep exactly.  And if we discover some other operation besides buildah push typically done in user pipelines needs something higher that restricted, they should get the necessary perms from using the pipeline SA

if we should take things a step further on @vdemeester 's point and downgrade the controller/webhook SA's to restricted

That sounds good to me. I was rather playing safe, but since you've verified, I'm happy to make it restricted.

Cool deal

The pipeline sa anyway would have the higher privileges for buildah push

Yep exactly.  And if we discover some other operation besides buildah push typically done in user pipelines needs something higher that restricted, they should get the necessary perms from using the pipeline SA

a) we need to resuscitate this PR and get it completed

Yes we should but it should be "re-done" on tektoncd/operator now 🙃.

b) if we should take things a step further on @vdemeester 's point and downgrade the controller/webhook SA's to restricted
I just tried it today with the master branch of tektoncd/pipelines and things work (at least the basics) with restricted ... see tektoncd/pipeline#3625 (comment) for details.

Nice, I am all for that, let's wait for 1.3 to be released and let's do that 🙃

a) we need to resuscitate this PR and get it completed

Yes we should but it should be "re-done" on tektoncd/operator now 🙃.

b) if we should take things a step further on @vdemeester 's point and downgrade the controller/webhook SA's to restricted
I just tried it today with the master branch of tektoncd/pipelines and things work (at least the basics) with restricted ... see tektoncd/pipeline#3625 (comment) for details.

Nice, I am all for that, let's wait for 1.3 to be released and let's do that 🙃

Yes we should but it should be "re-done" on tektoncd/operator now 🙃.

Will do, Vincent!

Yes we should but it should be "re-done" on tektoncd/operator now 🙃.

Will do, Vincent!

@sbose78  The SA pipeline is granted a role binding to the cluster role edit here
https://github.com/openshift/tektoncd-pipeline-operator/blob/2c35616a361ef2bb17a31e90bfc9995d9d6eddc6/pkg/controller/rbac/rbac_controller.go#L197
Is there any plan to make this configurable ? We would like to be able to restrict the resources that can be worked on in namespaces and granting the creation of pipeline/task/run indirectly opens pretty much everything without choice on the granularity.

@sbose78  The SA pipeline is granted a role binding to the cluster role edit here
https://github.com/openshift/tektoncd-pipeline-operator/blob/2c35616a361ef2bb17a31e90bfc9995d9d6eddc6/pkg/controller/rbac/rbac_controller.go#L197
Is there any plan to make this configurable ? We would like to be able to restrict the resources that can be worked on in namespaces and granting the creation of pipeline/task/run indirectly opens pretty much everything without choice on the granularity.

@gabemontero @vdemeester @sbose78 great continued conversation on this and related issues.
So from having the latest comments, does this mean that the change to actually be implemented is (at a high level): "downgrade the controller/webhook SA's to restricted".
And that because additional SCC access can be granted by SAs attached to the pipeline, the namespaceExclusions change isn't required?

@gabemontero @vdemeester @sbose78 great continued conversation on this and related issues.
So from having the latest comments, does this mean that the change to actually be implemented is (at a high level): "downgrade the controller/webhook SA's to restricted".
And that because additional SCC access can be granted by SAs attached to the pipeline, the namespaceExclusions change isn't required?

@gabemontero @vdemeester @sbose78 great continued conversation on this and related issues.
So from having read the latest comments, does this mean that the change to actually be implemented is (at a high level): "downgrade the controller/webhook SA's to restricted".

Yes one of the upcoming changes (@vdemeester said he wanted to pursue it I believe after v1.3 of openshift pipelines goes out) is to assign the restricted SCC to the various tekton controller/webhook SAs
That does not mean  that is the only change that will be pursued.  See below.

And that because additional SCC access can be granted by SAs attached to the pipeline, the namespaceExclusions change isn't required?

I think there is still a desire to facilitate which SCCs an administrator would like assigned to the pipeline SA openshift pipelines provides out of the box.  Among other things, different configurations of image building tools, like buildah, can require different SCCs (anyuid vs. privileged).  Also, there is ongoing work in buildah that potentially can reduce the permissions needed, so we want flexibility there down the road.
Otherwise, if I'm not up to date on the latest nuances, I'm sure @vdemeester and @sbose78 will clarify.

@gabemontero @vdemeester @sbose78 great continued conversation on this and related issues.
So from having read the latest comments, does this mean that the change to actually be implemented is (at a high level): "downgrade the controller/webhook SA's to restricted".

Yes one of the upcoming changes (@vdemeester said he wanted to pursue it I believe after v1.3 of openshift pipelines goes out) is to assign the restricted SCC to the various tekton controller/webhook SAs
That does not mean  that is the only change that will be pursued.  See below.

And that because additional SCC access can be granted by SAs attached to the pipeline, the namespaceExclusions change isn't required?

I think there is still a desire to facilitate which SCCs an administrator would like assigned to the pipeline SA openshift pipelines provides out of the box.  Among other things, different configurations of image building tools, like buildah, can require different SCCs (anyuid vs. privileged).  Also, there is ongoing work in buildah that potentially can reduce the permissions needed, so we want flexibility there down the road.
Otherwise, if I'm not up to date on the latest nuances, I'm sure @vdemeester and @sbose78 will clarify.


Add iOS build support, fix Metal command buffer leaks.

Superseded by https://github.com/godotengine/godot-angle-static/pull/7


netsh deprecation - need pointers to Powershell equivalents

This page indicates that netsh is deprecated in favour of powershell ("It is recommended that you use Windows PowerShell to manage networking technologies in Windows Server 2016 and Windows 10 rather than Network Shell [...]"), however there appear to be no (at least obvious) replacements for "netsh http" (URL reservations, SSL certificate binding, etc).
Can you provide pointers to the equivalent PS documentation for "netsh http"? If this doesn't exist, can this page indicate what features of netsh are not supported by powershell so we're not spending time pointlessly searching the internet.
More generally, it would be useful to list all the features of netsh together with their powershell equivalents to people porting scripts have a one-place easy reference to this information.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 104c0791-b5a1-6502-401a-d472ff6144bb
Version Independent ID: 5a8faa14-9cef-6a9b-26af-9782a799b23a
Content: Network Shell (Netsh)
Content Source: WindowsServerDocs/networking/technologies/netsh/netsh.md
Product: windows-server-threshold
GitHub Login: @shortpatti
Microsoft Alias: pashort

just stumbled across this:
https://stackoverflow.com/questions/6851161/net-or-win32-equivalent-of-netsh-http-add-urlacl-command
Maybe it helps :)

@Djfe, thank you for providing the extra information. I'll get it incorporated :)

Thank you for your patience.  I've updated the content.

cool, thx
The link is unavailable though (404)

Ah, it must not have been published yet.  Give it until this afternoon.
It's essentially this.


Match upstream

With these last few changes we should now match upstream!

Pull Request Test Coverage Report for Build 19

16 of 16 (100.0%)  changed or added relevant lines in 4 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+4.05%) to 92.662%

Totals

Change from base Build 18:
4.05%

Covered Lines:
846

Relevant Lines:
913

💛  - Coveralls

Thanks for all your work! I'd like to review this side-by-side with upstream before officially releasing.
One thing we should address (either by deleting or updating) is that the ZxcvbnTest is still skipped.

re that broken Travis build - it's a one-line workaround to get around that failure in PHP 5.5, but I think we should just support 5.6+. 5.5 has been unsupported for almost 2 years.

@mkopinsky I think this is done in terms of matching upstream and is ready to merge. I've tested against the top 1 million passwords from the rockyou password list and haven't found any further discrepancies.

Can you add the scripts or documentation on how you did that comparison? I think that would be a useful tool for the future when/if upstream releases a new version.


Create DB Models

Can use this dataset to populate: https://www.kaggle.com/datasets/darinhawley/new-york-times-crossword-clues-answers-19932021

Can start with just words and clues for now, eventually will need to add grids


Upgrade to Kubernetes 1.6.2 and Helm 2.3.1

Any chance this is going to happen?

I did this myself by updating glide.yaml and following these steps: https://github.com/Eneco/landscaper#from-source

I am not sure how that could have worked for you since some subpackages no longer exist in helm 2.3.1.
[ERROR] Error scanning k8s.io/helm/cmd/helm/downloader: open /home/scheetz/.glide/cache/src/https-k8s.io-helm/cmd/helm/downloader: no such file or directory
[ERROR] This error means the referenced package was not found.
[ERROR] Missing file or directory errors usually occur when multiple packages
[ERROR] share a common dependency and the first reference encountered by the scanner
[ERROR] sets the version to one that does not contain a subpackage needed required
[ERROR] by another package that uses the shared dependency. Try setting a
[ERROR] version in your glide.yaml that works for all packages that share this
[ERROR] dependency.
[ERROR] Error scanning k8s.io/helm/cmd/helm/helmpath: open /home/scheetz/.glide/cache/src/https-k8s.io-helm/cmd/helm/helmpath: no such file or directory
[ERROR] This error means the referenced package was not found.
[ERROR] Missing file or directory errors usually occur when multiple packages
[ERROR] share a common dependency and the first reference encountered by the scanner
[ERROR] sets the version to one that does not contain a subpackage needed required
[ERROR] by another package that uses the shared dependency. Try setting a
[ERROR] version in your glide.yaml that works for all packages that share this
[ERROR] dependency.

you're right, I was building with the old dependencies - sorry!

@scheetz yes

Helm 2.3.1 itself is on k8s 1.5.x. I'll focus on the same combination as well.


Moment date object not binding in input type time in angualrjs

We are using moment 2.14.2 version in the angularjs. Here, I'm not getting an updated value in the ng-model when I change the input type time. Here is the scenario to reproduce the issue.

Convert time from string to moment date object

angular.module('timeExample', [])
   .controller('DateController', ['$scope', function($scope) {
     $scope.startTime = moment('10:20', "HH:mm").toDate();
     //working snippet
     //$scope.startTime = new Date(1970, 0, 1, 10, 20, 0);
   }]);

<input type="time" placeholder="HH:mm" min="00:00" max="23:59" ng-model="startTime "/>
<tt>value = {{startTime | date: "HH:mm:ss"}}</tt>

This model value is undefined and doesn't get update when I change the time, but if I use javascript date object It is updating the model.
Is anyone faced this kind of issue? help me to solve or do I want to create new component/directive to watch the changes and update the model value? any idea why model value doesn't get update?

I'd suggest trying the angular-moment project.


How to call a method in another module and pass string parameter between multiple modules?

I can pass a integer param，but can't pass a char* buffer。What should i do?!

Hi, we don't support it yet. You can refer to #2919, where potential solutions are discussed. We will keep you updated once decisions are made and such features are implemented.


Remove the code enum

Follow-up to https://github.com/remcoder/gcode-preview/pull/211
The Code enum bothered me a lot. I figured how to make the interpreter work with a magic function without having to use an enum. I think Typescript did not complain in the previous implementation because enum values are the same type or subtype of the index method. I had to add an index signature to make it work with strings.
Now, to define a new gcode implementation there's only one file to touch (excluding tests). We only have to define the new method!

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#229  👈
#225 : 4 other dependent PRs (#220 , #224 , #227  and 1 other)
develop

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @sophiedeziel and the rest of your teammates on  Graphite

nice an DRY!
we could now generate the methods for T0 .. Tx! Metaprogramming! :-)


Templated Rules requires ID to create, but does not have id field in schema

When terraform applying changes to  configure templated rules, the plan succeeds without error but on apply I receive the error "ID not found".
When specifying ID in the terraform file, I get an error saying that is not a recognized key because it is not in the schema.

Nevermind, I was using REGISTRATIONSUCCESS instead of REGSUCCESS as the id, which fixed the issue


fix: if tracing is enabled, use if for logging too

When having crates which use tracing and log and have the tracing feature enabled, you'll get mixed log formats on the output:
2024-04-24T11:56:23.290168Z  INFO sea_orm_migration::migrator: Applying migration 'm0000220_create_package_relates_to_package'
2024-04-24T11:56:23.291295Z  INFO sea_orm_migration::migrator: Migration 'm0000220_create_package_relates_to_package' has been applied
2024-04-24T11:56:23.291484Z  INFO sea_orm_migration::migrator: Applying migration 'm0000230_create_qualified_package_transitive_function'
2024-04-24T11:56:23.291994Z  INFO sea_orm_migration::migrator: Migration 'm0000230_create_qualified_package_transitive_function' has been applied
2024-04-24T11:56:23.292199Z  INFO sea_orm_migration::migrator: Applying migration 'm0000240_create_importer'
2024-04-24T11:56:23.294018Z  INFO sea_orm_migration::migrator: Migration 'm0000240_create_importer' has been applied
[WARN  trustify_module_graph::graph::sbom::spdx] Replacing faulty SPDX license expression with NOASSERTION: Parsing for expression `Parsing for expression `GPLV2+,-LGPLV2+,-MIT` failed.` failed.
[WARN  trustify_module_graph::graph::sbom::spdx] Replacing faulty SPDX license expression with NOASSERTION: Parsing for expression `Parsing for expression `GPLV2+,-LGPLV2+,-MIT` failed.` failed.
[WARN  trustify_module_graph::graph::sbom::spdx] Replacing faulty SPDX license expression with NOASSERTION: Parsing for expression `Parsing for expression `GPLV2+,-LGPLV2+,-MIT` failed.` failed.
[WARN  trustify_module_graph::graph::sbom] unable to ingest relationships involving a non-fully-qualified package pkg://golang/github.com/openshift/ocs-operator
[WARN  trustify_module_graph::graph::sbom] unable to ingest relationships involving a non-fully-qualified package pkg://generic/gnulib
[WARN  trustify_module_graph::graph::sbom] unable to ingest relationships involving a non-fully-qualified package pkg://pypi/:sys-platform
[INFO  trustify_module_graph::graph::sbom::spdx] Package cache: PackageCache { cache: 8044, hits: 24040 }

This was due to the fact that we enabled the trace feature, but did not disable the default (and thus the log) feature. Having both enabled, will also enable both frameworks, one for logging, one for tracing.
Disabling the log feature however removes the log output.
I think the correct fix would be to enable the tracing-subscriber/tracing-log feature, which captures both log and tracing events, streaming both the tracing system, giving a consolidated format.
This would make the log and trace feature mutually exclusive.

Additionally, this enables the ansi feature of tracing-subscriber, to have colored output (unless the NO_COLOR env-var is set:

This would make the log and trace feature mutually exclusive.

That's a firm no-no. Features should be additive.
Perhaps if everything is transparently funneled through tracing if both features are enabled and otherwise the logic stays the same this would be workable.

Ok, so I reworked it in a way that:

when you enable log, you get env_logger.
when you enable tracing, you get tracing-subscriber
when you enable both log and tracing, you get both (logs and traces) through tracing-subscriber

Please revert spurious formatting changes, though.

That was applied by rustfmt, I was assuming that project was using it too. I can try to revert those.

Ok, this PR should now only contain the trace-logging stuff.

Thank you!


Export wakeup history

On the battery screen, where it shows the wakeup history (#922), allow this history to be exported.
For all background TS services:

[ ] Wake up start time
[ ] Wake up end time
[ ] Wake up service class name

Not going to implement.
Reasons:

Frequency of use (ex. edge case) vs effort to implement


January 11, 2022 Meeting Minutes

[x] Complete Team Contract
[x] Pick a topic
[x] Create project structure
[x] Summary paragraph to describe project (Lynn)
[ ] bulleted list of functions with small descriptions (everyone)
[ ] paragraph describing how functions fit into python ecosystem (Jacqueline)
[ ] function documentation (everyone)
Average word lengths (Kiran)
Get n most frequent words (Jacqueline)
Get count of punctuation (Lynn)
Remove stop words (Paniz)
[ ] CONDUCT.md (Paniz)
[ ] CONTRIBUTING.md (Kiran)

I will change my function to "count percentage of fully capitalised words".


Angular-carousel  & rn-carousel-controls

Using rn-carousel-controls causes TypeError: Cannot read property 'length' of undefined when using any ng filters such as orderBy in your forEach.

{{ image.name }}

If I remove either the rn-carousel-controls or the  | orderBy:'age'the error goes away.
I believe the error is on this line where we're calling scope[repeatCollection.replace('::', '')].length and that property doesn't exist on the scope so we're calling length of undefined.

Upon looking into this further it looks like the regex which is used to capture repeatItem and repeatCollection needs some work. Here's a regex tester showing what is going wrong.

i think the repeatCollection should be the final, filtered expression souldn't it ? so you get the order you asked in the carousel
can you please test with this change ? https://github.com/revolunet/angular-carousel/commit/0f2b303b37b631461881cab3ddfab1a7dc46a356

I'll try that. Another solution might be to adjust the regex to properly pull out the repeatCollection value. repeatCollection.
Something like this ^\s*(.+?)\s+in\s+(.+?)(?:\s+\S+)*?(?:\s+track\s+by\s+(.+?))?\s*$
would take this string slide in slides | orderBy:'age' track by slide.id
and assign:
Group 1: slide
Group 2: slides
Group 3: slide.id

OOh I see what you're saying. When making the regex change and removing that extra info from the repeatCollection it breaks when you use the rn-carousel-buffered b/c it omits your filtering/sorting/etc.

@revolunet Your solution doesn't work b/c its saved to the scope as the property images. So the repeatCollection = "images | orderBy:'-age'" and there isn't a scope[images | orderBy:'-age']. Just a scope.images. What if after pulling out the repeatCollection w/ with filter-components we then strip out just the collectionName. And then use the collectionName in when setting the canloop variable? I can fork it, test it, and submit a pull request.

** EDIT** I lied your solution works just fine. Do you have time to update it? If not I can create a pull request and make that change.

There will be additional work that will need done for this.. The ng-if on the left/right arrows for the controls end up looking like:
ngIf: carouselIndex < bundleDef.steps | orderBy : 'positionX'.length - 1 || false

I have a fix for this and it's pretty simple. You just need to wrap some parens around part of the line where nextSlideIndexCompareValue is set:
if (iAttributes.rnCarouselControls!==undefined) {
                            // dont use a directive for this
                            var canloop = ((isRepeatBased ? scope.$eval(repeatCollection.replace('::', '')).length : currentSlides.length) > 1) ? angular.isDefined(tAttributes['rnCarouselControlsAllowLoop']) : false;
                            var nextSlideIndexCompareValue = isRepeatBased ? '(' + repeatCollection.replace('::', '') + ').length - 1' : currentSlides.length - 1;
                            var tpl = '
\n' +
                                '   0 || ' + canloop + '">\n' +
                                '  <span class="rn-carousel-control rn-carousel-control-next" ng-click="nextSlide()" ng-if="carouselIndex < ' + nextSlideIndexCompareValue + ' || ' + canloop + '">\n' +
                                '
';
                            iElement.parent().append($compile(angular.element(tpl))(scope));
                        }

@revolunet Hey have you had a chance to review that pull request. If you have any feedback or suggestions I'd be happy to fix whatever.

This is still not working with<EMAIL_ADDRESS>

I was still having this issue too (though with Angular 1.5.5) until I installed via the following:
bower install angular-carousel#1.0.1 --save
I don't know if it was grabbing 1.0.0 without that specific target (I only noticed after that 1.0.1 is still numbered the 1.0.0 in the JS file), but this helped me, and it only takes a second to try!


[FEATURE] ZTE ZXHN F660 v9.0 Support

I would like the ZTE ZXHN F660 v9.0 router to be supported.
Link for the configuration file - https://drive.google.com/drive/u/2/folders/1cYJDqzNzU14MgI8yMwobdvhmogpGjv0e

@zhanshi123 I think that you need to access router via telnet or ssh and "download" scpd file

@zhanshi123 I think that you need to access router via telnet or ssh and "download" scpd file

Don't have access to telnet or ssh. Those ports are closed by default. factorymode-crack also doesn't work with this.

Do you have experiences with hardware mods ? There should be tty connector on board but you need multimeter and arduino uno/serial to usb adapter. Can you open case and post picture here ?

Do you have experiences with hardware mods ? There should be tty connector on board but you need multimeter and arduino uno/serial to usb adapter. Can you open case and post picture here ?

I'm not an expert but I'll give a try

@zhanshi123 I think that you need to access router via telnet or ssh and "download" scpd file

Hi, now I have access to telnet. What do you mean by scpd file. I got a copy of db_user_cfg.xml

@zhanshi123 I think that you need to access router via telnet or ssh and "download" scpd file

Hi, now I have access to telnet. What do you mean by scpd file. I got a copy of db_user_cfg.xml

you need to find cspd file on your router and copy it to your pc

@zhanshi123 I think that you need to access router via telnet or ssh and "download" scpd file

Hi, now I have access to telnet. What do you mean by scpd file. I got a copy of db_user_cfg.xml

you need to find cspd file on your router and copy it to your pc

I copied the cspd file. What should I do now?

@zhanshi123 I think that you need to access router via telnet or ssh and "download" scpd file

Hi, now I have access to telnet. What do you mean by scpd file. I got a copy of db_user_cfg.xml

you need to find cspd file on your router and copy it to your pc

I copied the cspd file to pc. What should I do now?

Open it in IDA or upload it somewhere

@zhanshi123 I think that you need to access router via telnet or ssh and "download" scpd file

Hi, now I have access to telnet. What do you mean by scpd file. I got a copy of db_user_cfg.xml

you need to find cspd file on your router and copy it to your pc

I copied the cspd file to pc. What should I do now?

Open it in IDA or upload it somewhere

I have uploaded the cspd file in this drive. Can you check it out? https://drive.google.com/drive/folders/1_rhjNL8WIHFJNJapHZTk4jaaGVLrG0Vl

Decryption successful. Follow the guide in the link. https://reverseengineering.stackexchange.com/questions/31848/decrypt-the-config-file-of-zte-f660-v9

Hello, how can I access the Telnet feature on this router? Can you share the method please?

Hello, how can I access the Telnet feature on this router? Can you share the method please?

Connect your pc to the router via Ethernet port and use the "zte_factroymode.py" tool to open the telnet port.

@zhanshi123 could you tell me how you decrypt the paramtag? Mine looks different, the only similarity is TAGH0201 at the beginning, then no MAC address or serial inside. ZXHN F670L V9.0.11P4N5E.

@zhanshi123 could you tell me how you decrypt the paramtag? Mine looks different, the only similarity is TAGH0201 at the beginning, then no MAC address or serial inside. ZXHN F670L V9.0.11P4N5E.

Open the paramtag file in a hex editor. Like in the guide I have mentioned above.

@zhanshi123 could you tell me how you decrypt the paramtag? Mine looks different, the only similarity is TAGH0201 at the beginning, then no MAC address or serial inside. ZXHN F670L V9.0.11P4N5E.

Open the paramtag file in a hex editor. Like in the guide I have mentioned above.

Sure I did it, it looks completely different.

@zhanshi123 https://bashupload.com/gktKX/paramtag

@zhanshi123 https://bashupload.com/gktKX/paramtag

I think it is encrypted. No idea how to decrypt that. Didn't you try the decode.py program with CF28772B (Serial) + your router MAC. MAC can be obtained from the router settings.

@zhanshi123 https://bashupload.com/gktKX/paramtag

I think it is encrypted. No idea how to decrypt that. Didn't you try the decode.py program with CF28772B (Serial) + your router MAC. MAC can be obtained from the router settings.

@zhanshi123 tried, doesn't work. Always returns the AssertionError.

@zhanshi123 https://bashupload.com/gktKX/paramtag

I think it is encrypted. No idea how to decrypt that. Didn't you try the decode.py program with CF28772B (Serial) + your router MAC. MAC can be obtained from the router settings.

@zhanshi123 tried, doesn't work. Always returns the AssertionError.

Open a new question at reverseengineering and someone might help.

@leshik Can you send me a sample config file from your router + router serial number and MAC address to try something out.

@zhanshi123 sure. It has an unknown payload type 6.
$ python info.py db_backup_cfg.xml
Payload Type:       6 (UNKNOWN)
Payload Start:      60
Decompressed size:  0 bytes
2nd last chunk:     0
Chunk size:         0 bytes
Payload CRC:        0
Header CRC:         0

All the files are here: https://bashupload.com/HPCfT/enIiz.zip

@zhanshi123 sure. It has an unknown payload type 6.
$ python info.py db_backup_cfg.xml
Payload Type:       6 (UNKNOWN)
Payload Start:      60
Decompressed size:  0 bytes
2nd last chunk:     0
Chunk size:         0 bytes
Payload CRC:        0
Header CRC:         0

All the files are here: https://bashupload.com/HPCfT/enIiz.zip

I couldn't download the files. Can you reupload please?

@zhanshi123 sure. It has an unknown payload type 6.
$ python info.py db_backup_cfg.xml
Payload Type:       6 (UNKNOWN)
Payload Start:      60
Decompressed size:  0 bytes
2nd last chunk:     0
Chunk size:         0 bytes
Payload CRC:        0
Header CRC:         0

All the files are here: https://bashupload.com/HPCfT/enIiz.zip

I couldn't download the files. Can you reupload please?

Try this link: https://drive.google.com/file/d/1yl9TawruAT59XtmI5488wnrcYBb-Q3fH/view?usp=sharing

@leshik Thanks

@leshik It can be decrypted. I'm needing the serial number. you can get it from router web interface.

@leshik I found it from the setmac file. Decryption is susccessful.

@leshik I found it from the setmac file. Decryption is susccessful.

How?

@leshik I found it from the setmac file. Decryption is susccessful.

How?

Download the python scripts "decode.py" and "encode.py" from here.
Your router key is 2326F0574045346843e8
Then use the below commands using the downloaded scripts.

decode config.bin to config.xml
python decode.py --key 2326F0574045346843e8 config.bin config.xml

encode config.xml to config_new.bin
python encode.py --key 2326F0574045346843e8 config.xml config_new.bin --include-header --signature "ZXHN F670L"

decode db_backup_cfg.xml to cfg.xml
python decode.py --key 2326F0574045346843e8 db_backup_cfg.xml cfg.xml

encode cfg.xml to db_backup_cfg_new.xml without signature and header
python encode.py --key 2326F0574045346843e8 cfg.xml db_backup_cfg_new.xml

@zhanshi123 Thanks man, it worked! Would you create a  pull request to this repo for adding support for payload type 6 then?

Hi
please tell me how to download scpd file and the others files with telnet

Hi
My son from web is f864b8-zteeqg2m4l00102
Is not in hex format

Hi
Finally i have do it by using tftp server but I have no locate 2 files hard code and data protocol please help me

Hi
Finaly  have do it with tftp server but i have not find 2 files location hardcode and dataprotocol
please help me

Hi
Finaly  have do it with tftp server but i have not find 2 files location hardcode and dataprotocol
please help me

You can download the config file from web interface. Go to Management & Diagnosis > System Management > User Configuration Management.
Then install the utility in this repo via python and use the above decode.py and encode.py scripts to decrypt or encrypt your config file.
Your key will be the ONT serial number (ZTEGXXXXXXXX - get only last 8 hex characters in uppercase) + MAC address of your ONT (from right to left).
If you need further support, contact me via<EMAIL_ADDRESS>

@zhanshi123 I think that you need to access router via telnet or ssh and "download" scpd file

Hi, now I have access to telnet. What do you mean by scpd file. I got a copy of db_user_cfg.xml

how did you manage to get telnet access ? i have same router

how did you manage to get telnet access ? i have same router

In my case, it defaulted to HTTPS with a self-signed certificate. Use zte_modem_tools and modify zte_factroymode.py (replace http:// with https:// everywhere, and add verify=False option).

i'll give it a try, thanks.

where should i add verify=False ?

where should i add verify=False ?

to each line where you changed http to https, e.g.
resp = self.S.post(f"https://{self.ip}:{self.port}/webFac", data='SendSq.gch', verify=False)

did not work for me, i got errors:

requests.exceptions.SSLError: HTTPSConnectionPool(host='<IP_ADDRESS>', port=80): Max retries exceeded with url: /webFac (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1091)')))

keeps looping and giving this error:

InsecureRequestWarning: Unverified HTTPS request is being made to host '<IP_ADDRESS>'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
InsecureRequestWarning,
OK!

the loop:

facStep 1:
C:\Users\yassine\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\LocalCache\local-packages\Python37\site-packages\urllib3\connectionpool.py:1068: InsecureRequestWarning: Unverified HTTPS request is being made to host '<IP_ADDRESS>'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
InsecureRequestWarning,
OK!

facStep 2:
C:\Users\yassine\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\LocalCache\local-packages\Python37\site-packages\urllib3\connectionpool.py:1068: InsecureRequestWarning: Unverified HTTPS request is being made to host '<IP_ADDRESS>'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
InsecureRequestWarning,
OK!
facStep 3:
C:\Users\yassine\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\LocalCache\local-packages\Python37\site-packages\urllib3\connectionpool.py:1068: InsecureRequestWarning: Unverified HTTPS request is being made to host '<IP_ADDRESS>'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
InsecureRequestWarning,
OK!
facStep 4:
C:\Users\yassine\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\LocalCache\local-packages\Python37\site-packages\urllib3\connectionpool.py:1068: InsecureRequestWarning: Unverified HTTPS request is being made to host '<IP_ADDRESS>'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
InsecureRequestWarning,
user/pass error
try next...
trying  user:"cqadmin" pass:"nE7jA%5m"
reset facTelnetSteps:
C:\Users\yassine\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\LocalCache\local-packages\Python37\site-packages\urllib3\connectionpool.py:1068: InsecureRequestWarning: Unverified HTTPS request is being made to host '<IP_ADDRESS>'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
InsecureRequestWarning,
reset OK!

keeps looping and giving this error:

InsecureRequestWarning: Unverified HTTPS request is being made to host '<IP_ADDRESS>'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
InsecureRequestWarning,
OK!

This is a warning, not an error. When it ends looping, there will be telnet username / password in the last line. You may also need using --user and --pass flags, or adding your modem username / password to the file.

thank you for your time, it is working now, like you said, i had to add my password in the file.

one more thing guys, how to download or upload a file to this router ?

one last thing, how to create a file inside the router ? send or download a file from the router ?

one last thing, how to create a file inside the router ? send or download a file from the router ?

You can't do changes to the file system of the router. It is read-only. To download files, plug a usb drive to the router and use 'cp' command to copy files to the usb drive.
Eg:
cp /bin/cspd /mnt/usb1_1/cspd

thanks a lot guys.

does anyone know where the HTML, JS files are in the router ? UI files ?

found it in /home/httpd/

how to decode and encode paramtag ? what is function cspd? I have cspd from onu, how to edit backup mtd from onu ?

how to decode and encode paramtag ? what is function cspd? I have cspd from onu, how to edit backup mtd from onu ?

using the setmac function, you can edit data in paramtag. After logging into telnet, use "setmac show" to view the configuration.

what is the use of paramtag ?

Paramtag includes almost all the configuration parameters like serial numbers, mac addresses, passwords, etc. Using setmac function in telnet, you can change those data as you wish. For example, you can replace your ONT with another one without informing your ISP by changing serial number of the new ONT with the old ONT's.

@zhanshi123 I think that you need to access router via telnet or ssh and "download" scpd file

Hi, now I have access to telnet. What do you mean by scpd file. I got a copy of db_user_cfg.xml

Hello would you please help to access the telnet I have the same modem

@zhanshi123 I think that you need to access router via telnet or ssh and "download" scpd file

Hi, now I have access to telnet. What do you mean by scpd file. I got a copy of db_user_cfg.xml

Hello would you please help to access the telnet I have the same modem

Connect your pc to the router via Ethernet port and use the "zte_factroymode.py" tool to open the telnet port. Use the readme file as the guide.

Hello again, is there a way to enable connection type bridge ? because there is only Routing. We can change it via HTML and we see that the connection config changes but it will change back to routing.

Hello again, is there a way to enable connection type bridge ? because there is only Routing. We can change it via HTML and we see that the connection config changes but it will change back to routing.

Bridge settings are hidden by default but you can unhide it via telnet. Send an email to me so I can guide you<EMAIL_ADDRESS>

@zhanshi123 check you email.

one last thing, how to create a file inside the router ? send or download a file from the router ?

You can't do changes to the file system of the router. It is read-only. To download files, plug a usb drive to the router and use 'cp' command to copy files to the usb drive.
Eg: cp /bin/cspd /mnt/usb1_1/cspd

For any one looking for a method to copy the files without usb. you can use the:
curl -X POST -F 'file=@/bin/ping' http://yourlocalipaddress:8000
and use these python script to receive the files on your computer:
from http.server import BaseHTTPRequestHandler, HTTPServer
import os
class ServerHandler(BaseHTTPRequestHandler):
def do_POST(self):
    # Determine the length of the data
    content_length = int(self.headers['Content-Length'])
    # Read the raw POST data
    post_data = self.rfile.read(content_length)

    # You can specify the filename or extract it from the headers if provided
    filename = "uploaded_file.bin"

    # Save the data to a file
    with open(filename, 'wb') as output_file:
        output_file.write(post_data)

    # Send a response to the client
    self.send_response(200)
    self.end_headers()
    self.wfile.write(b"File saved.")

def run(server_class=HTTPServer, handler_class=ServerHandler, port=8000):
server_address = ('', port)
httpd = server_class(server_address, handler_class)
print(f'Starting httpd on port {port}...')
httpd.serve_forever()
if name == "main":
from sys import argv
if len(argv) == 2:
    run(port=int(argv[1]))
else:
    run()

this way you can send wichever file you want after u have a telnet shell

trying  user:"admin" pass:"admin"
reset facTelnetSteps:
C:\Users\sadiq\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host '<IP_ADDRESS>'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
warnings.warn(
reset OK!
facStep 1:
C:\Users\sadiq\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host '<IP_ADDRESS>'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
warnings.warn(
OK!
facStep 2:
protocol error?
OK!
No Luck!
please help me with this im facing this issue

Hi, recently i was able to enable my telnet (thanks to you), i was lookin at config.xml file and came accross these settings, it looks like we can have QOS enabled. If so how to do that ?
<Tbl name="QOSBasic" RowCount="1"> <Row No="0"> <DM name="Enable" val="1" defval="0"/> <DM name="DefaultTrafficClass" val="1" defval="-1"/> <DM name="DefaultPolicer" val="" defval=""/> <DM name="DefaultDSCPMark" val="-1" defval="-1"/> <DM name="DefaultVlanprioMark" val="-1" defval="-1"/> </Row> </Tbl> <Tbl name="QOSClassification" RowCount="0"> </Tbl> <Tbl name="QOSPolicer" RowCount="0"> </Tbl> <Tbl name="QOSStats" RowCount="0"> </Tbl> <Tbl name="QOSQueue" RowCount="0"> </Tbl> <Tbl name="QOSShaper" RowCount="0"> </Tbl>

trying user:"admin" pass:"admin" reset facTelnetSteps: C:\Users\sadiq\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host '<IP_ADDRESS>'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings warnings.warn( reset OK!
facStep 1: C:\Users\sadiq\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host '<IP_ADDRESS>'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings warnings.warn( OK!
facStep 2: protocol error? OK!
No Luck!
please help me with this im facing this issue

just add your router's username and password in zte_factroymode.py in def parseArgs() method:
parser.add_argument('--user', '-u', nargs='+', help='factorymode auth username', default=["your_username"])     parser.add_argument('--pass', '-p', metavar='PASS', dest='pw', nargs='+', help='factorymode auth password', default=[                         "your_password"])

just unlocked my F660 V9 telnet by Port Forwarding with both port 23 on
WAN Start Port | LAN Host Start Port |
WAN End Port | LAN Host End Port

oh btw is f660 V9 are possible to run openWRT?

and btw is there a possible way to open the port 22 (ssh) ?

Hello, in the past few days i was trying to find a way to enable QOS and Trafic Shapping on this router, i found that the files already exist in: /home/httpd/webmodules/modules :

qos_basic_m.lua
qos_queue_js.lp
qos_queue_lua.lua
qos_queue_t.lp
qos_shaper_js.lp
qos_shaper_lua.lua
qos_shaper_t.lp
qos_speed_js.lp
qos_speed_lua.lua
qos_speed_t.lp
qos_statistics_js.lp
qos_statistics_lua.lua
qos_statistics_t.lp
qos_type_m.lua

The issue is, there is a script responsible of building UI Menus in /home/httpd/webmodules/config that override the main configuration that have all features enabled dmenu.lua , one of the scripts dmenu_modifier_XXX.lua is doing that. Since that partition is read only, i could not override it.
 Thanks.

Your router key is 2326F0574045346843e8

Hello, are all ZTE router has key with SN and MAC? i have test on ZTE F670 key with use SN and MAC but its say wrong key


Tweak the delete backwards behavior

Pressing and holding the delete button has a longer delay than the native iOS keyboard. This should be fixed.

I will fix this after releasing KK7.

This will be fixed in 7.1.2.


Update WEBUIHOTKEYS.md

Currently the WebUI Hotkey List displays extra plus signs. This pull request will change it back so the commands look like keys without the extra symbols so it matches the style of "Toggle Console" command.

Damn- missed this PR
@Yorzaren: The Plus sign is there since you have to press a combination of two or more keys (for example the ctrl key and the enter key). Now it is displaying not existing keys.
Reference can be found here https://squidfunk.github.io/mkdocs-material/reference/formatting/?h=keyboard#adding-keyboard-keys
@lstein @blessedcoolant can this PR be reverted?

@mauwii I don't understand? What page is now messed up after this pull request? I only wrapped the button presses in quotes to fix it because it didn't seem to be displaying right on most of the keys except for Toggle Console command which was wrapped in quotes. The other keys were displaying like ++Ctrl+Enter++ in the table instead of the keycaps.
Before pull request:

The live page at https://invoke-ai.github.io/InvokeAI/features/WEBUIHOTKEYS/ after this pull request looks like this:

@mauwii Any reason why this needs to be reverted? The docs seems fine to me?

Before pull request:

Oh - I am sorry @Yorzaren!!! This seems to be from another PR which I missed then as well, will investigate when done with the problem I am currently looking in!
This is what the keys should look like:

@mauwii Ah, I see what happened. There was capitalization of the words in the old version causing the display issues. When I corrected it with quotes, I made custom keys, instead of using the ones with the symbols. I can fix it and open an other pull request.


Use @apply in custom scss?

Is there a way to import the tailwind into application.css/scss instead of the stylesheet_link_tag "tailwind" that would let us use @apply in the rest of our custom css? (No Webpacker)

Thank you @dmarcoux. I was hoping this gem would be advanced to handle this customization for the asset pipeline by default without node/yarn but I can see the complications. 🤔


Accessing from outside of a controller

Hi, I'm trying to access my websockets from outside in another service.  Is there a way similar to how SignalR allow you to access Hubs via IHubContext?
Thanks!

Take a look at this sample here
Basically, you can have a messageHandler injected via constructor injection in any class you need.

hmm, well for my example, i'm utilizing Service Fabric which has this code in Program.cs.  Which I want to pass the handlers into the constructor of "Service", but at this point Startup.cs has yet to be called therefore the handlers have yet been stored in the IOC container... does that make sense?
`
private static void Main()
{
try
{
// The ServiceManifest.XML file defines one or more service type names.
// Registering a service maps a service type name to a .NET type.
// When Service Fabric creates an instance of this service type,
// an instance of the class is created in this host process.
            ServiceRuntime.RegisterServiceAsync("ServiceType",
                context => new Service(context)).GetAwaiter().GetResult();

            ServiceEventSource.Current.ServiceTypeRegistered(Process.GetCurrentProcess().Id, typeof(NotificationAPIService).Name);

            // Prevents this host process from terminating so services keeps running.
            Thread.Sleep(Timeout.Infinite);
        }
        catch (Exception e)
        {
            ServiceEventSource.Current.ServiceHostInitializationFailed(e.ToString());
            throw;
        }
    }

`

Would i need to do something like such inside of Program.cs?
public static void Main(string[] args)
    {
        //setup our DI
        var serviceProvider = new ServiceCollection()
            .AddLogging()
            .AddSingleton<IFooService, FooService>()
            .AddSingleton<IBarService, BarService>()
            .BuildServiceProvider();

        //configure console logging
        serviceProvider
            .GetService<ILoggerFactory>()
            .AddConsole(LogLevel.Debug);

        var logger = serviceProvider.GetService<ILoggerFactory>()
            .CreateLogger<Program>();
        logger.LogDebug("Starting application");

        //do the actual work here
        var bar = serviceProvider.GetService<IBarService>();
        bar.DoSomeRealWork();

        logger.LogDebug("All done!");

    }


tool length offset G43.1

In the offset sub tab there is a tool length offset G43.1 value box, but there is no means to send a value that you have changed, you can only set it to zero

Doh, I didn't add code to set that :-(. It would seem you are the first to use this. Workaround is f course to send G43.1 z from MDI.
Fix is tested, release soon.

MDI is what I am using. its great for phantom runs Just haven't sent you a bug report.
As your getting closer to a very polished piece of software, I will start writing down some of my thoughts. I have assess to professional CNC kit so have more exposure to see the good and the bad.

Fixed, thank you for finding this.


prepare for v0.11.1 release

[x] I agree to follow the project's code of conduct.
[x] I added an entry to CHANGES.md if knowledge of this change could be valuable to users.
[x] I ran cargo fmt

published https://crates.io/crates/wkt/0.11.1
and tagged https://github.com/georust/wkt/releases/tag/0.11.1


buildrefs source=commons doesn't work

The problem seems to be that OAuth isn't performed successfully.
Self contained broken example to ask the Wikimedia search platform people about https://www.mediawiki.org/wiki/Wikimedia_Search_Platform :
https://pastebin.com/nq9aLSYL
Need to figure out how to contact them

https://stackoverflow.com/questions/65303450/how-to-authenticate-to-wikimedia-commons-query-service-using-oauth-in-python

Answered: https://stackoverflow.com/a/65719927/678387

Answered: https://stackoverflow.com/a/65719927/678387


[Event Request] Table 37 "Sales Line" - CurrFieldNo

Please add the following Event CurrFieldNo in Table 37 "Sales Line"
        field(38; "Appl.-to Item Entry"; Integer)
<...>
                    "Location Code" := ItemLedgEntry."Location Code";
                    //>><< New event
                    IF CurrFieldNo <> 0 THEN
                        if not ItemLedgEntry.Open then
                            Message(Text042, "Appl.-to Item Entry");
                end;
            end;

Hello @AdrijaSipaviciute
Could you please review your request, as it seems like that you did not set all needed information.

Hello @AleksandricMarko
Thank you for noticing. Made some changes, should be fine now.

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 12501.


Not all software for building packages is included in repo

Some software like drivers are large (500mb+), and not included in this repo.
Something needs to be done to improve this experience. Whether it is simply updating the documentation to better inform users of this, adding scripts or steps to download necessary files, etc.

I think I want to not include software in the repo anymore. This includes .cab files.

This will be marked as closed in #41. I'm satisfied with this for now. Moving forward however, I would like more documentation and maybe automation for automatically downloading software required by the predefined packages in this repository. That can be a separate issue however.


[eas-cli] [ENG-9957] Don't overwrite distribution for simulator builds

Why
https://linear.app/expo/issue/ENG-9957/fix-internal-distribution-pages-for-simulator-builds
Companion for: https://github.com/expo/eas-build/pull/283, https://github.com/expo/universe/pull/13524, and https://github.com/expo/universe/pull/13523
How

distribution value is kept instead of hardcoded to simulator if simulator flag is set, keeping its original value
simulator field has been added to metadata to retain its value

Test Plan
Existing tests pass
Deployment

www
website
cli

/changelog-entry chore Add simulator flag to metadata

Added screens for comparison


Memory leak of CollapsedCodeCellExecutionIcon

Open a long notebook
Scroll up and down multiple times (say a dozen)
Close the notebook
Dump the memory (from the dev tools)
There are hundreds of CollapsedCodeCellExecutionIcon left

Thanks, is it fixed now with that commit?

@roblourens there are still a couple left (the ones which are rendered in the view before the editor is closed), which seem to be still linked by some handlers. Didn't dig into who is leaking so we can take a look in debt week.

same here + gpu and cpu leak.

@byehack this issue is specific to the notebook cell status bar leaks, for your issue, can you please follow https://github.com/microsoft/vscode/wiki/Performance-Issues to provide more info?

The rest of the leaks are from unreleased CodeEditorWidget which are hold by drop handler, fixed via https://github.com/microsoft/vscode/commit/7d4064191ba7e4c7922f319ccdabd174b1900752


[HUDI-4399][RFC-57] Protobuf support in DeltaStreamer

What is the purpose of the pull request
RFC for adding support for Protobuf in the DeltaStreamer along with a Protobuf+kafka source for the delta streamer
Brief change log
(for example:)

Modify AnnotationLocation checkstyle rule in checkstyle.xml

Verify this pull request
(Please pick either of the following options)
This pull request is a trivial rework / code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end.
Added HoodieClientWriteTest to verify the change.
Manually verified the change by running a job locally.

Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

cc @veenaypatil to review as well


[may I ask] (Chapter 3) a very bumpy precision curve

Dear Aurélien, I'm following your code for Chapter 3 to plot the precision and recall vs threshold curve as shown in line [36] of your jupyter notebook.
I understand that my output may not be exactly the same as yours, but the top-right part of the precision curve I plotted is "wildly" bumpy as compared to yours as shown below.
May I ask if this indicate that there are errors in my code while replicating yours?
Or, this could be possible for different run of the same data set?
My jupyter notebook is shown here.
Thank you.

@hopezh my precision curve looks like yours. Even more than the zig-zags, I worry about the more qualitative difference that they don't intersect at threshold = 0.

@EricThomson Thanks for your reply.
I'm not aware that the two curves are to be intersecting at the position where threshold=0.
May I ask if you can point me to some reference to learn more about it?

@hopezh I searched some yesterday and couldn't find anything on that. Now that I've read the rest of the chapter I'm now thinking that my intuition was just flat-out wrong, that result was a coincidence.  For instance, I just re-ran things with random_state = 20 and they intersect a little above zero. So my intuition seems misplaced, it seems just a noisy thing.
Also, in practice people typically just focus on precision-recall curves (Figure 3-5 in the book), so I will probably focus on building intuitions about those for evaluating classifiers (and ROC curves, which in my field are the most standard tool).

Hi both,
There is two possible reasons I can think off.
The first is the random_state argument. If you open his github notebook on the topic and see if you've replicated the model exactly the same.. It may differ slightly from the book.
Secondly, their are two ways to access the Mnist dataset. There is a chance you used load_digits() and not fetchml?
One produces digits that are 8x8 and the other is much larger something like 28x28
Hope this helps

I having the same results in the same bumpy precision curve in the plot.

I also get the same bumpy precision curve. We may not be using the same dataset the author used at the time... Maybe there were some updates/changes to the data.

Installing the sklearn version 0.20.0 resolve all the problems, I get the same results as the book in the entire notebook.

Sorry for the long silence, I've been busy moving to New Zealand, but with the lockdown I now have some time to unpile all the pending issues. As @EricThomson guessed and @mauricio-ms demonstrated, the latest Scikit-Learn made some internal changes so the results are not exactly the same as in the book. Same for TensorFlow by the way (in version 2.1, they entirely changed the way random numbers are generated). So you should not expect things to be exactly the same, but hopefully they'll be in the same ballpark.


Revert "Use latest instead of release for butane"

This reverts commit c4158a61870825086c5119dd67a37e7797779ed5.
There has been a 0.15.0 butane release with aarch64 payload, so we can
revert this commit as planned:
« When there's a new butane release, we can switch back to using
release instead of latest »

/lgtm


private instance: fewer associations than expected (18.06)

Summary from support email conversation. Entire thread has been forwarded to core and is available in mail365. Talk to @deniseOme if you want to access the original mail365 messages.

partner ran into memory issues when loading the data (18.06 JSON)
we recommended them to disable the global stats and provided the settings
memory issue solved
however results did not match the snapshot: they got fewer associations
files provided with the numbers (DEV: data loaded via JSON files; PROD: from a server where they loaded the snapshot)

index_comparison_DEV_PROD (2).xlsx
We believe that the discrepancy of DEV x PROD could be due to a bug we had in our 18.06 release, which affected the number of associations (fewer than expected).
Latest reply from us on the 31st Aug:

we requested the following details:

the pipeline commit used to produce the data
the command line used to run the evidence step
and the association step used to run the pipeline

@ElaineMcA @mkarmona, more details to follow.

@mkarmona @ElaineMcA, here are the details from the user:

user executed the pipeline using the docker image built based on this commit:
https://github.com/opentargets/data_pipeline/commit/526d94ae6258c98b7d2b3c69305d0ce2c8853070

these were their steps:

checkout commit above
made the changes to GLOBAL_STATS_* as discussed in previous emails
build image: docker build -t mrtarget-image .
evidence up to ddr steps are executed using the script below:

#!/bin/bash
DATA_RELEASE_VERSION=18.06
mkdir -p /ot_files/logs/$DATA_RELEASE_VERSION
echo "Running post processing steps" | tee /ot_files/logs/$DATA_RELEASE_VERSION/log_post.txt
steps=(evs
as
qc
sea
ddr)
for step in ${steps[]}
do
echo ""
echo "*"
echo "Running step --$step for data release version $DATA_RELEASE_VERSION"
echo ""
echo ""
docker run --rm --net=opentargetsautomation_ot-net
-e ELASTICSEARCH_NODES=opentargetsautomation_es-ot_1:9200 -e ELASTICSEARCH_NODES_PUB=<IP_ADDRESS>:39200 \
mrtarget-image mrtarget --$step --log-level INFO $DATA_RELEASE_VERSION | tee -a /ot_files/logs/$DATA_RELEASE_VERSION/log_post_$step.txt
PS: user thanked the invitation to come over and work on the pipeline together. They think this is a good idea, and would be happy to come over for a few days. User will propose this to PIs.

Latest:

Hi ????,
I have included in a previous email (please check the thread below) that "We believe that the discrepancy of DEV x PROD could be due to a bug we had in our 18.06 release, which affected the number of associations (fewer than expected)".
Hence our latest suggestion that the bug should no longer be in the 18.08 files.
Thanks and kind regards,
Denise
From: >
Sent: 11 September 2018 12:58
To: Open Targets Support
Cc:
Subject: Re: Question regarding loading all .json files
Hi Denise,
I think it would be useful to have some explanation on why you think the issue would no longer be there. E.g. are there specific problems you identified and fixed between June and August that could explain the observed behavior?
I just want to avoid getting into a trial and error cycle without a reasonable expectation for success, since running this pipeline takes quite some time.
Thanks,
????
2018-09-11 12:20 GMT+02:00 Open Targets Support<EMAIL_ADDRESS>Dear ??,
Thanks for the reminder.
We have had a new release (18.08) and we were hoping that if Pieter updates his code to the latest version, the issue would no longer be there.
??, could you try that please?
Thanks,
Denise
From:
Sent: 10 September 2018 21:48
To: Open Targets Support; Pieter Lukasse
Cc:
Subject:
Hi Denise and OT Support,
Any update on the issues ???? reported?  At ????, we are eager to start the effort of incorporating internal data into ??? hosted instance.  But first of all, I want to make sure that we can replicate the processing of JSON files from public data sources using the same pipeline.  So I greatly appreciate your assistance in resolving the discrepancies ???? reported.
From: Open Targets Support<EMAIL_ADDRESS>Sent: Friday, August 31, 2018 3:23 AM
To: ???
Cc: ????
Subject: Re: Question regarding loading all .json files
Thanks ??? for providing this so promptly.
I have shared the details with my colleagues and will get back to you as soon as we can.
Glad to hear you may be able to pay us a visit for an intensive pipeline running and bug troubleshooting here on the Genome campus.
Cheers,
Denise

sounds ok to me.
On Tue, 11 Sep 2018, 13:33 deniseOme<EMAIL_ADDRESS>wrote:

Latest:
Hi ????,
I have included in a previous email (please check the thread below) that
"We believe that the discrepancy of DEV x PROD could be due to a bug we had
in our 18.06 release, which affected the number of associations (fewer than
expected)".
Hence our latest suggestion that the bug should no longer be in the 18.08
files.
Thanks and kind regards,
Denise
From: >
Sent: 11 September 2018 12:58
To: Open Targets Support
Cc:
Subject: Re: Question regarding loading all .json files
Hi Denise,
I think it would be useful to have some explanation on why you think the
issue would no longer be there. E.g. are there specific problems you
identified and fixed between June and August that could explain the
observed behavior?
I just want to avoid getting into a trial and error cycle without a
reasonable expectation for success, since running this pipeline takes quite
some time.
Thanks,
????
2018-09-11 12:20 GMT+02:00 Open Targets Support
<EMAIL_ADDRESS>Dear ??,
Thanks for the reminder.
We have had a new release (18.08) and we were hoping that if Pieter
updates his code to the latest version, the issue would no longer be there.
??, could you try that please?
Thanks,
Denise
From:
Sent: 10 September 2018 21:48
To: Open Targets Support; Pieter Lukasse
Cc:
Subject:
Hi Denise and OT Support,
Any update on the issues ???? reported? At ????, we are eager to start the
effort of incorporating internal data into ??? hosted instance. But first
of all, I want to make sure that we can replicate the processing of JSON
files from public data sources using the same pipeline. So I greatly
appreciate your assistance in resolving the discrepancies ???? reported.
From: Open Targets Support<EMAIL_ADDRESS>Sent: Friday, August 31, 2018 3:23 AM
To: ???
Cc: ????
Subject: Re: Question regarding loading all .json files
Thanks ??? for providing this so promptly.
I have shared the details with my colleagues and will get back to you as
soon as we can.
Glad to hear you may be able to pay us a visit for an intensive pipeline
running and bug troubleshooting here on the Genome campus.
Cheers,
Denise
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/opentargets/platform/issues/109#issuecomment-420257734,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFJPOBfOXF33Yw0plfQe1AJwuoSZm0IYks5uZ62mgaJpZM4WU03R
.

@mkarmona @ElaineMcA
Latest reply from user:

yes, I saw that email. In this same email you also mentioned your colleagues would investigate further if I send the details regarding the exact version I used (which I did in my reply). So, in my last email I was referring to the results of this further investigation by your colleagues, since I have not received any updates on that yet. I was hoping to receive links to the issue and to the git commit that fixes the issue (these details help me understand the pipeline better and could even help me troubleshoot faster in case of a similar issue in the future). Thanks for your help!
I understand that having a new release would seem a waste of time trying to troubleshoot something going wrong with the new release; but it'd be useful to try to investigate the user's issue based on the specs they had provided.
If you think it's not worth it, I can get back to the user and thank them (again) for the specs and apologise for not having had a chance to investigate the issue and say we won't be able to investigate it.

Adam, would you mind please have a look at this issue and the issue
reported on GitHub pipeline about Pieter having less data than expected
with associations. I think it could be related to the bug I created when I
added the modification about removing routing and number of shards. he
provides all needed information to reproduce his cause
thank you.
On Wed, 12 Sep 2018, 12:46 deniseOme<EMAIL_ADDRESS>wrote:

@mkarmona https://github.com/mkarmona @ElaineMcA
https://github.com/ElaineMcA
Latest reply from user:
yes, I saw that email. In this same email you also mentioned your
colleagues would investigate further if I send the details regarding the
exact version I used (which I did in my reply). So, in my last email I was
referring to the results of this further investigation by your colleagues,
since I have not received any updates on that yet. I was hoping to receive
links to the issue and to the git commit that fixes the issue (these
details help me understand the pipeline better and could even help me
troubleshoot faster in case of a similar issue in the future). Thanks for
your help!
I understand that having a new release would seem a waste of time trying
to troubleshoot something going wrong with the new release; but it'd be
useful to try to investigate the user's issue based on the specs they had
provided.
If you think it's not worth it, I can get back to the user and thank them
(again) for the specs and apologise for not having had a chance to
investigate the issue and say we won't be able to investigate it.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/opentargets/platform/issues/109#issuecomment-420618680,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFJPOFKsfPuMMHANVc_ekxBAL4Hlu6nqks5uaPQKgaJpZM4WU03R
.

Sure, I'll dig into this a bit - though I'll need a bit of time as I'm not familiar with the global stats yet.
Some of the steps use an API, or the "latest" release, of a resource, so as that resource updates the steps will produce different results. We're working on using JSON files that we can redistribute to ensure better reproducability. This includes:
18.06_efo-data
18.06_mp-data
18.06_reactome-data
18.06_uniprot-data
This might have then had downstream effects on other steps - for example, 18.06_evidence-data-generic depends on 18.06_efo-data.
However, I agree that this doesn't explain the difference in associations. I'll try to reproduce the steps and see how it goes.

Okay, after looking into this further, it seems that there was a bug between 526d94ae6258c98b7d2b3c69305d0ce2c8853070 and f4ed7dca45c6671e46fec3030e408af1fd10536d related to how the Elasticsearch configuration was managed (routing of documents and number of shards). Unfortunately, the version your using is in the middle of that window, and so it suffers from this bug.
If you update to the more recent version (526d94ae6258c98b7d2b3c69305d0ce2c8853070) then it should be then you should get all the associations into Elasticsearch correctly. You'll still need to disable global stats to operate with lower memory requirements however.
If this problem persists for you with the alternative version, please open a new issue.


[otbn] Rename AddWithCarry and SubtractWithBorrow to make width explicit

As Felix suggested on PR #3658, it would be good to change the names of these functions to include the width. This applies to the spec and the ISS.

@rswarbrick Is this issue still relevant with the recent refactorings?

Probably not. Let's close it.


[Product Topics] Takeaway (Purple block component)

This task has been completed as part of 334-product-topics-sidebar-layout.

Marking as completed.


feature: 插件管理支持版本标签 (closed #732)

Codecov Report

Merging #986 (6417b25) into V2.2.20-rc (8ea0952) will increase coverage by 0.35%.
The diff coverage is 79.96%.

@@              Coverage Diff               @@
##           V2.2.20-rc     #986      +/-   ##
==============================================
+ Coverage       72.16%   72.52%   +0.35%
==============================================
  Files             303      316      +13
  Lines           22249    22722     +473
==============================================
+ Hits            16055    16478     +423
- Misses           6194     6244      +50

Impacted Files
Coverage Δ

apps/backend/plugin/views.py
39.94% <0.00%> (+2.71%)
:arrow_up:

apps/node_man/views/plugin.py
58.33% <0.00%> (+6.60%)
:arrow_up:

apps/utils/exc.py
82.75% <ø> (-0.58%)
:arrow_down:

config/default.py
81.17% <ø> (ø)

apps/backend/plugin/handler.py
39.28% <25.00%> (-19.26%)
:arrow_down:

apps/node_man/tools/plugin_v2.py
78.37% <42.85%> (-12.10%)
:arrow_down:

apps/node_man/handlers/policy.py
82.75% <54.54%> (-0.63%)
:arrow_down:

apps/core/tag/permission.py
60.00% <60.00%> (ø)

apps/utils/orm.py
79.00% <68.96%> (-14.03%)
:arrow_down:

apps/backend/subscription/steps/plugin.py
48.70% <72.72%> (+0.57%)
:arrow_up:

... and 27 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


joinRelation method without alias

Hi,
I need help to create a join sql query using only relation name just like joinRelation method in 0.5.0, but without aliasing table name. How can I do that?
I used other alternatives, but:

Eager methods embeds related data in parent data.
joinRelation (0.5.0) adds alias.
innerJoin needs join details, which is already defined in relation mappings.

Is there a method already available, which I miss? If not, how about adding an option object to joinRelation
{
    // true: adds alias,
    // false: uses original table name
    // string: uses given custom alias.
    alias: true,
}

There is no method available for this. Your suggestion is good! Let's do that.

That's great. Thanks.


[Feature][Dashboard] implement application topology

Show the application and all its components and traits in elegant topology.

I suggest to use some libraries from https://github.com/ahmetb/kubectl-tree

we should also support it in command line for vela app tree <appname>

almost done but still in need of merging dependency pr of oam controller.


[$1000] Web - Unexpected error displayed after adding a comment

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: <IP_ADDRESS>
Reproducible in staging?: n/a
Reproducible in production?: n/a
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: @jliexpensify
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1706069427229519
Action Performed:

Login in NewDot
Select the Workspace chat
Add a comment

Expected Result:
User is able to add comment without any error
Actual Result:
Unexpected error displayed after adding a comment
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos
Add any screenshot/video evidence

View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~0186123ce8bf2e2147
Upwork Job ID:<PHONE_NUMBER>580895744
Last Price Increase: 2024-02-19

I can't reproduce. No proposal yet

I'll buzz #expensify-open-source on Monday to drum up proposals.

Increased the bounty

Issue not reproducible during KI retests. (First week)

Could it be related to the Category? I do not have that option. Maybe the Service doesn't like it?
Additionally,  I'm clueless how to set up this reimbursement report as well as get the Department options.

I can't reproduce this issue with test step on the OP. @jliexpensify Could you help to detail the previous step? how we create that report and what did we do before?

I would say that we should close this - honestly, these Unexpected Error GH's are really frustrating as they happen once and generally only to a single user and are near-impossible to reproduce. I also can't reproduce anymore.
cc @johncschuster

Let's close this one

not overdue. Wait to close

Just caught up on this! Yeah, I agree we can close the issue. Thanks!


re-enable-Welcome-page-e2e-test

Signed-off-by: Chunxi Luo<EMAIL_ADDRESS>https://github.com/open-cluster-management/backlog/issues/13672
https://github.com/open-cluster-management/grc-chart/pull/45

/lgtm


Deploy a SimpleLogin instance

SimpleLogin for mails

Cancelled because it requires essentially the same commitment as a full blown email server and I am not ready to commit to that + all emails would still use my domain so that wouldn't really make sense as an anonymous solution


fix: don't fail parse if called again with identical input

It should be possible to use nuqs to parse search params in multiple places in the same request (e.g. page + generateMetadata) - as long as the input is identical (which it should be).
This change stores the original searchparams object passed to parse alongside the parsed data. If parse is called again we check for referential equality and if identical we just return the already cached data.
note: that this will still fail if a different object with the same contents is passed. fortunately next.js uses the same object for search params in the same request so it solves the original use case. I didn't want to get more clever and do deeper comparison for simplicity/perf.
i stored the original object as a new field in the existing cache object (not exposed to users in the type), but if this isn't ideal one option would be to create a separate cached object just for metadata like this. happy to adjust.
see https://github.com/47ng/nuqs/discussions/556
Testing
I added some tests by lightly mocking React's cache. It's not a very clever mock but it works. If maintainer would rather not have me add these with this approach I can remove them.

Actually I just realised you have e2e tests setup, let me update them too.

Actually I just realised you have e2e tests setup, let me update them too.

added a simple one


Issue #735: fix py27-x64 unit tests for win32

A quick fix to get win32 tests going. A better fix would be just align with the master.

Better squash on merge, to remove temp debug commits.

Closing since merging PR #742


Fallible Instruction::virtual_address

Allows callers which don't support every register to gracefully handle the unsupported cases, while indifferent callers, or callers with complete support, can opt into the current behavior by calling unwrap().

Thanks, merged!


Feature Request: Use machine learning models to estimate device position

I've noticed that the current algorithm has a few problems. Devices will bounce around the room randomly and sometimes appear in other rooms for a few seconds. The floor detection is also quite unreliable for me.
I think a machine learning model could probably do a much better job at estimating device positions and which floor they are on. It would be great if I could go around my house and take multiple measurements at various points. Some where I am holding my phone, some where it's in my pocket, etc. Then use this data to train a model to estimate my position in the house. I think I'm collecting more than enough data to do this accurately and reliably (16 ESPresense nodes for 4 devices), but I just need to train a model to filter out some of the noise.
Has anyone worked on something like this before? Is there a plugin or fork that I could try out? (I remember reading a blog post a while ago from someone who did something like this, but I don't think it was related to ESPresense.)

Yes, this is absolutely a thing to try.  I'm often reading papers and making new ILocators to try it out...  The bigger issue is you need a training set.  And it would need to be per floorplan.  How do you think we could do that?

I've actually built something that does that... as a byproduct while debugging my room classification model: It's an iOS app (https://github.com/gunnarbeutner/OpenUWBDemo - currently requires an Estimote UWB beacon, but I guess it could be made to not require that for some loss of positioning accuracy) that collects X, Y, Z -> RSSI pairs using ARKit/Nearby Interaction to track the device's position and an MQTT client to fetch the RSSI data:
The problems an app like that would have to solve (which mine sort of does... just not in a user-friendly way):
Align ARKit's coordinate system with the floor plan:

Could probably use the compass for the rotation, I just haven't really gotten around to make that work with NISession. Right now your phone just better be in the XZ plane facing towards positive X when the app starts.
For the translation part I just re-align the coordinates with the closest ESPresense node (based on RSSI) when the user taps the view.
For the Z coordinate: Calibrate that by having the user touch the floor and remember the lowest Z coordinate.
Both coordinate systems are already in meters so we don't need to scale them.

Collect/Output the data. It just spams location/RSSI pairs to the debug console which I then manually turn into a CSV file.
Here's an example map for one of my nodes:

The dataset is here, if you want to play around with it yourself (I found CloudCompare quite useful to visualize the point cloud):
nodes.csv
points.csv
As expected there's a sudden drop in signal quality between walls and other metal/concrete obstacles.
Personally I'd love to see a machine learning model or some kind of radio map - though calibrating that definitely requires more effort than just placing a few nodes here and there.
However, as a somewhat easier to attain step towards that maybe walls and other objects could have a tuneable absorption factor or fixed path loss ("+5 dBm for passing through this wall")? That would require ray-casting for the error function though which I'm sure is great for performance. Then again I guess you could pre-compute the per-node error map.

Thanks, I appreciate it!
UWB nodes would be kind of neat. A somewhat more accessible solution (for now anyway) would be using a second iPhone or an Apple Watch as a stationary UWB anchor. I'll probably add that to the app in a bit.
It's also now tracking the camera normal for each reference point. I figured that could be useful to exclude points where the camera is facing away from the node (i.e. when the user is between the phone and the node).
Oh, and there's a column for the Bluetooth channel.
points.csv
As for the model itself, well I don't know yet. I'd prefer something that's specific to the environment rather than to each node or worse... a collection of nodes.

So, just a quick update on some of the things I've tried - and oh boy, there are a lot, most of which just didn't pan out:

I'm pre-processing the samples by creating separate point clouds for each node and then combining those point clouds based on a simple nearest neighbor search. Each point in the new cloud has scalar features for each of the nodes' RSSI measurements:

https://gist.github.com/gunnarbeutner/5b57a5e7f1cf31388669b689fbb24019
It would make a lot more sense to combine the points based on location and timestamp, that's on my TODO list.

As a visual aid I'm applying bilateral filtering to the combined point cloud (basically: RSSI -> distance, apply filter, distance -> RSSI). The resulting cloud shows the signal propagation characteristics for our apartment quite nicely and intuitively seems to make sense (i.e. walls and air ducts attenuate the signal where you'd expect them to).

https://github.com/ESPresense/ESPresense-companion/assets/388571/c9c5cfd0-e1bb-430c-a3ac-3bb9d5170102

I've tried to train a single model which predicts x, y, z for each measurement based on the RSSI measurements:

RSSI values -> x, y, z
RSSI values converted to distances -> x, y, z
Difference in RSSI values (i.e. a - b for all combinations of nodes) -> x, y, z
Ratio of distances (i.e. a / b for all combinations of nodes) -> x, y, z
The idea behind using differences/ratios was to make the model somewhat device-independent, i.e. due to differences in 1m RSSI. Ideally this would also filter out constant attenuation (e.g. when my phone is in my backpack vs. when it isn't).
I've also tried adding the room classification as an additional output in an attempt to get the model to put more emphasis on room boundaries (2nd attempt: custom loss function that penalizes incorrect room predictions). Predicting x=1.0 vs x=1.1 is a more significant error compared to x=2.0 vs. x=2.1 when there's a wall at x=1.05.
That hasn't really worked well so far. The model has quite a bit of trouble generalizing to all the rooms. Where I'm currently at is two sets of models:
One classifier that predicts which room a measurement was taken in. And then one regression model for each room that tries to predict the location.
So, does it work? Well, it's definitely better at differentiating rooms, especially at some specific "trouble spots", i.e. where placement of nodes is limited due to the apartment layout. You wouldn't really need precise reference point measurements to train a room classification model. Also, the per-room location predictions seem to be better. The predicted locations are more uniformly distributed, whereas the optimization-based algorithm tends to favor locations closer to the nodes or closer to the rooms' center.
I'll have to do a lot more tests to see if this is better overall though. For now I haven't really run the model continuously for more than a few minutes at a time.
This could probably be expanded by having "areas of interest" rather than just rooms: Most of the time I don't really care about the X, Y, Z position, just knowing whether a device is in a specific zone would be enough. I'm currently using Home Assistant to do sensor fusion with ESPresense + motion/occupancy/door sensors to track who's occupying which rooms:
a) Room A is occupied and according to ESPresense my phone/watch is there: If HA doesn't already know my current room it initializes the state by picking room A.
b) If an adjacent room becomes occupied (motion/occupancy sensor, door is opened, etc. - there's both a door contact and a motion sensor for each room "barrier" so I know when someone's entering/leaving a room) HA assumes I moved to that room. Each room also has a list of occupants. HA removes me from room A's list of occupants after a timeout.
That's the easy part and ESPresense is only really needed to figure out where I'm at the start. Tracking room changes is instantaneous.
It gets more complicated when I enter a room that was already otherwise occupied: Did I just enter the room or did the other person leave the room (i.e. enter the room I was in)?
Again, this is easier to differentiate when none of my (or the other person's) devices are moving (which would make a lot of sense, but isn't actually something I'm doing atm, because my HA config for tracking occupants is already a convoluted mess, 800+ lines not including general room occupancy and motion detection stuff. Once this room classification/location regression model works reasonably well I'll probably get rid of the HA config and move the functionality into a separate daemon). Otherwise HA will wait for ESPresense's room predictions to settle down (i.e. we're back to step A). HA also somewhat filters out incorrect predictions because it knows which rooms are occupied (thanks to mmWave occupancy sensors and some simple heuristics: the room must still be occupied when it was previously occupied and the door hasn't been opened, or - if the door is open - the door's motion sensor hasn't been triggered). Devices can only move between rooms when both rooms are occupied and the barrier is traversable. There is a fallback with a much larger timeout to allow such moves so devices don't incorrectly get stuck in the wrong rooms.
Which... all kind of begs the question: Do I even really want X, Y, Z coords at all (at least for my use case)? Hmm.

Two more ideas:
The model allows for directional antennas, in fact it would probably make predictions better in general.
Reinforcement learning might be an idea for either predicting the location or learning to predict when a device is about to transition between areas of interest.

My idea (that I haven't tried) is to just use a model for the rssi->distance calc.  We can use the current data that is running the optimization as training for the model and then just use the current method for x,y,z.


Added a bunch of more configs for appveyor

Taken from this thread here https://users.rust-lang.org/t/rust-appveyor-windows-ci-scripts/4437 and this repro https://github.com/starkat99/appveyor-rust
It adds building on a combination of stable/unstable/nighly with msvc, gnu, etc. It adds a bunch of extra time for each build but increases coverage quite a bit.

Thanks for this! Even though it's slower I'm a huge fan of running all the tests we can.

:+1:

Unless there's pushback, or someone else gets to it first, I'll add back that section in a new PR in a bit.

Sorry @ryanhiebert , I merged before seeing that message! I'm still quite unfamiliar with appveyor, could you fix this?

gotcha, sounds perfect; make a PR when you can and I'll merge it. Thanks!

@yupferris will do. You can be forgiven for not seeing the message before you merged. I hadn't written it yet. Unless you have a time machine ;-)


Translations are not properly working.

When I change language settings I see that some part of site is translated, but navigation bar is not. Do you have same issue?

Thanks, fixed.
Please check.


Friday the 13th

Before jumping into Pr first comment for assign.All Problem from Edabit. Link is mandatory to add
Problem statement: Given the month and year as numbers, return whether that month contains a Friday 13th.
Problem Link: https://edabit.com/challenge/Xkc2iAjwCap2z9N5D

Before submitting a PR please Check some details.Check Edabit for problem task

[ ] fileName formet should be in camelcase
example- returnTheNextNumber.py
[ ]  Add label for Problem
[ ]  In file there should be problem statement and link to probelm
example -

'''
    Probem Task : This program will add two numbers
    Problem Link : https://edabit.com/challenge/SFzHtm63XT6EYNHWY
'''
*/
def main():
    return a + b

This repository is vaild for HacktoberFest2020
Read before making PR - How to contribute on Github
Good Luck,

Pleaes assign me for this.

can u assign this problem to me.

[ ] Task assigning to @VaderX
Best of Luck

I would like to work on this.


dependencies update

updated deps and fixed tests which failed due to regression in serde codegen

Seems to work, but now I get the same error with ethstore, I guess it needs a similar pr:
   Compiling ethstore v0.1.0 ()                                   │
ethstore/build.rs:33:33: 33:46 error: mismatched types [E0308]                                       │
ethstore/build.rs:33         serde_codegen::register(&mut registry);                                 │
                                                     ^~~~~~~~~~~~~                                   │
ethstore/build.rs:33:33: 33:46 help: run `rustc --explain E0308` to see a detailed explanation       │
ethstore/build.rs:33:33: 33:46 note: expected type `&mut syntex::Registry`                           │
ethstore/build.rs:33:33: 33:46 note:    found type `&mut inner::syntex::Registry`                    │
error: aborting due to previous error

I'm on it


Working code of Exceptions problem at Hackerrank

Hi there!!!
Thank you for having a very good repository. I have submitted python program on Exceptions problem at Hackerrank, in Python3.
please do have a check, and run in compiler directly.
Thank you!!!!

Hey there,
Added a working code on the problem Incorrect Regex at Hackerrank. Please run the codes and do the needful. This is my first Hacktoberfest. Thank you. I am 100 percent sure that the codes are working and passing test cases

Hey there,
The codes are fine. You can check at Hackerrank compiler. They work perfectly well. Please allow my submission to be merge it and allow me to take part in Hacktoberfest.
Thank you again for being kind, humble and generous..!!


Question about eventColorFor is not working even though implemented FSCalendarDelegateAppearance

My information:

language :  Swift 3
Version fscalendar : 2.7.3
Drag a fscalendar on the contentview of tableviewcell.
implemented FSCalendarDelegateAppearance but the eventColorFor is not working (actually it is strikethrough) and that the reason why it not show event dot on UI.

Please support.
Thanks
The following informations are requested in a bug report

A brief bug description.
Stack trace.
Integration method.(manually/cocoapods/carthage)
Full steps to reproduce.
Device modal and iOS version.  e.g. iPhone 6s iOS9.1
Xcode version.  e.g. Xcode 8.1
FSCalendar version. e.g. FSCalenda 2.5.1
Does this happen in the demo project? Which one? Or a link to another demo project.

如果在使用过程中遇到问题，请提供以下信息以便于尽快将问题修复

简单描述一下问题。
控制台输出的堆栈跟踪（如果有的话）。
安装方式. 如手动安装、cocoapods安装、或者通过carthage安装。
能够重现此问题的完整步骤。
设备型号及版本。 例如： iPhone6s iOS9.1
使用的Xcode版本。 例如： Xcode8.1
FSCalendar版本. 例如: FSCalendar 2.5.1
是否能在demo中重现，以及在哪个demo中重现？或者附加一个重现问题的demo链接。

Same problem over here! Seems like there is no implementation down to the framework. Is there any update on this?


Bump version to 4.2.0

@camilonova could you make a new release to PyPI?

@aleksihakli He received a mail about the tag you created, no need to open a separate issue.


[Suggestion] Shatter spell has a consistent result

Currently, the Shatter spell will randomly break generally 1-3 blocks but this can be very difficult to try to actually use practically as it keeps mining lower than I want in one use and higher for the next one. I would recommend that it simply mines the block it hits and one face directionally down from the face you aimed at.
Example:
[c] is a block of cobble
[x] is blocks to be broken
[X] is block to be broken and aimed at
[c] [c] [c]
[c] [X] [c]
[c] [x] [c]
The above works horizontally as well as it would mine the center block and the one closest to the player effectively averaging the random block break to 2 and making a simple 2 block tall player path and also allowing a quick and easy stairway to be dug as well.

Honestly, I'm for this.

If changes are made this is just a reminder that the text in the book would need to be updated as well as it talks about no pattern being discerned.

Do you mean like this?

or like this?
https://i.imgur.com/c5qoLcb.gif

I guess neither but I like both better than the existing. >.>
To use an existing mechanic for comparison:A Tinkers Construct hammer will mine a 3x3 area depending on the face at a 90° angle. I am talking about the same manner but a 1x2.

So, if I understand properly:

if your facing to the block is DOWN but your horizontal facing is NORTH, it would break the block and the one SOUTH of it
if your facing to the block is NORTH (horizontal), it would break the block and the block DOWN from it?
however if your facing to the block is DOWN but your horizontal facing is EAST, it would break the block and the one WEST of it?

I think that's right... I'm trying to get it from that. If I'm facing a standing wall and facing north (the side of a block) it would always break that block and the one below it.
If I'm facing a floor below me or ceiling above, you could have it go like this:
I'm facing north (and looking up or down) looking at the top or bottom of a block and it would always break it and the block south of it. If I'm facing east, it breaks it and west of it.

here's a demo of the reworked effect https://streamable.com/r9xeq this issue will be closed once I push the commit, although feel free to give me feedback and/or critiques about it

Perfection!


disable dbt cache logging by default (#1725)

Fixes #1725

Remove all the pprints
If cache logging is disabled, don't call dump_graph at all
Fixed a bug where you could never enable cache logging (haha)

Not rebasing, see #1734 for the 0.14.2 version.