BLE001: conditional handling of errors not detected

keywords: BLE001, log, exception
The following code snippet does not trigger the BLE001 rule:
from logging import exception

try:
    pass
except Exception:
    exception("An error occurred")

but this one does:
from logging import exception

try:
    pass
except Exception:
    if True:
        exception("An error occurred")
    else:
        exception("An error occurred")

This is because ruff doesn't seem to consider the case the both branches of an if handle the error.
The reason we run into this is because sometimes we use code like the following:
from logging import exception

try:
    pass
except Exception as ex:
    error_data = gather_diagnostic_data(...)
    if isinstance(ex, SomespecificError):
        exception("A specific error occurred", extra=error_data)
    else:
        exception("A generic error occurred", extra=error_data)

Something similar was discussed previously in https://github.com/astral-sh/ruff/issues/1119#issuecomment-1341147418.

We were recently investigating the 0.4.4 upgrade and found that snippets like this:
def method() -> None:
    try:
           raise Exception()
    except Exception:
           if False:
               raise
           else:
                  return None

would no longer raise BLE001 (Ruff playground for reference: https://play.ruff.rs/f65629f5-eda0-43f0-a85f-c05a0be1cb9c) because there's a conditional handling the exception. This seems a bit problematic, given that the original issue was about cases where both (or rather, all) branches of a conditional handle the exception, but that isn't the case here. Totally understandable if this is out of scope, just wanted to put it on the radar. Thanks (apologies if this should be filed in another issue)!

A more complex visitor would need to be implemented like in flake8-trio.
https://github.com/astral-sh/ruff/issues/1119#issuecomment-1341171085

Thanks @rahulssheth. My overall feeling was that if you're re-raising even within a branch, then the use of the catch-all exception was probably intentional, and so likely to be a false positive. Are you seeing real cases where you do want it to be flagged, despite a re-raise?

@charliermarsh good point! In most scenarios, if there's some sort of explicit handling with a re-raise, even if there isn't always a re-raise, it likely isn't an problem. I mostly flagged this issue because I think there's a case to be made that false negatives with an ignore should be preferred to false positives here, but I haven't seen any real cases that merit a change of approach. Appreciate the quick feedback!


fix(deps): update module github.com/fatih/color to v1.17.0

This PR contains the following updates:

Package
Type
Update
Change

github.com/fatih/color
require
minor
v1.14.1 -> v1.17.0

Release Notes

fatih/color (github.com/fatih/color)
v1.17.0
Compare Source
What's Changed

Fix multi-parameter println spacing by @klauspost in https://github.com/fatih/color/pull/228
ci: update Go and Staticcheck versions by @fatih in https://github.com/fatih/color/pull/222
Bump golang.org/x/sys from 0.14.0 to 0.17.0 by @dependabot in https://github.com/fatih/color/pull/221
Bump actions/setup-go from 4 to 5 by @dependabot in https://github.com/fatih/color/pull/217
Bump golang.org/x/sys from 0.17.0 to 0.18.0 by @dependabot in https://github.com/fatih/color/pull/224

New Contributors

@klauspost made their first contribution in https://github.com/fatih/color/pull/228

Full Changelog: https://github.com/fatih/color/compare/v1.16.0...v1.17.0
v1.16.0
Compare Source
What's Changed

Update dependabot.yml by @ilyabrin in https://github.com/fatih/color/pull/200
color: add newline after wrapping text by @fatih in https://github.com/fatih/color/pull/192
[Test] Nil check added by @hyunsooda in https://github.com/fatih/color/pull/203
fixes #206 (using underline with a different fg color breaks) by @gregpoirson in https://github.com/fatih/color/pull/210

Dependency updates

Bump dominikh/staticcheck-action from 1.2.0 to 1.3.0 by @dependabot in https://github.com/fatih/color/pull/201
Bump github.com/mattn/go-isatty from 0.0.17 to 0.0.18 by @dependabot in https://github.com/fatih/color/pull/193
Bump golang.org/x/sys from 0.6.0 to 0.8.0 by @dependabot in https://github.com/fatih/color/pull/195
Bump github.com/mattn/go-isatty from 0.0.18 to 0.0.19 by @dependabot in https://github.com/fatih/color/pull/196
Bump golang.org/x/sys from 0.8.0 to 0.10.0 by @dependabot in https://github.com/fatih/color/pull/199
Bump github.com/mattn/go-isatty from 0.0.19 to 0.0.20 by @dependabot in https://github.com/fatih/color/pull/212
Bump golang.org/x/sys from 0.10.0 to 0.13.0 by @dependabot in https://github.com/fatih/color/pull/209
Bump actions/setup-go from 3 to 4 by @dependabot in https://github.com/fatih/color/pull/202
Bump actions/checkout from 3 to 4 by @dependabot in https://github.com/fatih/color/pull/208
Bump golang.org/x/sys from 0.13.0 to 0.14.0 by @dependabot in https://github.com/fatih/color/pull/213

New Contributors

@ilyabrin made their first contribution in https://github.com/fatih/color/pull/200
@hyunsooda made their first contribution in https://github.com/fatih/color/pull/203
@gregpoirson made their first contribution in https://github.com/fatih/color/pull/210

Full Changelog: https://github.com/fatih/color/compare/v1.15.0...v1.16.0
v1.15.0
Compare Source
What's Changed

windows: enable virtual terminal processing, fixes #169 by @martinlindhe in https://github.com/fatih/color/pull/186
ci: update dependencies by @fatih in https://github.com/fatih/color/pull/191
Bump golang.org/x/sys from 0.5.0 to 0.6.0 by @dependabot in https://github.com/fatih/color/pull/189
Refactor color_windows.go by @pellared in https://github.com/fatih/color/pull/188

New Contributors

@martinlindhe made their first contribution in https://github.com/fatih/color/pull/186

Full Changelog: https://github.com/fatih/color/compare/v1.14.1...v1.15.0

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

ℹ Artifact update notice
File name: go.mod
In order to perform the update(s) described in the table above, Renovate ran the go get command, which resulted in the following additional change(s):

1 additional dependency was updated

Details:

Package
Change

github.com/mattn/go-isatty
v0.0.17 -> v0.0.20


Initial design of a production node's internals + design review

The exact details of production node internals will change over time but we need to start somewhere.

[x] Do an initial design document on the architecture of what will go inside each production node, including how those things will be deployed, connected, managed, secured, monitored, etc.
[ ] Have others provide feedback. Make changes as needed.

I finished a first draft of a design document about BigchainDB production nodes. Now it needs review and feedback. It's a fairly short document, with diagrams. Reviewing it shouldn't take very long. Anyone in the BigchainDB organization with the following link should be able to read it and comment.
https://docs.google.com/a/bigchaindb.com/document/d/1ro7hfmKIa0MulCJxRqnnkLI6zIsq-H-kCT8sYZc5Cko/edit?usp=sharing
Bat-signal to @vrde @r-marques @diminator @trentmc @gmcmullen @rhsimplex @sbellem  @TimDaub @SohKai


Test API integration

Closes #21
Kind of winging it with these changes but wanted to get the ball rolling on introducing this feature. Bare minimum functionality should be present with this.

More to do, and definitely some refactoring needed, but moving towards something functionally viable.

Thanks for the feedback, I'll spend some more time working on this when I can. This month is looking fairly busy for me, and I still have a talk that needs writing for TPRC!


add support for parallel diversity calculations

This won't be addressed in #1040 as there are bigger questions about how we generally want to support parallelization in scikit-bio.
We'll ultimately want to be support computing alpha diversity for multiple samples and computing beta diversity for multiple pairs of samples in parallel.
@wasade proposed an interface for this before I started working on that branch, but I can't find it now. @wasade, did you squash commits, and if so did that result in losing that code?

Didn't squash commits, looks like the stubbed code got dropped here

Thanks!

Just syncing out-of-band email discussion about this thread here. A proof of concept block decomposition of the distance matrix is can be found here with a not-extremely-clean-but-works parallel adaption here. This uses a shear method that supports operations on tree arrays (i.e., the result of TreeNode.to_array) which has the advantage of being about 40x faster when testing against Greengenes, and orders of magnitude less space required for large trees. Tests and discussion on shear are here. The shear is needed once per block, and as a result, the time and particularly the space requirements are important.
The motivation for the block decomposition method is that it allows for low memory consumption regardless of the number of samples being computed, with the exception of the memory required to represent the resulting distance matrix (which could be mmap'd if needed...). This method shines in particular with methods like unifrac which require an intermediary data structure. In the case of unifrac, that structure scales in space on the order of O(n * log n * m) where n is the number of OTUs and m is the number of samples. By using a smaller m, the space is scaled down considerably, with the added benefit that a smaller m typically means a smaller n too as it is unlikely that all OTUs in the full matrix exist in the subset of samples. The block method effectively pulls out groups of sample IDs at a time, computed beta diversity, and then provides that incremental result which can be filled into the final full matrix. The size of the block is parameterized and is denoted as k. A k that is too large will result in large amounts of memory due as more samples increases the number of OTUs represented by those samples. However, a small k incurs more overhead and more duplicated computation (see below, associated with #1221).
As a light example, below is a resulting distance matrix with sample IDs {1, 2, 3, 4, 5, 6}, and shows what "blocks" are computed using a size of k=2 where the blocks below are denoted as {(row_ids), (col_ids)}:

# final distance matrix
  1 2 3 4 5 6
1 0 a b c d e
2 a 0 f g h i
3 b f 0 j k l
4 c g j 0 m n
5 d h k m 0 o
6 e i l n o 0

# block {(1, 2), (3, 4)}

  1 2 3 4
1 0 a b c
2 a 0 f g
3 b f 0 j
4 c g j 0

# block {(1, 2), (5, 6)}

  1 2 5 6
1 0 a d e
2 a 0 h i
5 d h 0 o
6 e i o 0

# block {(3, 4), (5, 6)}

  3 4 5 6
3 0 j k l
4 j 0 m n
5 k m 0 o
6 l n o 0

As can be seen above, this approach leads to a large amount of duplicated calculations with m is much larger than k, on the order of ((m / k) * ((m / k) - 1) or approximately double the number of calculations when m=1000 and k=128. However, the primary burden of compute is the construction of the intermediary counts_array with the subsequent pairwise calculations being very fast. As such, the benefits of low memory and parallelism likely outweigh the expense of additional compute for large tables. There is also potential to mitigate the duplicated compute through modifications associated with #1221.

We have highly parallel and optimized solutions in place for UniFrac and Faith's PD maintained external to this library. Parallelization of beta metrics implemented elsewhere, such as Bray Curtis, should depend their respective upstream parallel strategies (i.e., rely on scipy to determine how best to optimize their functions). Parallelization of alpha metrics outside of Faith's PD remains a nice to have.

looks like scikit-learn offers parallel computation of beta-diversity metrics: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise_distances.html @qiyunzhu are there any plans to adopt this or do you shy away from making scikit-learn a dependency?

@sjanssen2 Thanks for pointing us to this! Looks cool. The team does have a plan to incorporate parallelization in scikit-bio in a more universal manner. There are several ways to achieve this. scikit-learn uses joblib, which is handy and effective in my tests. Let me forward this note to our software engineer @mataton . Let me also cc @wasade and @mortonjt .


add missing helmRepository option for input

What this PR does / why we need it:
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Release Notes:

yes, thanks, fixed....

That is strange, on the mandelsoft/helm branch is a different commit than in the PR.

I amended the commit an repuhed, not it gets the correct one (and also shows the correct old one, which was not visible in the UI)


Android: A failure occurred while executing com.android.build.gradle.internal.tasks.CheckDuplicatesRunnable

I'm trying to add this into an existing project that was previously using qr_code_scanner and when I go to build on android, I'm running into a dependency issue with a dependency of this package: com.google.mlkit:barcode-scanning:17.0.2
As a workaround, I'm overriding the version using firebase-bom: https://firebase.google.com/docs/android/learn-more#bom
In my app/build.gradle, I added the following to the end of my list of dependencies:
dependencies {
    implementation platform('com.google.firebase:firebase-bom:31.0.3')
}

The actual error:
Execution failed for task ':app:checkProdDebugDuplicateClasses'.
> A failure occurred while executing com.android.build.gradle.internal.tasks.CheckDuplicatesRunnable
   > Duplicate class com.google.firebase.components.AbstractComponentContainer found in modules jetified-firebase-common-18.0.0-runtime (com.google.firebase:firebase-common:18.0.0) and jetified-firebase-components-16.1.0-runtime (com.google.firebase:firebase-components:16.1.0)
[...]

I checked to see what's pulling in these dependencies via: ./gradlew app:dependencies
From that output I can see mobile_scanner depends on com.google.mlkit:barcode-scanning:17.0.2 which pulls in com.google.firebase:firebase-components:16.1.0 and conflicts with another library I'm using that uses com.google.firebase:firebase-common:18.0.0.
See the output of gradle app dependencies:
+--- project :mobile_scanner
|    +--- io.flutter:flutter_embedding_debug:1.0.0-e85ea0e79c6d894c120cda4ee8ee10fe6745e187 (*)
|    +--- org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.7.10 -> 1.7.20 (*)
|    +--- com.google.mlkit:barcode-scanning:17.0.2
|    |    +--- com.google.android.gms:play-services-basement:18.0.0 (*)
|    |    +--- com.google.android.gms:play-services-mlkit-barcode-scanning:18.0.0
|    |    |    +--- com.google.android.datatransport:transport-api:2.2.1
|    |    |    |    \--- androidx.annotation:annotation:1.1.0 -> 1.5.0 (*)
|    |    |    +--- com.google.android.datatransport:transport-backend-cct:2.3.3
|    |    |    |    +--- androidx.annotation:annotation:1.1.0 -> 1.5.0 (*)
|    |    |    |    +--- com.google.android.datatransport:transport-api:2.2.1 (*)
|    |    |    |    +--- com.google.android.datatransport:transport-runtime:2.2.5 -> 2.2.6
|    |    |    |    |    +--- androidx.annotation:annotation:1.1.0 -> 1.5.0 (*)
|    |    |    |    |    +--- com.google.android.datatransport:transport-api:2.2.1 (*)
|    |    |    |    |    \--- javax.inject:javax.inject:1
|    |    |    |    +--- com.google.firebase:firebase-encoders:16.1.0
|    |    |    |    |    \--- androidx.annotation:annotation:1.1.0 -> 1.5.0 (*)
|    |    |    |    \--- com.google.firebase:firebase-encoders-json:17.1.0
|    |    |    |         +--- androidx.annotation:annotation:1.1.0 -> 1.5.0 (*)
|    |    |    |         \--- com.google.firebase:firebase-encoders:16.1.0 (*)
|    |    |    +--- com.google.android.datatransport:transport-runtime:2.2.6 (*)
|    |    |    +--- com.google.android.gms:play-services-base:18.0.1 (*)
|    |    |    +--- com.google.android.gms:play-services-basement:18.0.0 (*)
|    |    |    +--- com.google.android.gms:play-services-tasks:18.0.1 (*)
|    |    |    +--- com.google.android.odml:image:1.0.0-beta1
|    |    |    +--- com.google.firebase:firebase-components:16.1.0

FYI: The workaround works for me - but I wanted to open an issue just to flag it for you and to make it searchable in case others run into it (and it may be something you wish to include in the documentation - if it turns out to be common enough).


bdc subcommand is missing if few places (bdc spark session)

bdc subcommand is missing if few places (bdc spark statement) also remove an extra character ']' from http://jmespath.org/  URL

@rl-msft : Thanks for your contribution! The author(s) have been notified to review your proposed change.

This pull request is for the Live branch. We can't merge to Live. Would you resubmit your changes to the master branch? Thanks.
Also, Microsoft authors and contributors are asked to make content contributions through the private repository so the content can be staged and validated by the current validation rules.
For more information, see Should I work in a private or a public repo?
If you need help getting set up to work in the private repo, see Fork and clone a reposistory.
Thanks.

#sign-off

Note: The link adjacent to the last change does not work for me--should it, or is this a permissions issue for me?

@MikeRayMSFT, @rl-msft, as with the other two PRs I just commented, that link adjacent to the last change does not work for me. Should it? Thanks.


database

codernityDB (pure python)
sqlite
json file with locking

requirements:

get x records closest to date. Mongodb: https://stackoverflow.com/questions/41773782/mongodb-query-how-find-the-nearest-to-current-date

json file with locking should work if we only store last x messages


Will Chirp be open source?

Will Chirp be open source?

Hope so

I hope at least a page with an api would be made public


Need Suggestion : What type of database table will be best to use your commentbox ?

I am using MySql for my web-application as you know MySql does not support  Hierarchical Data and we have to use extra table where we should make entry for

CREATE TABLE posts (
post_id bigint(20) unsigned NOT NULL AUTO_INCREMENT,
user_id int(7) unsigned NOT NULL,
post varchar(45) COLLATE utf8_bin NOT NULL,
content text COLLATE utf8_bin,
ts timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (post_id,user_id),
KEY post_id (post_id),
KEY user_id (user_id),
CONSTRAINT user_id FOREIGN KEY (user_id) REFERENCES users (user_id) ON DELETE NO ACTION ON UPDATE NO ACTION
) ENGINE=InnoDB AUTO_INCREMENT=952 DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

Second Table called  closure table

CREATE TABLE comment_hierarchy (
ancestor int(10) unsigned NOT NULL,
descendant int(10) unsigned NOT NULL,
depth tinyint(3) NOT NULL DEFAULT '0',
PRIMARY KEY (ancestor,descendant),
KEY descendant (descendant),
CONSTRAINT comment_hierarchy_ibfk_1 FOREIGN KEY (ancestor) REFERENCES posts (post_id),
CONSTRAINT comment_hierarchy_ibfk_2 FOREIGN KEY (descendant) REFERENCES posts (post_id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8

Is this approach fine ? or you have any suggestion which will best suits for your component  ?

I've added information about our database/API design to the documentation: http://viima.github.io/jquery-comments/#link-4. Quite easy to implement even without any hierarchical structure.


datepicker and datetimepicker are required?

I want to know if the lines for the application.js and application.css which includes the "bootstrap-datepicker" and "bootstrap-datetimepicker" are mandatory to use the gem, because it seems to only add a pretty datetimepicker  to simple_form, but i am already using a jquery datetimepicker and i want to know if i will have no problem removing those lines. If this is not mandatory, i think that IMHO it should be mentioned in the readme instructions.
Thanks in advance :D

I think you can remove js and css inclusion. But date and datetime inputs maped to custom classes:
map_type :datetime, to: SimpleForm::Inputs::DateTimePickerInput
map_type :date,     to: SimpleForm::Inputs::DatePickerInput

Maybe it will have some problems with you jquery picker, test it.


API CRUD Generator: correctly support type=text fields in filter and sort

fixes test added in https://github.com/vivid-planet/comet/pull/1491 (althouth differently)

We also need this for input. Do you want to add this in this PR?

input is already supported https://github.com/vivid-planet/comet/blob/main/packages/api/cms-api/src/generator/generate-crud-input.ts#L87

input is already supported https://github.com/vivid-planet/comet/blob/main/packages/api/cms-api/src/generator/generate-crud-input.ts#L87

But it generated field: text;. Fixed in #1497


Testing: Setup Development Environment

Done
https://github.com/slashkudos/kudos-web/blob/ccfd440763b036fd7ff0196f7a40b6e532078159/amplify/team-provider-info.json#L23-L43


Freeze gyp at the last version that supports Android.

Gyp no longer supports Android, this commit freezes it at the last version that did.

I created this PR based on master by mistake.  I'm closing it, will create another one from a branch.


Error of constraint failure

Hi @mballance,
I am getting constraint failure error for following use-case:
@vsc.randobj
class my_s(object):
    def __init__(self):
        self.a_list = vsc.rand_list_t(vsc.uint8_t(),7)
        self.temp_list = vsc.rand_list_t(vsc.uint8_t(),7)
        self.c = 0

    @vsc.constraint
    def ab_c(self):
        with vsc.foreach(self.a_list, idx=True) as i:
            if self.c:
                self.a_list[i] in vsc.rangelist(5,6,7,8)
            else:
                self.a_list[i] in vsc.rangelist(10,11,12,13)

        with vsc.foreach(self.temp_list, idx=True) as i:
            if self.a_list[i] in [6,7]:
                self.temp_list[i] == 0
            else:
                self.temp_list[i] == 1

my = my_s()
my.randomize()

And it is giving an error like this :
Failed constraints:
[1]: (temp_list.<unknown>[0] == 0);
[1]: (temp_list.<unknown>[0] == 0);
[2]: (temp_list.<unknown>[1] == 0);
[2]: (temp_list.<unknown>[1] == 0);
[3]: (temp_list.<unknown>[2] == 0);
[3]: (temp_list.<unknown>[2] == 0);
[4]: a_list.<unknown>[0] in [13, 12, 11, 10];
[4]: a_list.<unknown>[0] in [13, 12, 11, 10];
[5]: (a_list.<unknown>[0] == 6);
[5]: (a_list.<unknown>[0] == 6);
[6]: a_list.<unknown>[1] in [13, 12, 11, 10];
[6]: a_list.<unknown>[1] in [13, 12, 11, 10];
[7]: (a_list.<unknown>[1] == 6);
[7]: (a_list.<unknown>[1] == 6);
[8]: (a_list.<unknown>[2] == 6);
[8]: (a_list.<unknown>[2] == 6);
[9]: a_list.<unknown>[2] in [13, 12, 11, 10];
[9]: a_list.<unknown>[2] in [13, 12, 11, 10];
Traceback (most recent call last):
  File "2usecase.py", line 27, in <module>
    my.randomize()
  File "/home/adduser/.local/lib/python3.8/site-packages/vsc/rand_obj.py", line 107, in randomize
    Randomizer.do_randomize([model])
  File "/home/adduser/.local/lib/python3.8/site-packages/vsc/model/randomizer.py", line 861, in do_randomize
    r.randomize(ri, bounds_v.bound_m)
  File "/home/adduser/.local/lib/python3.8/site-packages/vsc/model/randomizer.py", line 160, in randomize
    raise Exception("solve failure")
Exception: solve failure

I think  if self.a_list[i] in [6,7] this condition, it is interpreting as constraint on self.a_list variable, but here I want to use as checking condition. So how can I resolve this ? Can you please help ?
Thanks & Regards,
Shraddha Devaiya.

Hi @ShraddhaDevaiya,
Okay, this is a bit of an interesting example because it mixes plain-python control-flow constructs with constraint constructs. While this does work in some cases, I would generally recommend against it.
The first 'foreach' in the constraint block will be simplified down to:
                with vsc.foreach(self.a_list, idx=True) as i:
                    self.a_list[i] in vsc.rangelist(10,11,12,13)

This is because 'c' is not a PyVSC variable. Depending on the application, this might be desired.
In the second case, because we're forming a constraint between two arrays, it is necessary to use the PyVSC constraint if/else. I've updated your example as follows and can confirm that elements of a_list are in [10..13] and elements of temp_list are '1'.
        @vsc.randobj
        class my_s(object):
            def __init__(self):
                self.a_list = vsc.rand_list_t(vsc.uint8_t(),7)
                self.temp_list = vsc.rand_list_t(vsc.uint8_t(),7)
                self.c = 0

            @vsc.constraint
            def ab_c(self):
                with vsc.foreach(self.a_list, idx=True) as i:
                    if self.c:
                        self.a_list[i] in vsc.rangelist(5,6,7,8)
                    else:
                        self.a_list[i] in vsc.rangelist(10,11,12,13)

                with vsc.foreach(self.temp_list, idx=True) as i:
                    with vsc.if_then(self.a_list[i].inside(vsc.rangelist(6,7))):
                        self.temp_list[i] == 0
                    with vsc.else_then:
                        self.temp_list[i] == 1

        my = my_s()
        my.randomize()

I'll leave this open for now, since I'd like to see if the solve-failure message could better indicate what is happening here.
Best Regards,
Matthew

Hi @mballance,
Yeah it is working this way. Thanks for helping!
Regards,
Shraddha Devaiya.

Hi @mballance,
I am trying to use this if_then construct with some other case combining this scenario but, when I am  writing else_if for comparison of Enum type, then it is giving me an error.
Following is the code for it:
class my_e(Enum):
    A = 0
    B = 1
    C = 2
    D = 3

@vsc.randobj
class my_s(object):
    def __init__(self):
        self.name = my_e.A
        self.a = vsc.rand_uint8_t(0)
        self.b = vsc.rand_list_t(vsc.enum_t(my_e), 4)

    @vsc.constraint
    def ab_c(self):
        with vsc.foreach(self.b, idx = True) as i:
            with vsc.if_then(self.b[i] == my_e.A):
                self.a == 0
            with vsc.else_if(self.b[i] == my_e.B):
                self.a == 1

my = my_s()

# Randomize
for i in range(5):
    my.randomize()
    print("ITERATION : ",i+1)
    print(my.a, list(my.b))

And it is giving an error like following:
Traceback (most recent call last):
  File "3usecase.py", line 31, in <module>
    my.randomize()
  File "/home/adduser/.local/lib/python3.8/site-packages/vsc/rand_obj.py", line 107, in randomize
    Randomizer.do_randomize([model])
  File "/home/adduser/.local/lib/python3.8/site-packages/vsc/model/randomizer.py", line 831, in do_randomize
    constraint_l.extend(ArrayConstraintBuilder.build(
  File "/home/adduser/.local/lib/python3.8/site-packages/vsc/visitors/array_constraint_builder.py", line 45, in build
    m.accept(builder)
  File "/home/adduser/.local/lib/python3.8/site-packages/vsc/model/field_composite_model.py", line 143, in accept
    v.visit_composite_field(self)
  File "/home/adduser/.local/lib/python3.8/site-packages/vsc/model/model_visitor.py", line 65, in visit_composite_field
    c.accept(self)
  File "/home/adduser/.local/lib/python3.8/site-packages/vsc/model/constraint_block_model.py", line 42, in accept
    v.visit_constraint_block(self)
  File "/home/adduser/.local/lib/python3.8/site-packages/vsc/visitors/constraint_copy_builder.py", line 85, in visit_constraint_block
    super().visit_constraint_block(c)
  File "/home/adduser/.local/lib/python3.8/site-packages/vsc/model/model_visitor.py", line 101, in visit_constraint_block
    self.visit_constraint_scope(c)
  File "/home/adduser/.local/lib/python3.8/site-packages/vsc/visitors/constraint_override_visitor.py", line 26, in visit_constraint_scope
    cc.accept(self)
  File "/home/adduser/.local/lib/python3.8/site-packages/vsc/model/constraint_foreach_model.py", line 47, in accept
    v.visit_constraint_foreach(self)
  File "/home/adduser/.local/lib/python3.8/site-packages/vsc/visitors/array_constraint_builder.py", line 72, in visit_constraint_foreach
    c.accept(self)
  File "/home/adduser/.local/lib/python3.8/site-packages/vsc/model/constraint_if_else_model.py", line 62, in accept
    visitor.visit_constraint_if_else(self)
  File "/home/adduser/.local/lib/python3.8/site-packages/vsc/visitors/constraint_copy_builder.py", line 142, in visit_constraint_if_else
    for cs in c.false_c.constraint_l:
AttributeError: 'ConstraintIfElseModel' object has no attribute 'constraint_l'

if we comment else_if block then it is working. So, can you please help me?
Thanks & Regards,
Shraddha Devaiya.

Hi @mballance,
Did you get a chance to look into this?
Thanks & Regards,
Shraddha Devaiya.

Hi @ShraddhaDevaiya ,
Apologies, I have not had a chance yet. I'll do my best to investigate over the weekend.
Best Regards,
Matthew

Hi @ShraddhaDevaiya,
I've corrected the issue -- which was specific to using else-if inside a foreach constraint. The testcase now runs and the fix is available both on GitHub and in the precompiled PyPi package.
I'll leave this issue open until you can confirm.
Best Regards,
Matthew

Hi @mballance ,
It is working now, Thanks for helping !
Regards,
Shraddha Devaiya.


Added (un)marshaller for enum

Allows labels instead of numbers for enum in yaml, should work with "sigs.k8s.io/yaml" package.
I need to add some unit test
EDIT:
added crd generation for enum as string + enum list

https://github.com/gingersnap-project/api/pull/10 must go in before this

Thanks @rigazilla


[regression-9.0.0] TypeError: Cannot read property 'show' of undefined

Website: https://web.hike.in/
Error:
Uncaught Exception:
TypeError: Cannot read property 'show' of undefined
at App. (/Users/user/Hike Web-darwin-x64/Hike Web.app/Contents/Resources/app/lib/webpack:/src/main.ts:101:18)
at App.emit (events.js:223:5)

I am experiencing the same issues. Have tried on multiple websites.

+1

Same issue with me

I also had the issue described by @kerketta
I followed the steps outlined by @psa-anddev but then got
Uncaught Exception:
Error: Image could not be created from /Users/....app/Contents/Resources/app/icon.png
at Object.createMainWindow (/Users/....app/Contents/Resources/app/lib/webpack:/src/components/mainWindow.ts:105:22)
at App. (/Users/....app/Contents/Resources/app/lib/webpack:/src/main.ts:149:18)
at App.emit (events.js:223:5)
I create the icon.png file inside the app bundle Contents/Resources/app/ and now it launches fine

Hi everybody. If @SSStechnical is right and seeing these two issues popped up at the same time, this issue is correlated to https://github.com/jiahaog/nativefier/issues/992 , which I fixed with Nativefier 9.1.0.
With that, closing, please shout if I'm wrong and the issue persists in 9.1.0, I will re-open the issue and give it a second look. Thanks!

@ronjouch 👋 I think this issues still exists:

which nativefier
/Users/aubreypwd/.nvm/versions/node/v10.21.0/bin/nativefier

public/wp-content [git::feature/71314-fix-issue-where-media-related-blocks-do-not-update-with-new-translated-image-from-the-neutralized-repo *] [17:28]
$ nativefier --version
9.1.0

public/wp-content [git::feature/71314-fix-issue-where-media-related-blocks-do-not-update-with-new-translated-image-from-the-neutralized-repo *] [17:28]
$ node --version
v10.21.0

public/wp-content [git::feature/71314-fix-issue-where-media-related-blocks-do-not-update-with-new-translated-image-from-the-neutralized-repo *] [17:28]
$ npm --version
6.14.7

public/wp-content [git::feature/71314-fix-issue-where-media-related-blocks-do-not-update-with-new-translated-image-from-the-neutralized-repo *] [17:28]
$ nativefier -n "RETROWAVE Radio" -i "$HOME/iCloud/Icons/retrowave-radio.png" "https://retrowave.ru/" /Users/aubreypwd/Applications/Nativefied

Preparing Electron app...

Converting icons...

Packaging... This will take a few seconds, maybe minutes if the requested Electron isn't cached yet...
Packaging app for platform darwin x64 using electron v9.1.0

Finalizing build...
App built to /Users/aubreypwd/Applications/Nativefied/RETROWAVE Radio-darwin-x64

public/wp-content [git::feature/71314-fix-issue-where-media-related-blocks-do-not-update-with-new-translated-image-from-the-neutralized-repo *] [17:28]

I also get:
Uncaught Exception:
Error: Image could not be created from /Users/aubreypwd/Applications/Nativefied/tlk.io-darwin-x64/tlk.io.app/Contents/Resources/app/icon.png
    at Object.createMainWindow (/Users/aubreypwd/Applications/Nativefied/tlk.io-darwin-x64/tlk.io.app/Contents/Resources/app/lib/webpack:/src/components/mainWindow.ts:105:22)
    at App.<anonymous> (/Users/aubreypwd/Applications/Nativefied/tlk.io-darwin-x64/tlk.io.app/Contents/Resources/app/lib/webpack:/src/main.ts:149:18)
    at App.emit (events.js:223:5)

Sometimes on some apps.

@aubreypwd when does this happen? On startup? On clicking the dock? A precise description, or screencast, would be extra helpful.

Funnily enough after a restart and re-running it appears to be working again? I wonder if some cached version of Electron was the issue? Essentially I had checked out the latest version, made a new app and I was getting the above issues. I downgraded and tried again to get the same errors, then upgraded back and it persisted. But over the weekend I had restarted my machine and I opened the app and it worked! Not sure ¯_(ツ)_/¯

Funnily enough after a restart and re-running it appears to be working again? I wonder if some cached version of Electron was the issue? Essentially I had checked out the latest version, made a new app and I was getting the above issues. I downgraded and tried again to get the same errors, then upgraded back and it persisted. But over the weekend I had restarted my machine and I opened the app and it worked! Not sure ¯\_(ツ)_/¯

@aubreypwd alright, thanks for responding 🙂. Closing again, if it happens again please shout with as much details as possible (steps to reproduce, screenshots/screencasts, logs, all the things). I'd bet it will happen again, though, I don't see why the rebuild/restart would have fixed it. Time will tell!
MOST IMPORTANT THING: I fixed your kamoji! On github you need to ¯\\\_(ツ)_/¯ else you break its left arm 😱


Increase Player Limit After session created

I started a server and the initial player maximum was four in my docker compose.  I would like to increase it to 6.  I did so in my compose file but it seems to be still limited to 4.

Have you rebuilt the container? If you recreate this should update just make sure you don't remove data.

Probably you need to manually modify the config if you changed the compose file after the first initial deploy.
You can do it just by editing the Engine.ini located in ./satisfactory-server/gamefiles/FactoryGame/Saved/Config/LinuxServer
Add this block to the end of the file, save it and restart the container.
[/Script/Engine.GameSession]
MaxPlayers=6

@fast1marine you need to rebuild the container whenever you make environment variable changes. You need to run docker compose up -d, then you should be good :)
@fddn you shouldn't use the ini files to make configuration changes. Not to mention, it won't work as qe override the ini config via the launch command

yup rebuilt the container and it still has our game capped at

@fddn your solution did work.
@wolveix they are set to the same value currently so if changing the environment variable eventually works they hopefully won't conflict with each other.

@fast1marine then you're using an old version of the container, as we dropped those ini files in 1.7.1 I believe

satisfactory-server:
container_name: 'satisfactory-server'
hostname: 'satisfactory-server'
image: 'wolveix/satisfactory-server:v1.7.4'
ports:
- '7777:7777/udp'
- '7777:7777/tcp'
volumes:
- './satisfactory-server:/config'
environment:
- MAXPLAYERS=8
- PGID=1000
- PUID=1000
- ROOTLESS=false
- STEAMBETA=false
restart: unless-stopped
healthcheck:
test: bash /healthcheck.sh
interval: 30s
timeout: 10s
retries: 3
start_period: 120s
deploy:
resources:
limits:
memory: 6G
reservations:
memory: 4G
@wolveix using 1.7.4

@fast1marine hm, that's interesting. My apologies, no one else has had any issues since the config change, and it doesn't really make sense that you have.
Could you try upgrading to v1.8.4? If that doesn't work, could you send your full container log?

@fast1marine I've just tested this on multiple machines, and cannot reproduce the issue whatsoever. Are you definitely rebuilding your container after making changes? Could you also fill out the full issue template, please?

@wolveix sorry I have dad duties interfering with me on and off here.  Let me do some save backups and try 1.8.4 we downgraded after getting stuck in a restart loop using latest yesterday.  Can't make my fellow factory friends mad if it's down.

@fast1marine completely understand, no stress! I'm keen to understand what the heck is causing the issue for you, but at least you have a solution that works in the meantime :D

@wolveix 1.8.4 stuck in a restart loop again :-(

@fast1marine stuck in a restart loop? Could you provide me with your container log?

@wolveix so I downgraded back to 1.7.4 but i changed the max player variable to 10.  Now it stuck!  not sure what but if there's any logs or whatever you want from me just let me know what to pull for you and I'll send them.
So for the restart loop.  Basically after changing to 1.8.4 the container won't start it basically keeps restarting ever 5 to 10 seconds.  And sorry I'm not the best at containers what directory do I need to pull those from?

@fast1marine that's interesting, I wonder if something was preventing rebuilding or something.
For the logs, when running v1.8.4, run this docker logs satisfactory-server &> output.log, and then attach the output.log file to your response here :)

@wolveix So I did the log as you said, and I had the QEMU issue listed in your other open issue.  Not sure if it's related at all but now I got 1.8.5 up and running fine so far.   So I'll attach that here and thanks again.  Hope the log file helps.
output1.log

@fast1marine yep, that explains the reboot loop you mentioned. If you haven't already, you need to change your CPU type to "host" in your VM's settings


Frequent freezes in certain games(ELEX/Monster Hunter: World)

When playing ELEX & Monster Hunter: World I get seemingly random freezes after 15 mins-2+ hours of gameplay(it would be impossible to play it with APITrace enabled for this long), with this in dmesg:
NVRM: Xid (PCI:0000:01:00): 31, Ch 00000033, intr 10000000. MMU Fault: ENGINE GRAPHICS GPCCLIENT_PE_2 faulted @ 0x1_0841a000. Fault is of type FAULT_PTE ACCESS_TYPE_READ

Software information

Elex, tried changing the graphics configurations, windowed, vsync, etc., — problem still persists
Also happens with Monster Hunter: World

System information

GPU: GTX 970
Driver: Nvidia 415.27
Wine version: 4.1 staging and 4.2 staging, tried both dll and winelib versions of DXVK

and the problem persists

DXVK version: 0.96

Log files

d3d11.log:ELEX_d3d11.log
dxgi.log: ELEX_dxgi.log

Unsure if this will be any use, but I did manage to capture some maybe interesting stuff using VK_LAYER_LUNARG_standard_validation
elex_dump.txt
Possibly related issues:
#816
https://github.com/doitsujin/dxvk/issues/759#issuecomment-453108515

Closing as a duplicate of #816, please drop a comment there.
The validation errors are not relevant, the invalid subpass barrier is how transform feedback barriers were originally meant to be implemented, but right now there is no satisfactory and "valid" solution.
Binding an UINT format view to a slot that expects SFLOAT format is a common game bug and should also not cause GPU hangs, since at worst it alters how the same data is interpreted.
Anyway, I'm unable to reproduce the issue with Elex, and even if I could, I wouldn't know how to debug Nvidia driver hangs.


IDP demo journey

User Story

As a customer I want to select a template in a backstage catalog so that I am provided with a skeleton application to work with, based on a template repository and per-populated with components that make up my deployment.

Details, Background
In order to take the user on a journey through the IDP, we have an overall story of creating infrastructure components via crossplane and deploying an app that then consumes that infrastructure.
To accomplish the story and really showcase the capabilities of all components in the pipeline the journey is as follows:

A user will access backstage and either create a new cluster or select an existing cluster to deploy the application to
Once the cluster is ready, the user will select a template inside backstage that is linked to a pre-defined github template repository
The user will fill out the project name and possibly some other details related to the template
When the user clicks "submit", a Kratix Promise CR is created. This CR will set up the github repo and bootstrap the repository into the cluster ready for flux to reconcile against
When flux reconciles the repository it will deploy a crossplane claim that creates infrastructure inside AWS. For the purposes of the demo this will be

a VPC with peering connections to the cluster VPC (note - backstage/kratrix will have to hand-off the cluster VPC name)
An RDS Database cluster
An Elasticache cluster using redis as a backend

Flux will additionally deploy an App CR which will be consumed by App Platform to be deployed to the cluster. This will be a small application that a user can interact with which:

writes information to the database
reads information from elasticache
if the cache reports cache-miss, updates the cache with the latest information from the database and then repeats the second step to send this back to the user

Architecture

Blocked by / depends on
- [ ] https://github.com/giantswarm/roadmap/issues/3472
- [ ] https://github.com/giantswarm/roadmap/issues/3429
- [ ] https://github.com/giantswarm/roadmap/issues/3583
- [ ] https://github.com/giantswarm/roadmap/issues/3469

Shouldn't it be a Kratix resource request? I thought the kratix promise is the template/definition of what should be done, and then a request basically instantiates that once, or am I mistaken?

@puja108 My understanding is that the Promise is kind of like a crossplane composition and the CR is a claim against that promise. Admittedly I'm not familiar with the terminology of Kratix at this point, perhaps @piontec has a clearer definition that can fit in here.

Updated to add high level architectural flow

Nice, I guess "deploy app" step also includes create namespace for you? I would honestly pull that step out and put it next to "create cluster" as for platform teams the creation of a new project namespace is quite an important step that includes OIDC?RBAC setup, quotas,  etc. so having it more or less as a "module" would make a lot of sense.

That's an interesting thought - I think my main question here would be "what would be the delivery mechanism" - I can see two possibilities here - We provide a pre-packaged helm chart that can template the namespace, roles, quotas, etc and can be delivered via the app platform
The second option is to use flux to deliver these via a kustomize base path
I'd probably lean towards flux here for sake of simplicity and extensibility
@piontec What are your thoughts on this?

I've seen different scenarios for development environment vs. production environment.
You want to make developers as productive as possible, while production environments are stricter and controlled.
Which use case are we targeting here exactly? Is it
A)  Resources for developers to iterate fast?
B)  Rolling an Application out into a production or pre production environment?
I do unterstand that we want and need to show that we can manage the infrastructure, but feel that we should be able highlight the value of what we are delivering. It this is an Integrated Development Environment, I would want to see a flow that allows a developer to quickly go through code / deploy / feedback / fix cycles.

but feel we should be able highlight the value of what we are delivering

Do you care to elaborate more on how this is not highlighting the value in what we are delivering?
On the contrary I am firmly of the belief this showcases the skills the team has to offer by bringing together a number of disparate tools into a single cohesive journey in a way that customers are already requesting capability towards. Having the ability to demonstrate that is IMHO an incredibly powerful tool, and one we do not have in our arsenal today.
Perhaps I should clarify. An IDP is nothing to do with what happens in an engineers local development environment. IDP in this instance relates to an Internal Developer Platform, an interaction point between engineers and the clusters, a place they can go to what is happening inside the cluster.
By understanding what is going on inside the cluster, engineers are empowered towards the products they themselves manage.
The IDP would be a place they can construct deployments from off-the-shelf products, be that applications delivered as community driven helm charts, infrastructure delivered as crossplane compositions or new applications they are building driven via github template repos represented as kratix promises inside the cluster. This is what this demo journey is showing.

A) Resources for developers to iterate fast?

This is covered in that engineers can use the platform to quickly bootrap new applications into the cluster.

B) Rolling an Application out into a production or pre production environment?

We do not care what environment the engineers are rolling to - all clusters are equal. We care only that the journey is the same irrespective of the target environment and in doing so, take away some of the pain of managing cross-environment deployments

In an Integrated Development Environment, I would want to show a flow that allows a developer to quickly go through code / deploy / feedback / fix cycles.

I do not get the relevance here. We are not interested in what happens during application development lifecycle. IDEs are not a topic I plan to support and is certainly out of scope of this journey.

This might be the wrong place for this discussion...
I do think your demo is a valuable tool that we don't have today. I'm just looking at the overall story and am wondering if we need another demo as well.
Our current iteration of the story & the value is :
We allow your platform engineering team to focus on what it most important in your company: Making developers more productive to speed up innovation and drive business agility and business outcomes. If developers can iterate quickly, you business stays agile and can adapt quickly to changing demands.
I've seen many platform demos. And I don't deny there is value in doing them. It is a lot of work to build these. Though the demo can be a bit unimpressive in the end ( It should be ). It often comes down to pushing a button to commit a change and then automation kicking in and delivering a new / changed cluster.
Value to the business is in developers iterating faster, delivering better software quicker.  Our story of "freeing up the platform engineers to be able to enable developers" is what we can show with this demo.
This feels like a bit of a stretch to me, like we should try and do more. As if we free you up to do important things but will not be able to help you with these more important tasks. Would it not be even better, if we could show that devs can actually iterate faster with our IdP? A comparison of the situation / work without the IdP and the situation with the IdP. This could be a combination of slide ware and demo ( demo with the IdP ).
I know that this is not where we are today. As I said in the beginning, this might not be the right place.

Architecture diagram updated to include the separation of delivery of components such as namespace, quotas, permissions, etc.
These will be delivered to the cluster using the out-of-band delivery method as described in out gitops-template

We provide a pre-packaged helm chart that can template the namespace, roles, quotas, etc and can be delivered via the app platform

We might need to talk to Big Mac here, as they already have some RBAC helper app IIRC and this gets close to their access management ownership. Maybe that app could be provided by Big Mac and deployed by whatever means Honeybadger feels most adequate.

As for @LutzLange comments, I feel this is beyond the scope of this PoC/demo.
This here is just about the getting started quickly step, i.e. setting up a new project with all the bells and whistles (we could over time add additional templates like e.g. for security or o11y to this). This is a big value driver that many current and most potential new customers have been asking for or even working on themselves.
Fast iteration cycles once the project is set up might be influenced by this as everything is set up right and we try to have all environments similarly configured. But there might be other things to show there, which would be based on other features we might work on at some point in the future, e.g. Flagger for canary deployments, automatic branch deployments, o11y setup and validation feedback for these,...

@mproffitt as the solution architect for the IDP Demo and I sat together to summarise where we are:
intermediate status:
Backstage:

Wireframes were created
PoC based on wireframes (scaffolder PoC)
based on capabilities changes were made (basically to reflect back the complexity of cloud) - mostly additional things that needed to be added, the basic structure stayed the same

Some complexity got moved to crossplane, but should not be in crossplane. From User perspective this should be outside of crossplane, mainly everything regarding the region. This requires some complexity in Backstage. The question is, is this complexity already available in backstage (based on post in #news-dev, the Installations page already shows Region information).
Kratix
At the moment we need to ask ourselves which role does Kratix have in our Developer Platform and do we really need it.
@piontec made it work to bootstrap the app-template Demo. @uvegla is figuring out details to make this work on golem, as it was only working in KinD clusters so far.
The initial idea was that Kratix solves some pain points by setting up  the gitops Repo, bootstrapping it to the cluster and reporting it back to backstage. This should solve the pain points that customers need to create their own gitopsRepo and creating secrets to bootstrap it to the cluster. We have seen issues with this before and need to solve this. It seems like Kratix is not the right solution for this, as Kratix controls the resources.
We need to rethink what Kratix role is in our platform (this is not about dropping Kratix, more about refining what Kratix role is)
Crossplane

Crossplane is bootstrapped into Management Clusters with all providers and functions relevant for the demo.
Crossplane has been integrated with ESO, enabling some dogfooding (we as GS never had a usecase for ESO, but now we have)
Crossplane Compositions were written for following AWS components:

VPC
Peering
Resource Access Manager
Transit Gateway (still WIP)
RDS
Elasticache
provisioning inside RDS (Database creations for example - untested)
binding that creates VPC, RDS and Elasticache
wrapper for the binding which enables looking up the workload cluster region and availability zones

Documentation for the Crossplane APIs got created

Demo App (app that gets deployed into the demo cluster)

work has not been started yet on this
App requirements:

App needs to be deployed into cluster
Needs to be capable of reading from Elasticache, if there is no value in Elasticache it then should read from RDS and write the value back into Elasticache and then return it preferably fron Elasticache.
Needs to accept data from User and write date into RDS (only goes into Elasticache if requested by read)
@mproffitt found a python script which was already capable of most of our requirements, updated the script and made it work (original script: https://github.com/mathurk1/Flask-Postgres-Redis-Docker-App)

The question that we need to ask ourselves, do we need to write something on our own in Go, or can we be happy with the app being written in python, which already exists. The goal is to show the platform and how it works and the goal should not be to show that we can write apps in Go - which is also clear because we have plenty of apps in Go.
Completeness of our components (approximately):

Backstage 70%
Crossplane 70% - 80%
Kratix needs to be discussed
Demo App 0% (or 80% if we take the python one)

Tasks:

[ ] Evaluate and discuss Kratix (like mentioned above)
[ ] Region lookup logic needs to go into Backstage
[ ] Transit Gateway needs to be completed
[ ] Crossplane needs to be tested on Management cluster end-to-end
[ ] Kratix needs to be implemented on golem and tested
[ ] Make a decision on are we going with Go or with Python for the app

[ ] If App needs to be in Go, app needs to be implemented
[ ] If we go with python app, app needs to be finalised

After these tasks are done, the final task is to put everything together and test the whole platform end-to-end.
Nice to have extension points (can also be discussed):

Is the workload cluster a prerequisite or do we want to create the workload cluster with the platform?
deploying additional resources like namespaces, quotas, permissions, rbac with the platform
preselect components for your app in backstage (for example, choosing specific preexisting RDS and then creating Database inside that specific RDS instance), this could be important for demoing as component creation takes time

cc: @marians @gusevda @piontec @mproffitt @uvegla

This is the more complete architecture as used in the interim demo

I'd like to phase this work a bit more so we can focus on getting a minimal thing out that is demoable soon.
To me that would mean
phase 1

aim to have this by end of month so we can achieve the company goal of doing the demos in September
Focus on getting the python app going and wire everything together into a coherent demo
Leave out Kratix, rely on Backstage git integration
No further optimisation of crossplane re region if it is working right now, consider cutting things out that are not working and moving to phase 2
not sure if transitgateway is needed here (is it?)

phase 2:

minor improvements to crossplane setup re region, remaining crossplane things

And at the same time, once @piontec is back we can discuss direction with Kratix and if what we are intending to do with it (aka API to Git) would work or not, but pull it out of this demo for now.

As for:

Is the workload cluster a prerequisite or do we want to create the workload cluster with the platform?

WC creation is out of scope. Could be a separate demo where we aim for WC creation from Backstage.

deploying additional resources like namespaces, quotas, permissions, rbac with the platform

Out of scope here, BUT to me this is the next separate demo/feature we should work on. Deploying a ready "environment" for a dev team to a cluster is a super common use case that also a lot of current customers have. This we should then do at least in cooperation with Big Mac, as they do have some early work towards at least the RBAC part of it.

preselect components for your app in backstage (for example, choosing specific preexisting RDS and then creating Database inside that specific RDS instance), this could be important for demoing as component creation takes time

If possible I'd like to keep this out of phase 1. Could be part of phase 2. For the demo in phase 1 I would rely on two instantions, one where we have run the demo in advance and everything works, and one where we show the creation and switch from one to the other when we want to show things working.

not sure if transitgateway is needed here (is it?)

I would have very much liked to avoid transit gateway - It's the one network component I have always struggled with but unfortunately yes, it is needed.
The issue with requiring it relates to being able to get crossplane talking from the Management Cluster to inside the RDS database in order to set up the application database, users/roles and grants - this I considered to be important enough to add additional effort to get working - It's all well and good creating a database server/cluster but if that database cannot be used by the app that needs to consume it, then it's not really much use and there is really no other way to communicate to the RDS without using a TGW
It could be argued "just set up an additional peering connection to the MC" but the second part of this is purpose. TGWs and Peering connections serve different purposes with peering connections being best for high-throughput / low latency and TGWs used for everything else.
On a far more positive note, despite it melting my brain, the core of the TGW is now done - I built a basic version this afternoon and I'm fairly confident that this will just work.

No further optimisation of crossplane re region

I was less clear with your meaning on this point. If this is relating to the composition wrapper that first looks up a cluster, retrieves region and availability zone data, then feeds that to the next composition wrapper then the outer wrapper has not been tested and does not require any downstream changes, it is simply a passthrough that looks up some additional details that's probably best visualised as in the diagram below - any coloured box is a separate composition, white boxes are either endpoint compositions or specific MRs

Hey all!
I just came back and I need to sync with Laszlo, but a few comments from my side:

we already have a go app (a template), that was used to test and build the demo scenario. This app has many features included, like a helm chart and a ready full-fledged CI/CD process, that encompasses full build pipeline, including signing and verification with cosign. I believe it is much easier to add a simple DB code into this app, than it is to try and port all of that into the python app. We've also tested all the project templating and bootstrapping with it.
As for kratix' role, I'm not sure what happened when I was away. Kratix' role seemed pretty well defined: get an object with a project request from a user, bootstrap everything, create resources to kick off Crossplane and commit the bunch to git. This was tested on a PoC level with KinD before I left. This is also what Dmitry was aiming at with backstage integration. I believe removing kratix now will create way more work than we need to keep it and make it work on our clusters, but again, I need to sync with Laszlo first (tomorrow) to see if there are some new problems I wasn't aware of.

Here is a high level process diagram I'd like us to maintain, so it represents what we are buidling. (Not finished yet)
https://miro.com/app/board/uXjVKnjQei8=/

the decision was made to follow the path with the Go App. Which still needs to be modified.

This is the list of apps deployed as part of the release
rdsapp                                1.0.1               2m54s        2m53s           deployed
rdsapp-app-operator                   6.11.0              2m45s        2m43s           deployed
rdsapp-aws-ebs-csi-driver-smons                           2m51s
rdsapp-aws-pod-identity-webhook                           2m51s
rdsapp-capi-node-labeler                                  2m51s
rdsapp-cert-exporter                                      2m51s
rdsapp-cert-manager                                       2m51s
rdsapp-chart-operator                                     2m45s
rdsapp-chart-operator-extensions                          2m51s
rdsapp-cilium-servicemonitors                             2m51s
rdsapp-cluster-autoscaler                                 2m51s
rdsapp-etcd-k8s-res-count-exporter                        2m51s
rdsapp-external-dns                                       2m51s
rdsapp-irsa-servicemonitors                               2m51s
rdsapp-k8s-audit-metrics                                  2m51s
rdsapp-k8s-dns-node-cache                                 2m51s
rdsapp-metrics-server                                     2m51s
rdsapp-net-exporter                                       2m51s
rdsapp-node-exporter                                      2m51s
rdsapp-observability-bundle                               2m51s   <-- Look what's here...
rdsapp-prometheus-blackbox-exporter                       2m51s
rdsapp-security-bundle                                    2m51s
rdsapp-teleport-kube-agent                                2m51s
rdsapp-vertical-pod-autoscaler                            2m51s

Regarding Backstage UI wording:

Card title
Target entity -> Creation progress

Target component is not available yet. See Kratix resources for more information.

should be changed to:

For resources to be created, this pull request must be merged. After merging, it can take several minutes for resource creation to start.
Once resources get created, you can track creation progress in the Kratix resources tab.

Once the catalog entity exists, we show this:

The pull request defining these resources has been merged.
See resource creation details in the Kratix resources tab.
View the entity page for marians-demo-service to see more details about the component and its deployments.

We are changing the demo flow as follows:

A database server/cluster is required to exist as a prerequisite
For any new service created during the demo, a logical database is added to the cluster/server associated with the deployment k8s cluster

Rationale:

We think that creating an RDS cluster for each service in development creates huge overhead, so the scenario is not very likely in a customer environment.
We want to create a minimum amount of RDS servers for cost saving reasons
RDS cluster provisioning takes time, and we want the demo to run as fast as possible
Creating a database server/cluster requires decisions from the end user that are hard to make for an engineer with limited knowledge of the infrastructure. This is more likely a platform team's job.

In my opinion this removes the capability of showing a very important aspect of the journey in that an app can be deployed along with any and all infrastructure required for its operation should that infrastructure not already exist.
One of the arguments about VPC CIDRS was "Where does that information come from" with the answer being "The platform team" - this argument was not suitable as in the opinion of the team it lead to a lack of self service for application teams who may need to spin up and tear down infrastructure without interaction with the platform team
Now the argument is that "The platform team should provide the RDS database" which contradicts the earlier argument.
If we're to argue that the platform team should handle all infrastructure builds then the purpose of the demo (deploying a new service) becomes mute as it does not demonstrate that a service can be deployed with all required infrastructure.
Whilst the argument made here does carry a lot of merit, it detracts from the capability.
Additionally to this, the arguments only consider RDS, ignoring entirely the Elasticache part of the service which would not work for an additional application as there are no application specific credentials attached and in fact to add credentials for a second application to Elasticache would require a modification to the replication group built by the original deployment and a restart of all replicated clusters.
This capability does not exist today and due to how Elasticache works is not something that can be built separately, as in the case of provisioning users inside RDS.

I do see both your point @marians and @mproffitt.
The big question here is: Who is the audience of the demo. I think it is targeting developers. And as such it should focus on more on speed than creating an environment that is ready to run production workloads. Setting up a full RDS database feels more like a getting ready for production workload task.
Developers are used to using virtual or lightweight dbs for testing & QA.
Saving costs with smaller dev environments as also quite common.

Who is the audience of the demo. I think it is targeting developers.

I would slightly refine this: our target audience are platform teams. The end user we impersonate for the demo flow is a developer.

Talked with @mproffitt and @piontec about Elasticache. To keep things simple, we are not going to provision anything new per new service created. All services/apps will use the same Elasticache redis server. Redis supports multiple databases, but the identifier is numeric, and we wouldn't have an easy way to map database and service/project. By writing to the same database, there may be a theoretical risk of key collision, but we accept this for now, as we don't run many demos concurrently and we can set the key lifetime very short in our demo application.

Interesting feedback around the VPC CIDRs we got from Conti when we showed them our architecture was that in their case, there's a team that has basically VPC provisioning as their main service, so they basically separate our demo into several use cases that play into each other. Still does not invalidate our demo, just that different companies might disect the use cases or services differently.
Similar, I'd say, to how we here now disect the "creation of an RDS cluster" from the "creation and provisioning of a DB in said cluster".
I think it is good to cut the demo into something rather small for now, and then be able to show the extended use cases and the complexity that @mproffitt mentioned separately, cause they will not get around the complexity, it will just move somewhere else, in the Conti case actually to a team that will for now not use our stuff to automate their processes, but that we could maybe convince at some point, which then makes it easier for them to chain and integrate platform services into a coherent user experience.

Interesting feedback around the VPC CIDRs we got from a potential customer when we showed them our IDP demo architecture was, that in their case, there's a team that has basically VPC provisioning as their main service, so they basically separate our demo into several use cases that play into each other. Still does not invalidate our demo, just that different companies might disect the use cases or services differently.
Similar, I'd say, to how we here now disect the "creation of an RDS cluster" from the "creation and provisioning of a DB in said cluster".
I think it is good to cut the demo into something rather small for now, and then be able to show the extended use cases and the complexity that @mproffitt mentioned separately, cause they will not get around the complexity, it will just move somewhere else, in the customer's case actually to a team that will for now not use our stuff to automate their processes, but that we could maybe convince at some point, which then makes it easier for them to chain and integrate platform services into a coherent user experience.

More ideas

It would be so nice if we could have progress comments after merging a PR like https://github.com/DemoTechInc/demotech-gitops/pull/82, directly in the same PR.

When running the demo, I am the creator of such a PR, so I am not able to review/approve it myself. To merge it, I have to bypass branch protection rules. It would be more realistic if some use (bot) would approve the PR instead.

@marians the first point I'm agnostic towards but I wonder if that's overcomplicating things a little
The second point, no. This would automate too much and detract from showing a) what requires or should have human input and b) creates a failure point such as you selected the wrong provider Config or (future) assigned permissions to users or roles that are incorrect.
Even though a lot of this is automated, I feel automating the pr approval is a step too far and introduces entropy into the system

Automating the PR approval was meant as a fake thing that would simulate what otherwise would of course be done through a human.

We used to just say: "And if you want, you can require PR reviews to merge your requests." And then merge them ourselves in the Demos that I did for Weaveworks.
We should be fine addressing this with the audio track.

Yeah, most companies I've spoken to have some kind of approval process. That said, if we automated validation could be done in the PR, at least some would enable some auto-merge functionality. Usually wouldbe some kind of PR bot checking for access control (i.e. is user allowed to request said resource) and  approve and then if all validation tests are green auto-merge goes through.

In the GoReleaser step of the release workflow I see this log message:
only configurations files on version: 2 are supported, yours is version: 0, please update your configuration

Is there a technical reason for all workloads landing in the default namespace? Would it make sense to create a namespace after the service name?

@marians The main driver for the demo at this stage was simplicity, also see the comment from @puja108 here https://github.com/giantswarm/roadmap/issues/3470#issuecomment-2288134103

deploying additional resources like namespaces, quotas, permissions, rbac with the platform

Out of scope here, BUT to me this is the next separate demo/feature we should work on.

As for technical reasons, In fact the crossplane compositions support a different namespace for delivering the secrets and we can use any namespace on the workload cluster for application deployment. The only thing that needs to happen is that namespace must pre-exist for ESO to send secrets to, and as per Pujas response, we had moved this out of phase 1 delivery

Just putting it here as a sidenote:
Namespace creation for a new project is a thing most companies have as a service and could be a cool module by itself. It could provision a namespace (with RBAC/OIDC, quota, security/network policy setup) for those use cases where there's no golden path (yet), and it could be chained with a golden path like in this demo, to remove the need for a two-step request.
The good thing is, that such a namespace provisioning service could be basically just a helm chart that takes values like project name, team name, OIDC group, and auto-maps things. It can then be extended with things like o11y multi-tenancy or network policy base by other teams like Atlas and Cabbage.
That said, I'd see that as a complementary thing that we can and should build as it's straight forward and used by many customers, but we should make that a separate project in area platform. cc @teemow this might be a nice project for Q4 or Q1 that aligns different capabilities of different teams and can generate value directly without the need for complex customer customization. We could talk to adidas and some others that already have such a thing, what features they would expect from it.

There is a lot of value in these basic templates. Another template that I have seen in the wild is : "Create a Git repo' They need to be setup in the right way to keep things in order. There is naming conventions and security settings to take into consideration. Those should not be left open for developers to chose if you want to keep chaos at bay.
We already have this implemented as part of the IDP demo. It would make sense to pull this out as a separate template as well..

Franz wanted us to have some Governance aspects in the demo as well.
Governance has 2 parts:
A) Security
B) Compliance
A: How do we make sure things are secure?
--> Security by default with kyverno.
--> Maybe Content Scanner + Rennovate?
B: Compliance
Is a combination of secure & compliant settings. Where a Company needs to comply with a certain set of regulations by creating organisational procedures guides and the correct security settings. A big part of Compliance is proving that you are compliant. You need auditable systems for this. If we are using GitOps it is easy to prove who did what and which settings were put in place at what time.
I think we can cover good parts of this without changing the technical part of the demo, but by addressing these in the audio track.

@LutzLange We were planning on addressing some of this with @giantswarm/team-shield next week and have already included trivy scan integration in the list of potential improvements provided in the description. of this issue.
For the moment though, for the audio-track we can already highlight how we ensure some security, split into two topics

in cluster (image scans, kyverno, etc)
cloud (disable default security groups, allow ingress only, no default VPCs, secure connections by default, encryption at rest, etc)

We should be careful on the cloud security side though as this is not a topic we traditionally cover and this would normally be the responsibility of the customer cloud security team - I would be hesitant to get bogged down here as it's a whole topic unto itself however as we're showing building infrastructure, we can anticipate some questions towards the topic.

AFAIK we already have SBOMs and signatures in the build process and store them in the OCI registry. Not sure if we are already checking for those in cluster, but that might be an easy next step (enabled only for the app namespace to not break the whole cluster).
We also already have PSS enforcement in-cluster, not sure if we also have network policies, but that could be added. On this level we could mention that you need a combination of in-cluster enforcement and "adding the actual security rules and exceptions to the app". As in this case we are creating an app from a template this means the template needs to include those things and be "secure by default", which I would guess it is, if it runs smoothly in our clusters.
CVEs scans and reporting would be a good next feature for platform in Q4, but we need to discuss that on a general level and I don't think it makes sense to just smash it into the demo right now, as there the process is more important than just showing CVEs.

I would like to propose a change to the review page.
There has been feedback from Conti regarding the Git Repos and that it is not quiet clear which repos are involved and what they are used for. Could we point out that the 1st repo that the initial PR lands in is the Platform Team Git Repo ? It is currently jjust listed as "Git Ops Repository Location".

Thinking more about the review page, we are not using names that speak to what we are doing here.
We should improve the naming here to make it more obvious:
Lifecycle: LifecycleStage or Stage
Name: ServiceName
Deployment -> Installation Name: ManagementCluster
Deployment -> Cluster Name: WorkloadCluster or TargetCluster
...

There is a lot of value in simpler templates. You could also call them building blocks. There are valuable PE services on their own:
A) Create a Git Repository (ready to use with security & policy)
B) Create a Namespace (ready to use with security, limits & policy )
C) Create an EC2 instance (...)
The self service aspect of these templates provides a lot of value. And If we can find a way to combine these building blocks into more complex templates easily. We would have a set of common building blocks and provide a lot of value to possible customers. I know these last points need further thought, investigation and discussion. but we could and should start with these simpler templates first.

Along those lines, I think we should start closing the first demo issue, and create follow-ups, for which we can then discuss priorities also wrt to the many other things Honeybadger should/could do in the next months.
I've prepared some list to show the complexity of the roadmap decision for the team, but we need to talk about it soon to get clarity what we want to do going forward (at least as long as we don't have a concrete customer to work with).

I just created a separate ticket with my suggestions for improvements. I've scheduled a call for 6-Nov with the Honeybadger team to discuss.


Training error

When I used the given code for training, I got the following error:
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [2]] is at version 2; expected version 1 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
What is the possible reason for that?

We have not met this before. Maybe you can strictly check the version of Pytorch Library and NVIDIA CuDNN.

When I used the given code for training, I got the following error: RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [2]] is at version 2; expected version 1 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True). What is the possible reason for that?

请问怎么解决这个问题？


"Each" and "If" syntax clarifications

A few things that should be pointed out on this page so that others don't have to suffer through the obtuse error messages and trial-and-error that I did:

You must have the trailing colon (:) after the expression or it will fail with a parse error
You must include the leading '-' if what you are trying to substitute or conditionalize is a sequence entry. Omit it if you are just trying to add a mapping value
If it is a sequence entry, you can do the whole sequence and insert variables as required
You can nest multiple each statements, indenting them a level after each. IF you do this, it will create a "matrix" of all possible combinations, a good substitute for how matrix functionality works in Github Actions.
Whatever comes after the each or if statement must be indented one level, even if it doesn't line up with the rest of the sequence/map.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 6724abea-bbdc-bf66-ed5e-3214fa6c3e66
Version Independent ID: 4f8dab21-3f0e-da32-cc0e-1d85c13c0065
Content: Templates - Azure Pipelines
Content Source: docs/pipelines/process/templates.md
Product: devops
Technology: devops-cicd
GitHub Login: @juliakm
Microsoft Alias: jukullam

I believe everything that you said is already reflected in the examples.

@Slooz but it's not explicit, it took a ton of trial and error to find what the syntax was for a map vs. a sequence. As this is documentation it should provide guidance.
If examples were enough, you would just publish the raw YAML reference and say "there you go, here's the documentation" :)

You should definitely provide the input values for the example code, so people can understand the syntax expected.
For example, the datatype StepList or JobList , all MS examples just show [].
I dare you to find me some documentation that actually shows an example of the syntax for either of those data types.
Same with all the stuff you are doing iterating through parameters etc.. It's 10x easier for someone to understand if you show actual real examples instead of this stripped down rubbish.

Nope. I can't get a conditional template to work.
This:
`jobs:

job: Scan
condition: eq(variables['toggle.build.ahe'], 'true')
steps:

${{ if eq(variables['toggle.build.ahe'], 'true') }}:

template: build-scan-tasks.yml
`
Results in the job being run but with no tasks. And yes, the variable is set to 'true'.

@paulmarshall, Is this what you want?
jobs:
  ${{ if eq(variables['toggle.build.ahe'], 'true') }}:
    - job: Scan
      steps:
        - template: build-scan-tasks.yml

You should definitely provide the input values for the example code, so people can understand the syntax expected.
For example, the datatype StepList or JobList , all MS examples just show [].
I dare you to find me some documentation that actually shows an example of the syntax for either of those data types.
Same with all the stuff you are doing iterating through parameters etc.. It's 10x easier for someone to understand if you show actual real examples instead of this stripped down rubbish.

@PirateBread the syntax is the same as for creating any other stage/job/step.

@slo

@Slooz thank you I finally worked it out with the help of this thread and banging my head against the commit tree until all the errors went away.
I wonder if you can help point out where I am going wrong:

parameters:

name: modules
type: object
default:
terraform: ["v0.1.0", "v0.1.1"]

steps:

${{ each repository in parameters.modules }}:

${{ each repo in repository }}:

${{ each tag in repository.value }}:

checkout: git://nostromo/${{ repository.key }}@refs/tags/${{ tag }}

checkout: self
script: tree

What I want to achieve is a checkout step for each tag.
Which is working in this example but instead of getting 2 checkouts, I'm getting 4
So I'm guessing my middle loop is maybe broken and therefore counting more than it should but I'm stuck and not sure.

@PirateBread I'm confused by your middle loop, I don't think you need it. Can you try the following?
 parameters:
   - name: modules
     type: object
     default:
       terraform: ["v0.1.0", "v0.1.1"]
 steps:
   - ${{ each repository in parameters.modules }}:
     - ${{ each tag in repository.value }}:
       - bash: echo git://nostromo/${{ repository.key }}@refs/tags/${{ tag }}
   - checkout: self
   - script: tree

@Slooz  - Thanks friend. Yeah that works perfectly..... I'm not sure what I was trying to do with the middle loop but I was convinced yesterday I needed but it makes perfect sense I only need two loops, one for each tag and then an outer loop for each repo.

Definitely yaml syntax is not obvious in any point of view. I struggling every time I edit my build template. I doing just as blind because here is no any document that do real help with understanding complex examples. I really hate yaml builds! I have to push tons of useless commits to check if it's work or not! Microsoft give us excessively documentation please.

It would be great if user had possibility to "compile" yaml on-premise without having to push changes and watch whole yaml and then check generated file for errors.

It would be great if user had possibility to "compile" yaml on-premise without having to push changes and watch whole yaml and then check generated file for errors.

@VerdonTrigance the vscode azure-pipelines extension does some client-side syntax error checking. There is also the test API that, while requires submitting online to Azure Devops, will check your syntax is correct and expand the variables/loops/etc without actually doing a run.

It would be great if user had possibility to "compile" yaml on-premise without having to push changes and watch whole yaml and then check generated file for errors.

@VerdonTrigance the vscode azure-pipelines extension does some client-side syntax error checking. There is also the test API that, while requires submitting online to Azure Devops, will check your syntax is correct and expand the variables/loops/etc without actually doing a run.

Actually it doesn't. I use VSCode right now and that extension is nothing when you working with templates and variables.
Talking about API - what do you mean?
Anyway, today we have no opportunity to 'compile' and understand where the error is.

@VerdonTrigance
https://github.com/microsoft/azure-pipelines-yaml/issues/34#issuecomment-584253286
Implementation as a powershell cmdlet:
https://github.com/DarqueWarrior/vsteam/blob/41c30c5cc8fa53e8fbab87e702a30f4bb629c2ce/.docs/Test-VSTeamYamlPipeline.md

@VerdonTrigance
microsoft/azure-pipelines-yaml#34 (comment)
Implementation as a powershell cmdlet:
https://github.com/DarqueWarrior/vsteam/blob/41c30c5cc8fa53e8fbab87e702a30f4bb629c2ce/.docs/Test-VSTeamYamlPipeline.md
And as mentioned in that thread, no plans to deliver a client-side equivalent (too much work)

They also mentioned it cannot check templates which mostly cause problems.

@VerdonTrigance
microsoft/azure-pipelines-yaml#34 (comment)
Implementation as a powershell cmdlet:
https://github.com/DarqueWarrior/vsteam/blob/41c30c5cc8fa53e8fbab87e702a30f4bb629c2ce/.docs/Test-VSTeamYamlPipeline.md
And as mentioned in that thread, no plans to deliver a client-side equivalent (too much work)

They also mentioned it cannot check templates which mostly cause problems.

In terms of multi-file templates, yes. You can still reference templates in the same file and expand the whole thing out. I agree it's a big limitation (if you look it's me who called it out) but it's the best we have for now.
Right now when testing templates my best feedback loop is that I set up a custom branch on my repository and have a watcher powershell script that when I save the file, it git pushes it and then watches and pulls the API result, including the full expanded version that shows up in the detail, and reports errors into my Problems tray in vscode. That's as good as it's going to get unless they release a local validator.

@JustinGrote have you seen that you can go to the YAML editor on Azure DevOps and make a check. It seems to validate also the whole nested template structure and tries to generate the full YAML. Haven't checked the API yet. Could it be that this is working now?


Weird issue after pex migration

I'm having serious issues using LP which is odd since I tested it on a different server where the migration worked perfectly fine.
Basically whenever a user joins the console is flooded with error messages, at first I thought it was related to Vault. Why would it nag about a bPermissions calculabletype class when I'm not using bPermissions?
Vault-info returns:
[12:27:04 INFO]: [Vault] Vault v1.6.1 Information [12:27:04 INFO]: [Vault] Economy: Essentials Economy [bInfo, LuckPerms] [12:27:04 INFO]: [Vault] Permission: LuckPerms [LuckPerms, SuperPerms] [12:27:04 INFO]: [Vault] Chat: bInfo [bInfo, LuckPerms]
So yeah, it does seem to think bPermissions is used, while it is clearly not?
Plugins returns:
[12:27:54 INFO]: Plugins (36): CoreProtect, dynmap, LuckPerms, WorldEdit, Animations, TheGaffer, RemoteToolkitPlugin, PlaceholderAPI, BannerBoard, Multiverse-Core, WorldGuard, ProtocolLib, MCMESuite, NoCheatPlus, Freebuild, Tours, VoxelSniper, React, PlotBuild, cTablist, Guidebook, Tab, Vault, TheHelper, AutoTeleport, MyWarp, VentureChat, Essentials, ResourceRegions, DiscordSRV, LeaderHeads, PluginUtils, PremiumVanish, MCME-Architect, MiniGames, NewPlayerQuiz
The only permission manager in there is LuckPerms.
Gist to quite lengty console output on player join: https://gist.github.com/q220/7a9f99611923c7f080036965963af94e

Also this is vault-info from my test server where all went smoothly:
[12:39:37 INFO]: [Vault] Vault v1.6.1 Information [12:39:37 INFO]: [Vault] Economy: Essentials Economy [LuckPerms] [12:39:37 INFO]: [Vault] Permission: LuckPerms [LuckPerms, SuperPerms] [12:39:37 INFO]: [Vault] Chat: LuckPerms [LuckPerms]

what I'm getting from your error is that WorldGuard doesn't see LuckPerms, and I think "bpermissions" might just mean the bukkit permission handler, not the bPerms plugin.

Could not pass event PlayerJoinEvent to WorldGuard v6.2.2-SNAPSHOT

Try going back to the latest release, instead of a snapshot
Searching for "luckperms" on the error has no results, but "worldguard" is on almost every error (except the cTablist and PremiumVanish errors).
Searching for "bperm" did yield a lot of results...There's a chance that those three plugins are not compatible with LP
This is just my observation, and I could be mistaken, I'm a Sponge user not bukkit/spigot.  I would still wait for a response from Lucko.

I tried going back to the recommended release versions of worldedit and worldguard, but the problem remains. If I have op it won't show the errors but the moment I'm de-opped it will start showing the errors every time I move or change point of view. I'm not sure if its LP related, but I started having this after the switch from pex. Could these errors be of permissions that are not set explicitly?

I'm gonna close this for now since it seems this is related to another of our plugins. When I was disabling them one by one I managed to deductively find which plugin is making the server think it is running bPermissions.

(sorry for the late reply, I've been away for a couple of days)
Which plugin was it, just so I know for future reference?
My first suggestion would've been to check the wepif config file. Seems like bPermissions was being defined there.

We have a custom made plugin that handles server resourcepacks depending on the region: https://github.com/MCME/ResourceRegions
The wepif file should be default, as far as I know it was never touched:
`#^M
WEPIF Configuration File^M
#^M
This file handles permissions configuration for every plugin using WEPIF^M
#^M
About editing this file:^M
- DO NOT USE TABS. You MUST use spaces or Bukkit will complain. If^M
you use an editor like Notepad++ (recommended for Windows users), you^M
must configure it to "replace tabs with spaces." In Notepad++, this can^M
be changed in Settings > Preferences > Language Menu.^M
- Don't get rid of the indents. They are indented so some entries are^M
in categories (like "enforce-single-session" is in the "protection"^M
category.^M
- If you want to check the format of this file before putting it^M
into WEPIF, paste it into http://yaml-online-parser.appspot.com/^M
and see if it gives "ERROR:".^M
- Lines starting with # are comments and so they are ignored.^M
#^M
About Configuration Permissions^M
- See http://wiki.sk89q.com/wiki/WorldEdit/Permissions/Bukkit^M
- Now with multiworld support (see example)^M
^M
ignore-nijiperms-bridges: true
resolvers:
enabled:
- PluginPermissionsResolver
- PermissionsExResolver
- bPermissionsResolver
- GroupManagerResolver
- NijiPermissionsResolver
- DinnerPermsResolver
- FlatFilePermissionsResolver
permissions:
groups:
default:
permissions:
- worldedit.reload
- worldedit.selection
- worlds.creative.worldedit.region
admins:
permissions:
- '*'
users:
sk89q:
permissions:
- worldedit
groups:
- admins
`

I guess just remove bPermissionsResolver from that configuration.

Removing the bPermissionResolver did not work (nor did removing every entry in the wepif.yml file) However, we found that there was a sort of 'indirect' dependency on wepif through the plugin I mentioned. So it had nothing to do with with LuckPerms :).
PS: Really loving your plugin!

Glad to hear the issue is fixed! :)


secret-conf.properties has not template to deployment.toml

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


catch operator regression

Observables returned by the catch operator are not properly unsubscribed when used as inner-subscriptions.
Here is a small repro-code:
Observable.interval(2000)
  .switchMap((i) => {
    return Observable
      .interval(200).do((j) => console.log("inner", i + "_" + j))
      .catch(() => {}); // <- notice this catch
  })
  .subscribe()

inner 0_0
inner 0_1
inner 0_2
inner 0_3
inner 0_4
inner 0_5
inner 0_6
inner 0_7
inner 0_8
inner 0_9
inner 1_0
inner 0_10
inner 1_1
inner 0_11
inner 1_2
inner 0_12
inner 1_3
inner 0_13
...

It looks like it has been introduced in this commit 280f7eda0385d0dfca7cea474d46f8a6a8f07196.

And fixed by #886 .

Haha ! I love this project. Thanks @staltz !


Fix OpenQASM 3 built-ins with classical conditions

Summary
The .condition attribute for Instruction instances corresponding to OpenQASM 3 built-ins (such as reset and measure) would previously be ignored.  This now correctly nests them inside a a branching statement in the exporter, so the condition is preserved.

Details and comments
Fix #8730.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

22 of 22 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.003%) to 84.308%

Totals

Change from base Build<PHONE_NUMBER>:
0.003%

Covered Lines:
57830

Relevant Lines:
68594

💛  - Coveralls


chore: minor DX/UX improvements

Description of change
Includes some minor DX improvements:

Developer toolbar icon for the debug log was changed (due to inconsistent stroke width when using a different icon set)
The date_issued from a Ferris test credential was changed to improve the sorting experience
Unused @iconify icon sets are removed from the devDependencies
Route /welcome/password/completed was renamed to /welcome/completed for consistency
Updated some comments
Mock scan page has disclaimer
New mock scan added: "Receive credential offer"
A new script named apply.sh allows to copy all files in gen-static ==> gen to the expected location.
This is useful when regenerating the (untracked) mobile apps.

Links to any relevant issues
n/a
How the change has been tested
Describe the tests that you ran to verify your changes.
Make sure to provide instructions for the maintainer as well as any relevant configurations.
Definition of Done checklist
Add an x to the boxes that are relevant to your changes.

[x] I have followed the contribution guidelines for this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

Unable to test onboarding workflow. Page redirection bug: from /welcome/terms to /welcome. Bug survives app restart.
Might be related to #246.

Onboarding could successfully passed in a mobile build. Issue seems to occur in local development only.


Add link to identify latest coppersmith version

:+1:

I get a 404 for that link, even when I'm logged in:
{
  "errors" : [ {
    "status" : 404,
    "message" : "Unable to find artifact versions"
  } ]
}

Link doesn't work for me either

@dkristian @triggerNZ Try again now?

👍 It's a shame there's no way to include the content of that resource in the rendered markdown, but it's still a good improvement.


Docs.MS Readme Update.

Docs.MS Readme Update.

Docs Build status updates of commit b370750:
:clock10: Full build: the average full build time is 146 min(s) 50 sec(s), this is based on the last 393 build(s) for this repository.

Docs Build status updates of commit b370750:
:warning: Validation status: warnings

File
Status
Preview URL
Details

docs-ref-services/mgmt-netapp-readme.md
:warning:Warning
View (azure-python)
Details

docs-ref-services/mgmt-netapp-readme.md

Line 21: [Warning-multiple-h1s] Multiple H1s are not allowed. You can only have one top-level heading.
Line 27: [Warning-multiple-h1s] Multiple H1s are not allowed. You can only have one top-level heading.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit 4bafcff:
:clock10: Full build: the average full build time is 147 min(s) 47 sec(s), this is based on the last 394 build(s) for this repository.

Docs Build status updates of commit 4bafcff:
:clock10: Full build: the average full build time is 147 min(s) 47 sec(s), this is based on the last 394 build(s) for this repository.

Docs Build status updates of commit 4bafcff:
:clock10: Full build: the average full build time is 147 min(s) 47 sec(s), this is based on the last 394 build(s) for this repository.

Docs Build status updates of commit 4bafcff:
:warning: Validation status: warnings

File
Status
Preview URL
Details

docs-ref-services/mgmt-datafactory-readme.md
:warning:Warning
View (azure-python)
Details

docs-ref-services/mgmt-netapp-readme.md
:warning:Warning
View (azure-python)
Details

docs-ref-services/mgmt-datafactory-readme.md

Line 21: [Warning-multiple-h1s] Multiple H1s are not allowed. You can only have one top-level heading.
Line 27: [Warning-multiple-h1s] Multiple H1s are not allowed. You can only have one top-level heading.

docs-ref-services/mgmt-netapp-readme.md

Line 21: [Warning-multiple-h1s] Multiple H1s are not allowed. You can only have one top-level heading.
Line 27: [Warning-multiple-h1s] Multiple H1s are not allowed. You can only have one top-level heading.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel


use latest odb-codgen

depends on https://github.com/ShiftLeftSecurity/overflowdb-codegen/pull/4
before merging, double check locally if this fixes the problem in https://github.com/ShiftLeftSecurity/ocular/tree/fabs/undobug
n.b. can't do that currently because ocular-latest is red

tested, works. can be merged once odb-codegen is (auto-) released. i just merged that pr