Feature proposal: meltano-manifest.json

Related to:

https://github.com/meltano/meltano/discussions/6944
https://github.com/meltano/meltano/discussions/6766

As discussed in office hours on 2022-10-12.
Feature goals/requirements:

Stability. Since this would serve as the base for many interop/integration scenarios, we want to make the format as stable as possible.
Self-contained. There should be no need to reference external files or services when using this format. So, for example, lock file content would be embedded so no external references are needed.
Pre-calculated inheritance, precedence, and override effects. The reader shouldn't need to know how inheritance works, or understand any other internal Meltano business logic.
No secrets. This artifact essentially needs to be treated as 'code' and may be passed around in less-trusted contexts. As such, it should not contain: env vars from local OS or terminal context, env vars from .dotenv, settings values from systemdb or other external settings/secrets providers.
Simple as possible. Bonus points if the new json manifest is able to be validated against existing JSON Schema rulesets, without creating or maintaining net new data structures.

So, given these guidelines, here's a possible path forward:

Start with contents and structure of the meltano.yml file itself.
Merge in all data from include_paths declarations.
Super-populate each plugin definition in the global context:

Inject all properties from the lock file, except of course, those properties overridden.
Calculate the value of each entry of the plugin's config.
Optionally, we can store 'extra' info about how the evaluation was performed under a meta or vendor key that is explicitly non-stable or at least explicitly free-form.
Environment variable declarations in plugin config should be left unresolved until runtime. If we are confident that the value can be resolved without leaking any sensitive info, the predicted evaluation could optionally be rendered under a meta or vendor key, without losing fidelity of the config's env var reference.

Repeat the above process for each declared Meltano Environment.
Inject other top-level entities (jobs, env, schedules, etc.) into the environment declarations if applicable, so that each environment definition is standalone.

Variations:

We optionally could give an environment name to the 'global' or --no-environment behavior, so that the top-level file is just the environments declaration: version: 1 \n environments: [...] \n <EOF>. This reduces the clutter in the file, and readers would only deserialize the environment definition they need in the given context.
We could optionally create separate manifest files per environment so that the manifest is specific to what is needed exactly in a given context.

A use case we discussed in office hours:
In squared we generate a json file with the schedule list output, during CI deployment of our docker files.
The DAG generator prefers the cache if it exists over repeatedly running schedule list so in production we dont run meltano in airflow containers. Its a static read only environment anyways so no need to refresh.

Timestamped recording of the office hour discussion: https://www.youtube.com/watch?v=0wzaXR5dofU&t=1404

Also relevant for generating documentation, e.g. sphinx-meltano. Ideally the manifest would include full definitions and descriptions of plugins/settings for rendering in html (as at the time of manifest generation).

This would also be great for the dagster extention. If it is possible to also include the (selected) schema of the extractors this would allow me to more easily create Software Defined Assets for the taps.
Let me know if there is anything I can contribute to!

@JulesHuisman, @kgpayne, @WillDaSilva, @pnadolny13 - I've updated the description above with a proposed implementation spec and some broad goals/principals that I thought would be helpful for discussion.
@JulesHuisman - This does not as of now include any scope for the schema of the extractors. Do you think it's okay to postpone this for a first edition? I imagine a path forward for schema may become clearer after we have something like the proposed meltano catalog feature.

@aaronsteers I can clarify our use case a bit more, this might help with further designing the manifest / catalog features.
For the Dagster extension we need to read both the Meltano jobs and schedules. This results in the fact that we need to run meltano job list --format=json and meltano schedule list --format=json. As invoking Meltano can be a bit slow, some async magic needs to be added to limit the time to get this information. Ideally this information could be gathered using one single command (or from one single file). If I'm interpreting it correctly this would be part of the Meltano manifest?
Furthermore, we want to read the (selected) schemas of the taps (or ideally the schema that arrives at the target, in case of mapping). As this allows us to, by combining with dagster-dbt, to automatically infer all your data assets (using Dagster's Software Defined Assets). Creating a really tight integration with Dbt. In an ideal world this information could also be extracted from the manifest file, as this makes it really easy for extensions to do some sort of introspection on the Meltano project itself. This would be comparable to how dbt creates its manifest.
From the dbt docs:

This single file contains a full representation of your dbt project's resources (models, tests, macros, etc) ...

This does not as of now include any scope for the schema of the extractors. Do you think it's okay to postpone this for a first edition?

Makes perfect sense. Inferring the schemas of the extractors is the missing piece from creating all the Dagster assets, so I am excited to see a feature like this, one way or another.

@JulesHuisman - re:

For the Dagster extension we need to read both the Meltano jobs and schedules. ... If I'm interpreting it correctly this would be part of the Meltano manifest?

Yes, 100%.

Furthermore, we want to read the (selected) schemas of the taps (or ideally the schema that arrives at the target, in case of mapping). As this allows us to, by combining with dagster-dbt, to automatically infer all your data assets (using Dagster's Software Defined Assets). Creating a really tight integration with Dbt. In an ideal world this information could also be extracted from the manifest file, as this makes it really easy for extensions to do some sort of introspection on the Meltano project itself. This would be comparable to how dbt creates its manifest.

Completely understood in terms of the use case here.

Do you think it's okay to postpone this for a first edition?

Makes perfect sense. Inferring the schemas of the extractors is the missing piece from creating all the Dagster assets, so I am excited to see a feature like this, one way or another.

While I'm hesitant to include catalog features in the manifest file before we've spec'd out meltano catalog as a feature, a compromise we could offer is to optimistically scan for any pre-cached catalog elements and include the json from those files under a meta or vendor tag. By putting this less-stable part of the content under a meta key or similar, we declare the they are not formally a 'stable' part of the spec, and future versions could (and probably would) break the initial behavior assumptions.
Also, for any taps that don't support discovery, or for taps that haven't yet had their catalogs cached, that entry would be missing. (When orchestrating directly, the user can mitigate this by forcing all taps to run discovery ahead of exporting/shipping their manifests.)
What do you think about this as a possible compromise?

@aaronsteers

What do you think about this as a possible compromise?

Yes, a solution like this would work. The assets part of the dagster extension would also be a work in progress, so that should be fine!

Edit: I realized after writing the above that a potential risk of including the catalog at this phase is that the inclusion of Singer catalog content could blow up the file size exponentially. Still doable, but probably something we'd want to be able to toggle on/off in cases where the extra bloat is not desireable and the catalog metadata is not directly needed for the intended application.

How would the manifest be generated? Would it happen automatically when running certain commands, or would there be a specific command like meltano manifest generate? If it's the latter, maybe there could be some kind of opt-in flag, like: meltano manifest generate --include-catalog.

@aaronsteers

What do you think about this as a possible compromise?

Yes, a solution like this would work. The assets part of the dagster extension would also be a work in progress, so that should be fine!

👍👍

Edit: I realized after writing the above that a potential risk of including the catalog at this phase is that the inclusion of Singer catalog content could blow up the file size exponentially. Still doable, but probably something we'd want to be able to toggle on/off in cases where the extra bloat is not desireable and the catalog metadata is not directly needed for the intended application.

How would the manifest be generated? Would it happen automatically when running certain commands, or would there be a specific command like meltano manifest generate?

Probably both methods - automatic and on-demand.

If it's the latter, maybe there could be some kind of opt-in flag, like: meltano manifest generate --include-catalog.

Yeah, I think something like that could definitely work.

@JulesHuisman - Out of curiosity, how much metadata does Dagster leverage? Does it use the top-level column list, data types, and/or subproperties when applicable?

@JulesHuisman - Out of curiosity, how much metadata does Dagster leverage? Does it use the top-level column list, data types, and/or subproperties when applicable?

@aaronsteers To make the Dagster assets work, I would only need the stream names of the taps. For example, here you can see the comments, commits and projects streams.

However, if you want (this is not required) you can also add all kinds of other metadata to each asset. Here is an example from the dagster-airbyte integration:

Where the schema of the data is also added.

@JulesHuisman - From the above, it looks like we'd ideally just capture the whole selected part of the JSON schema per stream. Assuming we just use schema as the property name for the JSON schema definition, and assuming we drop from the schema everything that has been deselected by the user, the spec starts to take shape here pretty organically...
Note: I'm including psuedocode below in yaml for ease of typing and readability. The actual manifest would list would be json - to distinguish between user-editable resources (that prefer yaml) vs pure machine-readable artifacts for interop (for which we generally would use json).
plugins:
  extractors:
    tap-github:
      pip_url: ...
      # ...
      select:
      - repositories.*
      - "!*.reactions"
      - # ...
      streams:
      # A very simplified pre-resolved version of the upstream schema
      # for each of the extractor's streams
      - name: repositories
        key_properties: [...]
        replication_key: null
        schema: {...} # json schema here
      # ...

We already have another function using the key catalog at this level (used for catalog overrides today), but perhaps a streams key could work (instead of hiding under an interim meta/vendor key). The JSON Schema is pretty non-controversial in terms of compatibility with our own Meltano/Singer paradigms and interop with other platforms - especially if we pre-filter it, since that removes the requirement for needing metadata and other Singer-specific catalog details. The primary keys and incremental replication key values would probably would be worth including as well.
Still worth calling out that the inclusion of a streams entry presupposes that either we've already ran discovery on the extractors, or that we have built the future meltano catalog feature which would store those in a long-term git artifact other than the local .meltano cache files.
Assuming that discovery has run and we already have a catalog cache, this is not a lot more surface area to add. And the fact that we would be excluding deselected streams makes me less worried about file sizes overall.

@aaronsteers I put this in Iteration 27 based on our Alpha needs.

@WillDaSilva - As you pick this up, I just want to summarize a couple expectations here.
CLI
The top-level description text in the main body of the meltano compile UX spec is up-to-date here. (Individual discussion threads might not be though.)
Data structure
Data structure. Even though we're creating a brand new file here to handle a new set of use cases, I would very much like to avoid creating a net new file format to the extent we can avoid it. This is to eliminate causes of failure and costly translation efforts.
Said another way, I would love to see this as a merge and adaptation of JSON Schemas from the plugin lock files and meltano.yml itself - but ideally not creating many/any new data structures that would need to be maintained separately from Meltano's already existing data structures.
As an example, anything that is legal in a plugin's lock file should probably be legal to be overlayed onto the plugin definition itself in the manifest. If we add or deprecate a field in the plugin object definitions, we'd like this to pass through to the manifest without having a second set of changes to make.
As much as possible, I'd like the first iteration to just be a passthrough of existing data elements and data structures. This is to minimize maintenance surface area.
A V2 could try to simplify or standardize inputs - but with those changes would come many more conversations around long-term spec needs that we don't want to creep into our scope here. (Renaming env: to env_vars: for instance, or deprecating variant and replacing with maintainer_id.)
Security and Deferred Env Var Resolution
I think we should not resolve env vars or access any remote settings stores when resolving the manifest. Meaning, anything set in the systemdb, the terminal env var context, or in .env would be unknown and unresolved from the perspective of the manifest file. This is because manifest file must be deployable to environments that have their own secrets access (or not, which would rightfully cause them to fail) and would have their own ability to override or inject their own env var contexts at runtime.
For example, a pip_url that contains a env var reference will remain unresolved in the manifest - which means the manifest itself does not need to be treated as a secret. The manifest would have the same sensitivity level of the git-committed resources: contents may be proprietary and contain potentially confidential IP, but they don't contain credentials or other secrets that would compromise connected users and systems.
Catalog excluded for now
While I think it's worthwhile to have catalog artifacts included over the long run, we're just not ready with the data spec as of yet - and it will introduce undesirable complexity and volatility in the first iteration.

@aaronsteers Thanks for the summary. One question for now: when compiling the manifest for a single environment, do we keep the structure the same as with the all-environment manifest (i.e. the single environment is found under the environments key, and the top-level "global" elements are present as normal), or do we pull the environment up into the top-level of the manifest, and exclude the environments key?
I'm leading towards the latter because it makes for a simpler environment-specific manifest. Once you've chosen to read e.g. meltano-prod-manifest.json  you shouldn't have to know whether you need to read .plugins or .environments[0].plugins - you've already made your choice to get information specific to the environment.

I'm leading towards the latter [pull the environment up into the top-level of the manifest, and exclude the environments key] because it makes for a simpler environment-specific manifest

I agree! Yeah, sounds like a good plan. This removes ambiguity from the user of the file - they can take the assumption that they have the file corresponding to their environment, and just run with it.

Once you've chosen to read e.g. meltano-prod-manifest.json you shouldn't have to know whether you need to read .plugins or .environments[0].plugins - you've already made your choice to get information specific to the environment.

💯
I think there may be an argument here for in the 'full file' to also just not have anything that isn't wrapped under environments collection. Anything at .plugins should be redundant with what we injected to .environments[0].plugins, and can therefor be removed to avoid ambiguity for the receiver. Then, every environment declaration is truly isolated from the others, with no unresolved or 'which should I use' questions left to those consuming.

I am trying to use the meltano.yml schema to determine which objects can have an env object as a child, but I am struggling to include the schema in the wheel.
If I add it to the Poetry include paths, it is included, but not as part of the Meltano package.
If I use a symlink then Poetry errors while building the wheel.
We could move the schemas into src/meltano/schema, but then we'd have to update their location in https://github.com/SchemaStore/schemastore/blob/master/src/api/json/catalog.json, and I'm not a fan of having them under src/meltano/ anyway (though I suppose it makes some amount of sense if we're using them to influence behaviour at runtime for a particular installed version of Meltano).
Perhaps build.py would work, but I'd rather avoid relying on it any more than we have to. That's what I'll try next.

@aaronsteers @tayloramurphy In order to have a relatively flat manifest wherein you don't have to perform computations on the config at runtime, while still making it compatible with the Meltano project file schema, we'll need to have more than just one manifest file per environment as suggested above.
This is because schedules currently have env blocks, and jobs likely will in the future too. We may also want to support plugin config at these levels in the future too.
To have a fully pre-computed manifest file we have to make each leaf node in the project files (i.e. contexts in which Meltano can actually do work) static. It can't change based on whether you're calling from one environment or another, or running under a schedule or not, or running a job or not. To accomplish this for environments we agreed that we'd create a separate environment for each manifest file, but at the time I didn't realize that we actually need one for each combination of these 3 contexts in which Meltano can be running. If I'm missing any other such execution contexts, we'll need to incorporate that in the same way.
To handle this, I propose we decide on an arbitrary order for these contexts, such as (environment, schedule, job), then create each manifest file for a particular combination of them in that order. These can determine the file name of each manifest file: meltano-manifest.<environment name>.<schedule name>.<job name>.json. This may be an issue if . is a legal character within any of these names, so we may have to hash each of the names to avoid that problem.
This is just the default name/location for a manifest file - when one is generated manually via meltano compile the --output parameter can be used to save the file to a different location, e.g. --output ./manifest.json. Specifying --output should only work if a single manifest file is being generated. If some context (e.g. the schedule) is left unspecified, then an --output-dir must be specified instead in which manifests for every value of the unspecified context will be generated.
Each context has a global/none option, so you end up with (e+1)(s+1)(j+1) manifest files, where e is the number of environments, s is the number of schedules, and j is the number of jobs.
If we compile them only as-needed and selectively, that's fine.
As-needed in the case where you're some part of meltano.core and you want to get config for whaterver you're doing now, so you request a manifest for the current environment-schedule-job triple. If it already exists for the current hash of the project files (maybe that hash can be stored within annotations?) then it's simply read.
Otherwise a new manifest is generated, and saved to disk. Selectively when you're operating outside of a Meltano process and need data about a project. You run meltano compile --environment <environment name> --schedule <schedule name> --job <job name> instead of meltano compile, and thereby only generate the manifest for the desired triple.
If it really is the case that all combinations are needed... oof. Hopefully we can generate them quickly, and the project files don't have too many environments/schedules/jobs.

On second thought, my proposal above suffers from an issue: each manifest file would have environments, schedules, and jobs lists that have only 0 or 1 entries. You wouldn't be able to get info about e.g. all schedules for a given Meltano project without generating/reading many manifest files, which is pretty clunky.
That same problem exists with environments in the prior proposal.
I'm not sure how to both create fully pre-computed files, and represent all of the info about a Meltano project one may want. Abandoning the Meltano project files schema may help. We may be able to make use of a dangerously large amount of nesting within a single-file manifest by putting every possible combination of the aforementioned context triples within it. For example, under each of the schedules list items would be the rest of a Manifest file, but with the schedules field removed. Likewise one could descend from there into one of the listed environments, and find the rest of a Manifest file nested within it, but now lacking both the schedules and environments.
This approach manages to represent all info about a Meltano project in a single pre-computed file, but suffers even more from combinatorial explosion because we no longer enforce an order for the elements of the context triples. Where the previous approach takes $(e+1)(s+1)(j+1)$ compilation steps, this new one takes 6 times that. In general, it takes $n!$ times as long as the previous approach, where $n$ is the number of contexts we need to account for. If we define $a_i$ as the number of elements in the $i^{th}$ context, then the number of compilation operations necessary to compute this full manifest is:
$$n!\prod^{n}_{i=0} (a_i+1)$$
We can't compute this on-demand or partially as we could with the prior approach, so this "solution" really just trades one problem for another.

Maybe the only viable approach is a hybrid, wherein we have multiple manifest files of the form meltano-manifest.<environment name>.<schedule name>.<job name>.json, which don't contain general information about e.g. what all the schedules are, or what all the environments are, etc., but then we also have manifest files of the form meltano-manifest.json which contains all non-ambiguous information when no context is specified, or meltano-manifest.<environment name>.json which contains all non-ambiguous information when an environment is specified, but no schedule or job is specified.
Ambiguous information would be anything the missing contexts could affect, which at least includes env vars for all higher levels, but also may include plugin configs in the case of unspecified Meltano environments.
Obviously the manifest filepath scheme would need to be reworked somehow.
This would enable the necessary performance benefits explained in this comment below "If we compile them only as-needed and selectively, that's fine", while still providing a method by which one can get general high-level information about a Meltano project without specifying a specific execution context, such as a list of all of the schedule names and their intervals.
The main problem with this hybrid approach is that it is the most complex for the consumer of this data, compared to the other two approaches discussed here. Still significantly easier than correctly processing a meltano.yml file, however.

@WillDaSilva - I hear the inherent conflict in trying to fully resolve all references and still have a viable resolved config set for all combinations.
What if we relax the requirement that schedules are jobs have fully resolved definitions and just allow that a meaningful execution of the file would still require overlaying any job-specific or schedule-specific overrides?
Also worth noting that while jobs should always be orthogonal to environments (meaning all jobs in theory should be runnable on any environment), schedules will soon have their own environment affinities.
I think it's ok to leave both jobs and schedules in a more or less unchanged from source material when storing in the manifest. There are some good reasons to do this, since an orchestrator that wants to invoke jobs or schedule jobs should not get lost in the details of internal plugin details and config paths. The orchestrator just needs to know how to call Meltano and what overrides (if any) to send to Meltano. Then, Meltano will handle the job of overlaying whatever env vars exist in the is.environ context (for instance) and how those sound interact (or not) with the called plugins and commands.
Would this work and does this help keep things simple?
orchestrator would then need to understand
So, we'd try to resolve each environment

We had a really productive synchronous discussion on this during codeshare today, which I'm summarizing here.
The conclusion we came to was that there is a core conflict in how we want to resolve environment variables in Meltano vs. how we would need to resolve environment variables to persist to a hierarchical data format like JSON with all possible invocation environments fully resolved.
For meltano, the different "nodes" where environment variables are resolved is a directed graph, but not a tree. At any given "node", the resolution of environment variables is dependent on the path taken to arrive at that node. E.g., a plugin's invocation environment can be inherited directly from a meltano environment (like dev) during meltano invoke;  it can be inherited from a job (which inherits from a meltano environment) during meltano run <job name>; it can be inherited from a schedule (which will soon inherit from an environment) during meltano schedule run if the schedule defines a set of plugins directly; or it can be inherited from a job (which inherits from a schedule, which inherits from a meltano environment) during meltano schedule run if the schedule references a job.
For meltano core and meltano cloud, this is workable since the logic is within meltano itself in the case of the former and in the case of the latter, we can iterate on the logic as needed without breaking any contracts/compatibility that consumers depend on -- we're the only consumer.
For other consumers of a meltano manifest, this is problematic since we want to provide the manifest in a widely-used hierarchical format like JSON. So would need to either significantly change things on the meltano core side to make environment variable resolution a strict tree that we can serialize into a hierarchical format or force consumers of the meltano manifest to implement (or possibly import via a separate small package/utility that we maintain, or by requiring them to use meltano core to read the manifest) the logic required to fully resolve the environment at every possible node, either by not resolving them at "compile" time and compiling to a hierarchical format like JSON with some env vars not resolved, or by compiling to a graph serialization format that is capable of representing the actual data. We quickly dismissed the former option but the latter option obviously conflicts slightly with the intended purpose of the meltano manifest.

@cjohnhanson Thank you for the accurate summary, and the good ideas/discussion during the codeshare call.
@aaronsteers What you suggest seems pragmatic, and doable. The catch with that approach is that it limits what we can have as the env var precedence. @tayloramurphy says that the following is ideal:
- environment level plugin env # highest
- schedule level env
- job level env
- root level plugin env
- environment level env
- root level env
- .env file
- terminal env # lowest

If we have one manifest file per environment, then the precedence may only have the schedule env and job env below the lowest level that the manifest compiler pre-expands:
- environment level plugin env # highest
- root level plugin env
- environment level env
- root level env # everything at or above this is pre-expanded by the Manifest compiler; everything below may be "hydrated" into it at runtime
- schedule level env
- job level env
- .env file
- terminal env # lowest

If this limitation is acceptable, then that gives us a path forward. If we are not comfortable with this limitation, then I can explore the idea that @cjohnhanson and I discussed in the codeshare call where the manifest would be a graph, and the expansions would be computed based on which path along the edges were taken to reach a particular vertex. Or more realistically, we can wait until we've had an opportunity to discuss this synchronously with more people, and come to a better solution.


Release new version with upgrade support

Upgrade support has been sitting in master for a while, can we get a new (official) gem version?

using 0.5.2 i got the following error
Failures:

HTTP::Parser should ignore extra content beyond specified length
Failure/Error: @parser <<
HTTP::Parser::Error:
invalid method
org/ruby_http_parser/RubyHttpParser.java:341:in `<<'

Finished in 2.73 seconds
51 examples, 1 failure
Failed examples:
rspec ./spec/parser_spec.rb:233 # HTTP::Parser should ignore extra content beyond specified length
is there is a version of http_parser that works fine with jruby ?

using 0.5.2 i got the following error
Failures:

HTTP::Parser should ignore extra content beyond specified length
Failure/Error: @parser <<
HTTP::Parser::Error:
invalid method
org/ruby_http_parser/RubyHttpParser.java:341:in `<<'

Finished in 2.73 seconds
51 examples, 1 failure
Failed examples:
rspec ./spec/parser_spec.rb:233 # HTTP::Parser should ignore extra content beyond specified length
is there is a version of http_parser that works fine with jruby ?


refactor(tech): create display name / color entries info

As first step to bring icons back, splitting process in two:

Data for display name and colors
Icon data

Here the first is created. By doing that, now there's only need for 1 custom extra definitions
Removes icons that won't be ever added

#417  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @davidlj95 and the rest of your teammates on  Graphite


Error thrown when using HTTP filter

Bug Description
A user ran mirrord with an HTTP filter, in both the CLI and Goland, and got this error:
502 Bad Gateway
mirrord tried to forward the request to the local application and got hyper::Error(Canceled, "connection was not ready")

Steps to Reproduce
N/A
Backtrace
No response
Relevant Logs
No response
Your operating system and version
N/A
Local process
N/A
Local process version
No response
Additional Info
https://discord.com/channels/933706914808889356/1096092286821089461

Fixed with #1340 according to user on Discord.


Logout Problem

Hello,
I use cordova/phonegap, when I logout from facebook, token becomes null, there is no problem here. However, It doesnt ask me facebook password or email when I click the login button again.
So It doesnt allow me login different users.
I think I can logout from app but cannot logout from inapp browser.
How can I solve this problem?

I think you need to deauth the application
Through the native js api it would be
FB.api("/me/permissions", "delete", function(response){});

For that you could use the revokePermissions method which encapsulates the native api call @hutchic mentioned.
Back to topic: I got the same issue with that. I guess its because when you run it as an hybrid app on your phone there might be a problem with the redirect-url as the app does run within the filesystem and not under some domain.
@ccoenraets mentioned this as a comment above the logout method.
* IMPORTANT: For the Facebook logout to work,
the logoutRedirectURL must be on the domain
specified in "Site URL" in your Facebook App Settings

the ,clearcache=yes fixed the logout problem for me. I only added it to the logout function and i removed the settimeout function because it's not needed for me..

I beleive the problem is a BUG, the tokenStore key is incorrect, it should be tokenStore.removeItem('fbAccessToken');
// openfb.js
var logoutWindow,
            token = tokenStore.fbAccessToken;

        /* Remove token. Will fail silently if does not exist */
        tokenStore.removeItem('fbtoken');  // change to tokenStore.removeItem('fbAccessToken');

I beleive the problem is a BUG, the tokenStore key is incorrect, it should be tokenStore.removeItem('fbAccessToken');
// openfb.js
var logoutWindow,
            token = tokenStore.fbAccessToken;

        /* Remove token. Will fail silently if does not exist */
        tokenStore.removeItem('fbtoken');  // change to tokenStore.removeItem('fbAccessToken');

I can confirm that changing
tokenStore.removeItem('fbtoken');

to
tokenStore.removeItem('fbAccessToken');

fixes the logout issue.

Yes, after changing it to -> tokenStore.removeItem('fbAccessToken') the issue has been fixed. Thanks


Avoid popup on exit after saving remote settings

When exiting the repository remote settings, you got a confirmation popup although the settings were just saved. In particular, this happened when changing from a pull request to a branch or tag.
The comparison of saved to unsaved values did not take the change of the head type into consideration.
After the fix, the exit popup after values were saved is avoided.

I wonder if there is a different way to do this? It will be difficult to remember this logic needs changing if the form changes?

The exit/state logic is a bit messy. One day Alexander will get back to refactoring it.


The src/path_planner folder is empty

Thank you very much for your open-source project, but I found that the src/path_planner folder is empty.
Did I do something wrong or is it necessary to wait until the paper is accepted before this part of the code is open-sourced?

Our paper is still under review and we will open source all the code once it is accepted.


Chinese environment mode-line date/month format error

emacs25 when set gnome-formats to chinese, Dec 11 become 日12月11.

besides when use packages gitignore-mode in emacs25 the error "Failed to verify signature" raised so i setq package-check-signature to nil and it seems to be fixed.


Active learning example: Filtering greeting messages.

Request type

[X] Correction or update
[x] New documentation

Details

Reference: Filtering greeting messages.
Item 1: I know this first item could certainly be called "splitting hairs", however, this did cause me some initial confusion, as I sometimes have a tendency to take things very literally instead of subjectively.
In the code comments it reads;
// Your conditional test needs to go inside the parentheses
// in the line below, replacing what's currently there

however, concerning the code in the parentheses of the default reset example;
 if (greetings[i]) { ,
it is really "adding to"  instead of replacing what is already currently there,
if (greetings[i].indexOf('Christmas') !== -1) { .
Sorry if this item is too much of a insignificant issue, but, it did send me for a loop for a few minutes!
Item 2: The below code line needs to be added to the default reset code, as per the solution example. It's not in the default reset example, nor any mention of adding this line is the directions, as I understand them anyway, but, I could be incorrect.
let result = input;  .
Item 3: The below should be changed to = result. Not = input, as in the default reset example.
listItem.textContent = input;
needs to be changed to;
listItem.textContent = result;
as shown in the solution example. Again, as far as I understand, the need to do this is not inferred in the exercise directions, so I'm assuming it's a typo in the default reset example.
Item 4: Additional documentation (sub articles) request, concerning the concept  of
for (let i = 0; i < greetings.length; i++) {
  let input = greetings[i];

and the  !== -1 part of line  if (greetings[i].indexOf('Christmas') !== -1) { .
I've read the sub article materiel on the subject;

You could use this to find all instances of strings that don't contain the substring 'mozilla', or do, if you use the negation operator, as shown below. You could do something like this:
if(browserType.indexOf('mozilla') !== -1) {
// do stuff with the string
}

However, I'm having difficulty in fully understanding and seeing the methodology of how these methods and operators work in this particular exercise.
(let i = 0; i < greetings.length; i++) and  !== -1 .
I know it's asking much, however, if you could write a sub article(s), that expounds in additional detail the functioning of these lines, it would be greatly appreciated!
Thanks, thanks, thanks!

By the way, I wrote a differing humorous "story" version of the exercise...
Link to working example here.
// ---------- My Version of this exercise --------------------------------------

  const volunteerRoster = document.querySelector('.oneWayMission ol');

  volunteerRoster.innerHTML = '';
  let reportToCommandCall = ['DigitalEngine',
                             'Wags',
                             'ChrisMills',
                             'Yo-Yo',
                             'Lt. Bot 000',
                             'Lt. Bot 001',
                             'Lt. Bot 002',
                             'Lt. Bot 003'];
  for (let ibot = 0; ibot < reportToCommandCall.length; ibot++) {
    let callForVoluteers = reportToCommandCall[ibot];
	if (reportToCommandCall[ibot].indexOf('Lt. Bot') !== -1 ) {
	  let didNotStepBackFastEnough = callForVoluteers;
	  let missionRoster = document.createElement('li');
 	  missionRoster.textContent = didNotStepBackFastEnough;
	  volunteerRoster.appendChild(missionRoster);
	}
  }

After rereading and studying the tutorials,
Item 1 Trivial, as if the subject is understood well enough, myself, or a fellow student should be able to recognize the intent, I'd think, and secondly, perhaps its good to force the student to think about it.
Items 2 & 3
Although the default initial/ reset and the solution example are different, the both seem to work, and it perhaps good for the diligent students who do notice the differences, to be presented with the two slightly differing way of accomplishing the same result.
Item 4
I'll open a new issue request for new documentation for the subject only.


rename docs/source-app & adjust docs links for lightning

What does this PR do?
seems the deploy was failing bease we have not generated lightning_app
in fact, we shall be changing all imports in unified docs to lightning instead

Before submitting

Was this discussed/agreed via a GitHub issue? (not for typos and docs)
[x] Did you read the contributor guideline, Pull Request section?
[x] Did you make sure your PR does only one thing, instead of bundling different changes together?
Did you make sure to update the documentation with your changes? (if necessary)
Did you write any new necessary tests? (not for typos and docs)
[x] Did you verify new and existing tests pass locally with your changes?
Did you list all the breaking changes introduced by this pull request?
Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review
Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

Reviewer checklist

[x] Is this pull request ready for review? (if not, please submit in draft mode)
[x] Check that all items from Before submitting are resolved
[x] Make sure the title is self-explanatory and the description concisely explains the PR
[x] Add labels and milestones (and optionally projects) to the PR so it can be classified

cc @borda @carmocca @akihironitta @justusschock @awaelchli

@awaelchli, all good catches, and sorry I missed then... :(

the true check is in: https://github.com/Lightning-AI/lightning/actions/runs/4149219807


ALLOW traffic inside the namespace and the LoadBalancer

Hi everyone
Im trying to config the networkpolicy(cies) to allow traffic only in the same namespace and access from the LoadBalancer

AWS cluster
Calico 2.4.1
K8s 1.7.4

I tried many approaches for example:
kind: Namespace
apiVersion: v1
metadata:
  name: nginx-test
  labels:
    role: nginx-test

---
apiVersion: v1
kind: Pod
metadata:
  name : nginx-server
  namespace: nginx-test
  labels:
    app: nginx-server
spec:
  containers:
    - name: nginx-server
      image: nginx:1.12-alpine
      ports:
        - containerPort: 80
          protocol: TCP

---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
  namespace: nginx-test
  labels:
    app: nginx-server
spec:
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: 80
  selector:
    app: nginx-server
  type: LoadBalancer

---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: nginx-allow
  namespace: nginx-test
spec:
  podSelector:
    matchLabels:
      app: nginx-server
  ingress:
    - from:
      - namespaceSelector:
          matchLabels:
            role: nginx-test
      - podSelector:
          matchLabels:
            app: nginx-server
      ports:
        - protocol: TCP
          port: 80

After the above config the load balancer is OutOfService
It seems that I need to add external access to the pod, and this means access from other namespaces.
Any idea?
btw, thanks for this awesome tutorial.

According to my understanding you want to allow only (1) external traffic (2) traffic from the same namespace.
You can't allow external traffic while restricting SOME internal traffic.
  ingress:
  - from: []

will allow you to accept all internal/external to the pod and you only need this if there's another NetworkPolicy denying such traffic to the pod.
Does this help?

I understood I think. That means this is not possible! :\
My idea was to isolate each namespace, but a namespace contains services (type=LoadBalancer)

Yeah I found out about while developing it.
If you think about it, allowing public access while restricting internal access doesn't make much sense.  ¯_(ツ)_/¯


ble_lywsd03mmc_simpletest.py trigger Type error in BLE with 8.0.0-beta.0

I am not sure this is a bug with this library or something in the BLE library...
But ble_lywsd03mmc_simpletest.py running on:

Adafruit CircuitPython 8.0.0-beta.0 on 2022-08-18; Adafruit Feather Bluefruit Sense with nRF52840

Does trigger a Type error:
Auto-reload is on. Simply save files over USB to run them or enter REPL to disable.
code.py output:
]0;🐍BLE:Reconnecting | code.py | 8.0.0-beta.0\Scanning...
Traceback (most recent call last):
  File "code.py", line 18, in <module>
  File "adafruit_ble/__init__.py", line 282, in start_scan
  File "adafruit_ble/advertising/__init__.py", line 319, in matches
  File "adafruit_ble/advertising/__init__.py", line 329, in matches_prefixes
TypeError: extra keyword arguments given
]0;🐍BLE:Off | 329@adafruit_ble/advertising/__init_ TypeError | 8.0.0-beta.0\
Code done running.

Press any key to enter the REPL. Use CTRL-D to reload.

This is the line 18:
    for adv in ble.start_scan(Advertisement, timeout=5):

Same error on an ItsyBitsy NRF.
Here are the library I use as updated by circup:
adafruit_ble==8.3.7```

The all keyword must now be ``match_allin CircuitPython 8.0.0, transferring toAdafruit_CircuitPython_BLE`!


Improve clarity and readability of password input documentation

Refined descriptions for password input methods: file:, fd:, and stdin

Enhanced readability and consistency in the instructions

Clarified handling of newline and carriage return characters in password input

[x] documentation is added or updated

I wonder how this works on Windows then given that the \r is always present...

I assume it does not work very well, but I haven’t found any issues or change requests related to this behavior so far.

If a file is open in text mode, a \r\n is converted to a \n on Windows, while in binary mode it's not.

If a file is open in text mode, a \r\n is converted to a \n on Windows, while in binary mode it's not.

FDs and stdin as well?

@kroeckx Good point. The library opens the file in text mode, so the file: option is not affected. This issue only arises with existing streams that are opened in binary mode.

@kroeckx I’ve rephrased the documentation slightly. The note about the carriage return has been moved to the fd: section, as it only applies there. The file: and stdin options are unaffected by this behavior because the library correctly opens these streams in text mode.

@nhorman I accidentally typed fixed instead of fixes in the commit, it seems that didn't properly linked this PR to issue #7310.

@erbsland-dev ack, fixed it up.  Thanks for the heads up!

@nhorman please reconfirm

Ack, approval holds

This pull request is ready to merge

Merged to all the active branches. Thank you for your contribution.


⚠ SceneTime has degraded performance

In f8df50b, SceneTime ($ST) experienced degraded performance:

HTTP code: 200
Response time: 2435 ms

Resolved: SceneTime performance has improved in 1a93a93 after 6 minutes.


It looks like the majority of the problems here are with spark.sql.ansi.enabled=true.  The tests are passing, with ANSI mode disabled.


Error in line 125 of noaa.py running sample

C:\Users\Jerry\Python_Projects\Noaa-SDK
(Noaa-SDK) λ python
Python 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 23:03:10) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

from noaa_sdk import noaa
n = noaa.NOAA
res = n.get_forecasts('06405', 'US', True)
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\Jerry\Python_Projects\Noaa-SDK\lib\site-packages\noaa_sdk\noaa.py", line 125, in get_forecasts
lat, lon = self._osm.get_lat_lon_by_postalcode_country(
AttributeError: 'str' object has no attribute '_osm'

@slowtoaccept you are missing bracket when you instantiate the class:

seems not an issue, closing


test-branch/execute-job/fork-branch/c14e505b-43c3-43c6-a575-8803bdb344a1

PR for testing d11227cb9e1feb7c48703ae21ae81421d6ee8f63

padding /canonical/self-hosted-runners/run-workflows d11227cb9e1feb7c48703ae21ae81421d6ee8f63 padding


[build] Lower required cmake version to match Sleipnir

CI is passing so I don't see any reason the required version should be higher than Sleipnir's. Also means I can build on PopOS lts without having to install CMake through something other than the distros repo :smile:

CI passing doesn't actually confirm the project will build with CMake 3.21. Ubuntu CI ships with CMake 3.29, and based on my local testing, setting the minimum version to 3.8 didn't prevent the script from using FetchContent, a feature added in 3.11. The version check just ensures configure was run with a minimum CMake version.
With that said, I looked through the release notes, and the latest required feature was the "Visual Studio 17 2022" generator that shipped in CMake 3.21.


WARN unable to query hord.sqlite: no such column: inscription_id

After a fresh install I get the following error message:
$ ordhook scan blocks --interval 767430:767753 --mainnet
Jan 22 22:03:20.060 INFO Basing ordinals evaluation on database /home/someuser/ordhook/ordhook/hord.sqlite
Jan 22 22:03:20.060 WARN unable to query hord.sqlite: no such column: inscription_id
Jan 22 22:03:20.060 WARN unable to prepare query hord.sqlite: no such column: inscription_id
Jan 22 22:03:21.061 WARN unable to prepare query hord.sqlite: no such column: inscription_id
$ ordhook service start --config-path=./Ordhook.toml
Jan 22 22:04:47.567 WARN unable to query hord.sqlite: no such column: inscription_id
Jan 22 22:04:47.567 INFO Starting service...
Jan 22 22:04:47.672 INFO Indexing inscriptions from block #826780 to block #826895
Jan 22 22:04:50.768 INFO Processing 7 blocks
Jan 22 22:04:50.768 INFO Inscriptions data computation for block #826780 started
Jan 22 22:04:50.768 WARN unable to prepare query hord.sqlite: no such column: inscription_id
Jan 22 22:04:51.771 WARN unable to prepare query hord.sqlite: no such column: inscription_id
$ git clone https://github.com/hirosystems/ordhook.git
$ cd ordhook
$ cargo ordhook-install
$ ordhook config generate --mainnet

(Ordhook,toml)
[storage]
working_dir = "ordhook"

# The Http Api allows you to register / deregister
# dynamically predicates.
# Disable by default.
#
# [http_api]
# http_port = 20456

[network]
mode = "mainnet"
bitcoind_rpc_url = "http://<IP_ADDRESS>:8332"
bitcoind_rpc_username = "XXXX"
bitcoind_rpc_password = "XXXXXXXXX"

# Bitcoin block events can be received by Chainhook
# either through a Bitcoin node's ZeroMQ interface,
# or through the Stacks node. Zmq is being
# used by default:

bitcoind_zmq_url = "tcp://<IP_ADDRESS>:18543"

# but stacks can also be used:
# stacks_node_rpc_url = "http://<IP_ADDRESS>:20443"

[limits]
max_number_of_bitcoin_predicates = 100
max_number_of_concurrent_bitcoin_scans = 100
max_number_of_processing_threads = 16
bitcoin_concurrent_http_requests_max = 16
max_caching_memory_size_mb = 32000

# Disable the following section if the state
# must be built locally

[bootstrap]
download_url = "https://archive.hiro.so/mainnet/ordhook/mainnet-ordhook-sqlite-latest"

[logs]
ordinals_internals = true
chainhook_internals = true

System is Ubuntu 22.04.3 LTS, Release: 22.04, Codename: jammy

@shadadowz: 👋🏽 ; Thanks for opening the issue.
@smcclellan: For context, @shadadowz reached out to us via support@ distro, and I suggested opening an issue here with logs and the environment specifics.

I got the same problem. I guess it due to the SQLite data version is inconsistent with the develop branch code version.
I checked the pr250, this seems is the correct version for the SQLite archive after 2024-01-20.
pr https://github.com/hirosystems/ordhook/pull/250

Is develop different than 2.0.1? I tried using that version on a fresh install again and got the same issue:
~/ordhook-2.0.1$ wget https://github.com/hirosystems/ordhook/archive/refs/tags/v2.0.1.tar.gz
~/ordhook-2.0.1$ tar -zxvf v2.0.1.tar.gz
~/ordhook-2.0.1$ cd ordhook-2.0.1/
~/ordhook-2.0.1$ cargo ordhook-install
Installing ordhook-cli v2.0.0 (/home/someuser/ordhook-2.0.1/components/ordhook-cli)
Updating crates.io index
warning: package memmap2 v0.9.2 in Cargo.lock is yanked in registry crates-io, consider running without --locked
warning: package secp256k1 v0.28.0 in Cargo.lock is yanked in registry crates-io, consider running without --locked
warning: package secp256k1-sys v0.9.1 in Cargo.lock is yanked in registry crates-io, consider running without --locked
Compiling proc-macro2 v1.0.70
Compiling unicode-ident v1.0.12
Compiling version_check v0.9.4
<<SKIPPED - No errors>>
Compiling rocksdb v0.21.0
Compiling ordhook v1.0.0 (/home/someuser/ordhook-2.0.1/components/ordhook-core)
Compiling ordhook-cli v2.0.0 (/home/someuser/ordhook-2.0.1/components/ordhook-cli)
Finished release [optimized] target(s) in 4m 59s
warning: the following packages contain code that will be rejected by a future version of Rust: rstest v0.11.0
note: to see what the problems were, use the option --future-incompat-report, or run cargo report future-incompatibilities --id 1
Replacing /home/someuser/.cargo/bin/ordhook
Replaced package ordhook-cli v2.0.0 (/home/someuser/ordhook/components/ordhook-cli) with ordhook-cli v2.0.0 (/home/someuser/ordhook-2.0.1/components/ordhook-cli) (executable ordhook)
~/ordhook-2.0.1$ ordhook scan blocks --interval 767430:767753 --mainnet
=> https://archive.hiro.so/mainnet/ordhook/mainnet-ordhook-sqlite-latest.tar.gz
Jan 23 00:52:32.484 INFO Unable to retrieve hord.sqlite file locally
Jan 23 00:52:32.485 INFO Downloading https://archive.hiro.so/mainnet/ordhook/mainnet-ordhook-sqlite-latest.tar.gz
[==========================================================]<PHONE_NUMBER>5/18916275145)
Jan 23 01:14:27.662 WARN unable to query hord.sqlite: no such column: inscription_id
Jan 23 01:14:27.662 WARN unable to prepare query hord.sqlite: no such column: inscription_id
Jan 23 01:14:28.663 WARN unable to prepare query hord.sqlite: no such column: inscription_id
Jan 23 01:14:29.667 WARN unable to prepare query hord.sqlite: no such column: inscription_id
<< Repeats until killed >>

Hey there! this issue will be solved with https://github.com/hirosystems/ordhook/pull/250.
Thank you for your patience!

Thank you @lgalabru ! Appreciate the update!

I just downloaded a fresh install to try again and here are my results. Unfortunately I am still getting the no such column error.
~/ordhook$ cargo ordhook-install
  Installing ordhook-cli v2.0.0 (/home/someuser/ordhook/components/ordhook-cli)
    Updating crates.io index
warning: package `memmap2 v0.9.2` in Cargo.lock is yanked in registry `crates-io`, consider running without --locked
warning: package `secp256k1 v0.28.0` in Cargo.lock is yanked in registry `crates-io`, consider running without --locked
warning: package `secp256k1-sys v0.9.1` in Cargo.lock is yanked in registry `crates-io`, consider running without --locked
   Compiling proc-macro2 v1.0.70
....redacted but no errors....
   Compiling stacks-rpc-client v2.0.0
   Compiling chainhook-sdk v0.12.2
   Compiling rocksdb v0.21.0
   Compiling ordhook v1.0.0 (/home/someuser/ordhook/components/ordhook-core)
warning: unused variable: `memory_available`
   --> components/ordhook-core/src/db/mod.rs:251:44
    |
251 | fn rocks_db_default_options(ulimit: usize, memory_available: usize) -> rocksdb::Options {
    |                                            ^^^^^^^^^^^^^^^^ help: if this is intentional, prefix it with an underscore: `_memory_available`
    |
    = note: `#[warn(unused_variables)]` on by default

warning: `ordhook` (lib) generated 1 warning (run `cargo fix --lib -p ordhook` to apply 1 suggestion)
   Compiling ordhook-cli v2.0.0 (/home/someuser/ordhook/components/ordhook-cli)
    Finished release [optimized] target(s) in 4m 38s
warning: the following packages contain code that will be rejected by a future version of Rust: rstest v0.11.0
note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1`
   Replacing /home/someuser/.cargo/bin/ordhook
    Replaced package `ordhook-cli v2.0.0 (/home/someuser/ordhook/components/ordhook-cli)` with `ordhook-cli v2.0.0 (/home/someuser/ordhook/components/ordhook-cli)` (executable `ordhook`)

Install seems to work as well as scan blocks:
$ ordhook scan blocks --interval 767430:767753 --mainnet
=> https://archive.hiro.so/mainnet/ordhook/mainnet-ordhook-sqlite-latest.tar.gz
Feb 06 18:05:58.695 INFO Unable to retrieve hord.sqlite file locally
Feb 06 18:05:58.695 INFO Downloading https://archive.hiro.so/mainnet/ordhook/mainnet-ordhook-sqlite-latest.tar.gz
[==========================================================]<PHONE_NUMBER>2/19092554042)
Inscription 6fb976ab49dcec017f1e201e84395983204ae1a7c2abf7ced0a85d692e442799i0 revealed at block #767430 (inscription_number 0, ordinal_number<PHONE_NUMBER>444387)
Inscription<PHONE_NUMBER>444387
	→ Transferred in transaction 0x6fb976ab49dcec017f1e201e84395983204ae1a7c2abf7ced0a85d692e442799
Inscriptions revealed: 1, inscriptions transferred: 1
-----
Inscription 26482871f33f1051f450f2da9af275794c0b5f1c61ebf35e4467fb42c2813403i0 revealed at block #767753 (inscription_number 1, ordinal_number<PHONE_NUMBER>84699)
Inscription<PHONE_NUMBER>84699
	→ Transferred in transaction 0x26482871f33f1051f450f2da9af275794c0b5f1c61ebf35e4467fb42c2813403
Inscriptions revealed: 1, inscriptions transferred: 1
-----

However looking at individual inscriptions still produces the error:
~/ordhook$ ordhook scan inscription 6fb976ab49dcec017f1e201e84395983204ae1a7c2abf7ced0a85d692e442799i0 --mainnet
Feb 06 18:26:49.495 INFO Basing ordinals evaluation on database /home/someuser/ordhook/ordhook/hord.sqlite
Inscription 6fb976ab49dcec017f1e201e84395983204ae1a7c2abf7ced0a85d692e442799i0 revealed at block #767430 (inscription_number 0, ordinal_number<PHONE_NUMBER>444387)
Feb 06 18:26:49.509 WARN unable to prepare query SELECT offset, outpoint_to_watch, tx_index, block_height FROM locations WHERE inscription_id = ? ORDER BY block_height ASC, tx_index ASC: no such column: inscription_id
Feb 06 18:26:54.512 WARN unable to prepare query SELECT offset, outpoint_to_watch, tx_index, block_height FROM locations WHERE inscription_id = ? ORDER BY block_height ASC, tx_index ASC: no such column: inscription_id

@Shadadow are u still having this issue ? we tried doing same thing as u mentioned from develop as well as from release/2.1.0 and endup with same error
unable to prepare query SELECT offset, outpoint_to_watch, tx_index, block_height FROM locations WHERE inscription_id = ? ORDER BY block_height ASC, tx_index ASC: no such column: inscription_id
please let us know if u were able to resolve ?

I was not. I tried installing on other fresh systems and kept running into the same issue.

I was trying throughout the day finally got some luck.. this is wat i did
git clone https://github.com/hirosystems/ordhook.git
cd ordhook
git checkout 420579
cargo ordhook-install
ordhook config generate --mainnet  # this created Ordhook.toml
#edited Ordhook.toml
[snapshot]
download_url = "https://archive.hiro.so/mainnet/ordhook/mainnet-ordhook-sqlite-latest.tar.gz"

NOTE: the original Ordhook.toml had https://archive.hiro.so/mainnet/ordhook/mainnet-ordhook-sqlite-latest and not https://archive.hiro.so/mainnet/ordhook/mainnet-ordhook-sqlite-latest.tar.gz

Thanks! I will check this out and report back!


Connection loss not detected

Good morning,
I am developing a software that should be able to work either in "online" and "offline" mode.
When an internet connection with the RabbitMQ server is not available, the software should continue performing its duties and store the messages for the server in a proper data structure; when a connection becomes available, the software should send first the accumulated messages and then start consuming the ones coming from the server.
I am able to make the software reconnect in case of connection error, although this only works when I start the software without plugging the ethernet cable. When I plug the cable the software detects the connection availability and reconnects.
The problem is the AMQP client is not able to detect a connection loss, e.g. when I plug out the ethernet cable, hence the software keeps sending messages instead of storing them in the offline message queue. I expected the client fired the connection onError callback, but it does not seem to be so.
I tried exploiting the connected() method of channel prior sending a message, without succeeding.
Here's a minimal example software showing this issue.
Thanks for help in advance.
#include <iostream>
#include <string>
#include <unistd.h>
#include <event.h>
#include <amqpcpp.h>
#include <amqpcpp/libevent.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/ioctl.h>
#include <sys/reboot.h>
#include <netinet/in.h>
#include <net/if.h>
#include <arpa/inet.h>
#include <cfloat>
#include <vector>
#include <queue>
#include <chrono>         // std::chrono::seconds
#include <thread>

typedef struct {
	std::string message;
	std::string expiration;
	std::string queue;
} OfflineMessage;

std::queue<OfflineMessage> offlineMessages;

// Declare variables and data structures
AMQP::SuccessCallback successCbChannel;
AMQP::ErrorCallback errorCbChannel;

AMQP::Address *address;
AMQP::TcpConnection *connection;
AMQP::TcpChannel *channel;
auto loop = event_base_new();

const std::string serverQueue = "server";
std::string gatewayQueue = "";
std::string gatewayId;
std::string rabbitServer;

void reconnect();

class MyConnectionHandler : public AMQP::LibEventHandler
{
	/**
	*  Method that is called by the AMQP library when the login attempt
	*  succeeded. After this method has been called, the connection is ready
	*  to use.
	*  @param  connection      The connection that can now be used
	*/
	virtual void onConnected(AMQP::TcpConnection *connection)
	{
		// @todo
		//  add your own implementation, for example by creating a channel
		//  instance, and start publishing or consuming
		std::cout << "Gateway connection to RabbitMQ server succeeded" << std::endl;
		channel = new AMQP::TcpChannel(connection);
		// Register callbacks
		channel->onError(errorCbChannel);
		channel->onReady(successCbChannel);
	}

	/**
	*  Method that is called by the AMQP library when a fatal error occurs
	*  on the connection, for example because data received from RabbitMQ
	*  could not be recognized.
	*  @param  connection      The connection on which the error occured
	*  @param  message         A human readable error message
	*/
	virtual void onError(AMQP::TcpConnection *connection, const char *message)
	{
		// @todo
		//  add your own implementation, for example by reporting the error
		//  to the user of your program, log the error, and destruct the
		//  connection object because it is no longer in a usable state
		std::cerr << "Connection error: " << message << "; retrying connecting in 5s" << std::endl;
		// Retry connecting after 5 seconds
		std::this_thread::sleep_for (std::chrono::seconds(5));
		delete connection;
		delete channel;
		connection = new AMQP::TcpConnection(this, *address);
	}

	/**
	*  Method that is called when the connection was closed. This is the
	*  counter part of a call to Connection::close() and it confirms that the
	*  connection was correctly closed.
	*
	*  @param  connection      The connection that was closed and that is now unusable
	*/
	virtual void onClosed(AMQP::TcpConnection *connection) {
		std::cout << "Gateway disconnection to RabbitMQ server succeeded" << std::endl;
	}

	/**
	*  Method that is called by the AMQP-CPP library when it wants to interact
	*  with the main event loop. The AMQP-CPP library is completely non-blocking,
	*  and only make "write()" or "read()" system calls when it knows in advance
	*  that these calls will not block. To register a filedescriptor in the
	*  event loop, it calls this "monitor()" method with a filedescriptor and
	*  flags telling whether the filedescriptor should be checked for readability
	*  or writability.
	*
	*  @param  connection      The connection that wants to interact with the event loop
	*  @param  fd              The filedescriptor that should be checked
	*  @param  flags           Bitwise or of AMQP::readable and/or AMQP::writable
	*/
	/*
	virtual void monitor(AMQP::TcpConnection *connection, int fd, int flags)
	{
		// @todo
		//  add your own implementation, for example by adding the file
		//  descriptor to the main application event loop (like the select() or
		//  poll() loop). When the event loop reports that the descriptor becomes
		//  readable and/or writable, it is up to you to inform the AMQP-CPP
		//  library that the filedescriptor is active by calling the
		//  connection->process(fd, flags) method.
	}*/

public:
	/**
	*  Constructor
	*  @param  evbase  The event loop to wrap
	*/
	MyConnectionHandler(struct event_base *evbase) : AMQP::LibEventHandler(evbase) {}

	/**
	*  Destructor
	*/
	virtual ~MyConnectionHandler() = default;
};

MyConnectionHandler *handler;

// Reconnection method
void reconnect() {
	if (connection)
	connection->close();
	delete connection;
	delete channel;
	connection = new AMQP::TcpConnection(handler, *address);
}

void sendMessage(std::string msg, std::string queue, std::string expiration)
{
	AMQP::Envelope envelope(msg.c_str(), msg.length());
	//console->debug("Sending: {};length: {}", envelope.body(), envelope.bodySize());
	if(expiration.length() > 0)
	envelope.setExpiration(expiration);
	channel->publish("", queue, envelope);
	std::cout << "Sent message: " << msg <<  std::endl;
}

// callback operation when a message was received
auto messageCb = [](const AMQP::Message &message, uint64_t deliveryTag, bool redelivered) {
	std::string data (message.body(), message.bodySize());
	std::cout << "Received message: " << data << std::endl;
	// acknowledge the message
	channel->ack(deliveryTag);
};

// callback function that is called when the consume operation starts
auto startCb = [](const std::string &consumertag) {
	std::cout << "Consume operation started by " << consumertag << std::endl;
	std::cout << "Sending messages every 1s" << std::endl;
	int i = 1;
	while (true){
		if (channel->connected()){
			std::cout << "Sending message " << i << std::endl;
			sendMessage("Message " + std::to_string(i), serverQueue, "5000");
			i++;
		}
		else{
			OfflineMessage msg;
			msg.message = "Message " + std::to_string(i);
			msg.queue = serverQueue;
			msg.expiration = "5000";
			offlineMessages.push(msg);
			std::cout << "Stored message " << i << std::endl;
			i++;
		}
		std::this_thread::sleep_for (std::chrono::seconds(1));
	}
};

// callback function that is called when the consume operation failed
auto errorCb = [](const char *message) {
	std::cerr << "Consume operation failed: " << message << "; retrying connecting in 5s" << std::endl;
	// Retry connecting after 5 seconds
	std::this_thread::sleep_for (std::chrono::seconds(5));
	reconnect();
};

// create a custom callback
auto gatewayQueueCreationCallback = [](const std::string &name, int msgcount, int consumercount) {
	std::cout << "Queue " << name << " declared; pending messages " << msgcount << "; consumers " << consumercount << std::endl;
	// start consuming from the queue, and install the callbacks
	channel->consume(gatewayQueue)
	.onReceived(messageCb)
	.onSuccess(startCb)
	.onError(errorCb);
};

auto serverQueueCreationCallback = [](const std::string &name, int msgcount, int consumercount) {
	std::cout << "Queue " << name << " declared; pending messages " << msgcount << "; consumers " << consumercount << std::endl;
	// Send accumulated messages

	OfflineMessage message;
	while (!offlineMessages.empty()){
		message = offlineMessages.front();
		sendMessage(message.message, message.queue, message.expiration);
		std::cout << "Sent stored message: " << message.message << std::endl;
		offlineMessages.pop();
	}

	std::cout << "Messages accumulated offline sent" << std::endl;
};

// Main method
int main(int argc, char* argv[]){
	// Define callbacks
	errorCbChannel = [](const char *message) {
		std::cerr << "Channel error: " << message << "; retrying connecting in 5s" << std::endl;
		// Retry connecting after 5 seconds
		std::this_thread::sleep_for (std::chrono::seconds(5));
		reconnect();
	};

	successCbChannel = [](void) {
		std::cout << "Channel creation succeeded" << std::endl;
		channel->declareQueue(gatewayQueue, AMQP::durable).onSuccess(gatewayQueueCreationCallback);
		channel->declareQueue(serverQueue, AMQP::durable).onSuccess(serverQueueCreationCallback);
	};

	rabbitServer = "";
	gatewayQueue = "gateway";

	std::cout << "Connecting to RabbitMQ server " << rabbitServer << ", rabbitServer" << std::endl;
	//handler = new AMQP::LibEventHandler(loop);
	handler = new MyConnectionHandler(loop);
	address = new AMQP::Address(rabbitServer.c_str());
	connection = new AMQP::TcpConnection(handler, *address);

	event_base_loop(loop, 0);

	//Cleanup
	event_base_free(loop);
	if (connection)
	connection->close();
	delete connection;
	delete handler;
	delete address;

	std::cout << "Exiting application" << std::endl;

	return 0;
}

What do you example mean by "Here's a minimal example software showing this issue." -- that is not minimal at all!!!

Sorry but it's the minimal example I can provide that shows this issue.

It's error to override the monitor() function,  I have made the same mistake.

@FabioDominio  Did you achieve to solve this problem?
@CharmingZhou The monitor function is included inside block comments...

Hi Gabriel (@gchaviaras),
nope, my company and I decided to abandon this approach and move to another technology a long time ago.
And yes, @CharmingZhou as Gabriel said the monitor function is actually commented, I just reported it for completeness.
Thanks for the interest.
Bests.


update Dockerfile. Part of issue #17

This version is better structured for docker layer caching, so it is quicker to build new image after just code change. From previous version however there is increase of  image size from 290MB to 490MB.
AFAIK this is suitable for now, with focus more on development speed.  I would like to lower the overal size together with basing it not on alpine:3.8 (without python) but on python:3.6-alpine or similar (and install just mongodb to it).
Tested locally that the image is can be built and odfuzz is executable inside the container.

Codecov Report

Merging #18 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #18   +/-   ##
=======================================
  Coverage   31.58%   31.58%
=======================================
  Files          15       15
  Lines        3134     3134
=======================================
  Hits          990      990
  Misses       2144     2144

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0188a74...4402396. Read the comment docs.


Natural Keys: Find/Create Library for Framework-level Natural Key Support

This work ended up being addressed in #2764


merge from arlac77/npm-package-template-esm-only

package.json

chore(package): add ^7.1.0 remove ^7.0.1 (devDependencies.c8)

:tada: This PR is included in version 1.5.10 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


i want to block radio function when handcuffed.

what can i use for that. i dont have any idea

LocalPlayer.state.disableRadio would be what you're looking for
On Thu, May 11, 2023 at 3:54 PM Development 5 @.***>
wrote:

what can i use for that. i dont have any idea
—
Reply to this email directly, view it on GitHub
https://github.com/AvarianKnight/pma-voice/issues/413, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADIMKE6G5ZOK2UXPFC5F2DTXFVG2DANCNFSM6AAAAAAX6U6ZKA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

i can use it clientside in lua script ?

how would this line be ?

vll you could also briefly explain to me how I could get the registerkeymapping out and set a fixed key for all :)

This isn't the place for this kind of conversation, if you need help with understanding how to use state bags refer to this forum post if you need help understanding key maps refer to this forum post.
If you need help understanding how to code, use lua, etc, use google.


⚠ Bofrost.be has degraded performance

In e0c00b9, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1454 ms

Resolved: Bofrost.be performance has improved in 13c2a33.


Bazel build failed building Android demo

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 16.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A
TensorFlow installed from (source or binary): source
TensorFlow version: 1.12.0
Python version: 2.7.12
Installed using virtualenv? pip? conda?: no
Bazel version (if compiling from source): 0.19.0
GCC/Compiler version (if compiling from source): 5.4.0
CUDA/cuDNN version: N/A
GPU model and memory: N/A

Describe the problem
Provide the exact sequence of commands / steps that you executed before running into the problem
Running the following command:
bazel build --cxxopt='--std=c++11' -c opt //tensorflow/examples/android:tensorflow_demo --verbose_failures

causes the following build errors:
INFO: From ProtoCompile tensorflow/core/example/example.pb.cc:
bazel-out/android-armeabi-v7a-opt/genfiles/external/protobuf_archive/src: warning: directory does not exist.
bazel-out/android-armeabi-v7a-opt/genfiles/external/protobuf_archive/src: warning: directory does not exist.

Target //tensorflow/examples/android:tensorflow_demo failed to build
INFO: Elapsed time: 176.016s, Critical Path: 25.68s, Remote (0.00% of the time): [queue: 0.00%, setup: 0.00%, process: 0.00%]
INFO: 340 processes: 328 local, 12 worker.

However, the build suceeds if the cloned repository is hard reset to the commit with the following hashcode:
30ebb9b712b2008525944deb47dee8b653f522a3.
This also happened with bazel 0.21.0, which is the latest version to date.
Full Log
stack.txt

sorry, my friend, i am not familiar with this bazel and Android, where is the .tf_configure.bazelrc file?? i do not find it. plz. thank you very much

@tferrerITBA plz

It most likely depends on how you configured tensorflow in the first place. In the .bazelrc file you can see the line try-import %workspace%/.tf_configure.bazelrc. I think you are asked to set your workspace's directory after running ./configure.


I believe that this public document may mislead some customers into thinking that they can enable a firewall on their storage account.

[Enter feedback here]
At the top of this document we can find the following sentence:

The service uses a user-assigned managed identity to access files in an Azure storage account, without your having to make the files publicly accessible.

I understand that above sentence “without your having to make the files publicly accessible” does not mean that connections to a storage account can be restricted by a firewall, but rather that security to a storage account can be achieved by authenticating with a managed identity.
However, in my experience, this statement can confuse customers that they can enable a firewall on the storage account where they store their scripts.
Therefore, I believe it is more appropriate to change the text to state that authentication restrictions can be applied, as shown below.

The service uses a user-assigned managed identity to access files in an Azure storage account,  and it can achieve blocking unauthenticated access to the storage account.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 0a20b5ff-2b70-3f9d-4194-d83aea662286
Version Independent ID: d7155f07-4657-2516-8700-af6ee4d19994
Content: Create a virtual machine image and use a user-assigned managed identity to access files in an Azure storage account - Azure Virtual Machines
Content Source: articles/virtual-machines/linux/image-builder-user-assigned-identity.md
Service: virtual-machines
Sub-service: image-builder
GitHub Login: @kof-f
Microsoft Alias: kofiforson

@shoheioda
Thanks for your feedback! We will investigate and update as appropriate.

@shoheioda
Thanks for reporting this! We have created a PR for this issue and the changes should go live soon.


chore: v7.3.0 changelog

Closes: #XXX
What is the purpose of the change
Fixes the changelog to use the desired release number
Testing and Verifying
This change is a trivial rework / code cleanup without any test coverage.
Documentation and Release Note

Does this pull request introduce a new feature or user-facing behavior changes? no
Is a relevant changelog entry added to the Unreleased section in CHANGELOG.md? no
How is the feature or change documented? not applicable

Codecov Report

Merging #1397 (f6baf8c) into v7.x (44791eb) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head f6baf8c differs from pull request most recent head 090ddcc. Consider uploading reports for the commit 090ddcc to get more accurate results

@@           Coverage Diff           @@
##             v7.x    #1397   +/-   ##
=======================================
  Coverage   20.63%   20.63%
=======================================
  Files         199      199
  Lines       25803    25803
=======================================
  Hits         5325     5325
  Misses      19509    19509
  Partials      969      969

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 44791eb...090ddcc. Read the comment docs.


Relation validation gets less useful while typing

Invalid with nice error message:
model Foo {
  id Int @id
  bar Bar @relation(fields: [bar_id])
  bar_id Int
}

model Bar {
  id Int @id
  foo Foo?
}

C:\Users\Jan\Documents\throwaway\11022>npx prisma validate
Environment variables loaded from .env
Prisma schema loaded from prisma\schema.prisma
Error: Schema parsing
error: Error parsing attribute "@relation": The relation fields `bar` on Model `Foo` and `foo` on Model `Bar` do not provide the `references` argument in the @relation attribute. You have to provide it on one of the two fields.
  -->  schema.prisma:15
   |
14 |   id Int @id
15 |   bar Bar @relation(fields: [bar_id])
16 |   bar_id Int
   |
error: Error parsing attribute "@relation": The relation fields `foo` on Model `Bar` and `bar` on Model `Foo` do not provide the `references` argument in the @relation attribute. You have to provide it on one of the two fields.
  -->  schema.prisma:21
   |
20 |   id Int @id
21 |   foo Foo?
22 | }
   |

Validation Error Count: 2

Add a comma while typing, and becomes invalid with no nice error message:
model Foo {
  id Int @id
  bar Bar @relation(fields: [bar_id], )
  bar_id Int
}

model Bar {
  id Int @id
  foo Foo?
}

C:\Users\Jan\Documents\throwaway\11022>npx prisma validate
Environment variables loaded from .env
Prisma schema loaded from prisma\schema.prisma
Error: Schema parsing
error: Error validating: This line is not a valid field or attribute definition.
  -->  schema.prisma:15
   |
14 |   id Int @id
15 |   bar Bar @relation(fields: [bar_id], )
16 |   bar_id Int
   |

Validation Error Count: 1

Might also be fixed, recheck with current version.


Use Example in README.MD Gives Error

The following example in README.md:
python plotit.py -f cos(x)
Gives me the following error:
zsh: no matches found: cos(x)
This is corrected by:
python plotit.py -f 'cos(x)'

Nice catch, please send a PR to correct it if you are interested :)

I will work on it.

closing this as this has been fixed.


Rule no-invalid-media-type-examples not properly evaluated for types using allOf

Describe the bug
Rule no-invalid-media-type-examples is not triggered for unevaluated properties for examples of allOf types.
To Reproduce
Steps to reproduce the behavior:

Given this redocly.yaml configuration

lint:
  extends:
    - recommended

And this (zipped) OpenAPI file: bikes.yaml.zip
Run this command with these arguments... redocly lint bikes.yaml
See linting output
Only one warning is produced:

Example value must conform to the schema: must NOT have unevaluated properties 'unevaluatedProperty2'.

Expected behavior
Two warnings should be included in the linting output:
Example value must conform to the schema: must NOT have unevaluated properties 'unevaluatedProperty1'.
Example value must conform to the schema: must NOT have unevaluated properties 'unevaluatedProperty2'.

Redocly Version(s)
1.0.0-beta.112
Node.js Version(s)
v18.2.0

Looks like ajv doesn't run the properties code for schemas that don't have any property keywords.
We can fix it in our fork.
Meanwhile you can workaround this issue by adding empty properties into the Wheel schema:
    Wheel:
      properties: {}
      allOf:
        -  $ref: "#/components/schemas/Tire"
        -  $ref: "#/components/schemas/Rim"

@RomanHotsiy  are you saying Ajv doesn't evaluate allOf unless type: object and properties keywords are included?  I think this is related to our prior discussion of using Ajv 8+ and passing the options. In order to maintain full JSON Schema spec compliance, you should pass strict: false without any other options related to strict mode. I think you include strictSchemas: true which is probably where this is failing because he expects every instance to declare the type with that option setting. Ignoring those schemas is not JSON Schema spec compliant behavior.

are you saying Ajv doesn't evaluate allOf unless type: object and properties keywords are included?

@jeremyfiel no, I'm saying AJV doesn't run the code for determining unevaluatedProperties if the schema doesn't have any property-related keywords (properties, additionalProperties):
Wheel:
      allOf:
        -  $ref: "#/components/schemas/Tire"
        -  $ref: "#/components/schemas/Rim"

We force unevaluatedProperties in AJV but the corresponding code is not run for the Wheel schema. We can fix it.
We don't use any strict mode settings, and we don't use strictSchemas: true, see here: https://github.com/Redocly/redocly-cli/blob/main/packages/core/src/rules/ajv.ts#L17

I'm also running into this issue. My schema looks something like this though:
Thing:
  type: object
  properties:
    type:
      type: string
    id:
      type: string
    attributes:
      type: object
      allOf:
        - properties:
            tags:
              type: array
        - '$ref': 'shared.yml#/DefaultFields'

When I put additional, unexpected properties inside attributes in an example, the no-invalid-media-type-examples rule does not return an error.
However, the workaround of doing:
Thing:
  type: object
  properties:
    type:
      type: string
    id:
      type: string
    attributes:
      unevaluatedProperties: false # added this -- properties: {} also works
      type: object
      allOf:
        - properties:
            tags:
              type: array
        - '$ref': 'shared.yml#/DefaultFields'

Gets no-invalid-media-type-examples to return something like this:
Example value must conform to the schema: attributesproperty must NOT have unevaluated propertiessomething-extra.

I'm getting the error to display when I add additional properties without using properties: {}.
Which version of the CLI are you using? I'm on 1.5.0

1. openapi: '3.1.0'
2. info:
3.   title: Title
4.   version: '1.0.0'
5. servers:
6.   - url: 'https://api.redocly.com'
7. security: []
8. paths:
9.   /api/v1/things/{thing-id}:
10.     get:
11.       summary: a summary
12.       description: delete with requestBody
13.       parameters:
14.         - name: thing-id
15.           in: path
16.           required: true
17.           schema:
18.             type: string
19.       responses:
20.         '200':
21.           description: OK
22.           content:
23.             application/json:
24.               schema:
25.                 type: object
26.                 required:
27.                   - attributes
28.                 properties:
29.                   attributes:
30.                     allOf:
31.                       - type: object
32.                         properties:
33.                           tags:
34.                             type: array
35.                             items:
36.                               type: string
37.                       - $ref: '#/components/schemas/anotherSchema'
38.               examples:
39.                 empty_schema:
40.                   value: {}
41.                 attr_w_tags:
42.                   value:
43.                     attributes:
44.                       tags:
45.                         - thing1
46.                         - thing2
47.                         - thing3
48.                 attr_w_other_fields:
49.                   value:
50.                     attributes: {}
51.                     things: {}
52.                     otherThings: []
53.                     booleanThing: true
54.                     anotherThing: false
55. components:
56.   schemas:
57.     anotherSchema:
58.       type: object

rule-error.yaml:
  40:26  warning  no-invalid-media-type-examples  Example value must conform to the schema: must have required property 'attributes'.
  51:21  warning  no-invalid-media-type-examples  Example value must conform to the schema: must NOT have unevaluated properties `things`.
  52:21  warning  no-invalid-media-type-examples  Example value must conform to the schema: must NOT have unevaluated properties `otherThings`.
  53:21  warning  no-invalid-media-type-examples  Example value must conform to the schema: must NOT have unevaluated properties `booleanThing`.
  54:21  warning  no-invalid-media-type-examples  Example value must conform to the schema: must NOT have unevaluated properties `anotherThing`.

Hey, thanks for the quick response. I should've been clearer about what example I'm trying to get to fail. Your attr_w_other_fields isn't quite right. This reproduces what I'm seeing:
openapi: '3.1.0'
info:
  title: Title
  version: '1.0.0'
servers:
  - url: 'https://api.redocly.com'
security: []
paths:
  /api/v1/things/{thing-id}:
    get:
      summary: a summary
      description: delete with requestBody
      parameters:
        - name: thing-id
          in: path
          required: true
          schema:
            type: string
      responses:
        '200':
          description: OK
          content:
            application/json:
              schema:
                type: object
                properties:
                  attributes:
                    allOf:
                      - type: object
                        properties:
                          tags:
                            type: array
                            items:
                              type: string
                      - $ref: '#/components/schemas/anotherSchema'
              examples:
                attr_w_tags:
                  value:
                    attributes:
                      tags:
                        - thing1
                        - thing2
                        - thing3
                attr_w_other_fields:
                  value:
                    attributes:
                      tags: []
                      things: {}
                      otherThings: []
                      booleanThing: true
                      anotherThing: false
components:
  schemas:
    anotherSchema:
      type: object

I expect to see
Example value must conform to the schema: `attributes` property must NOT have unevaluated properties `things`.
Example value must conform to the schema: `attributes` property must NOT have unevaluated properties `otherThings`.
Example value must conform to the schema: `attributes` property must NOT have unevaluated properties `booleanThing`.
Example value must conform to the schema: `attributes` property must NOT have unevaluated properties `anotherThing`.

but I don't. Including properties: {} on the same level as the allOf, does cause these errors to manifest.
And FWIW I was on CLI version 1.3 but I updated to 1.6 and still see this issue.

So a couple things here..
The allOf by itself, without any properties or additionalProperties keyword used is basically an open schema at the root. So you shouldn't have an error if you provide additional properties in the attributes schema.
It would be the equivalent of using addtiionalProperties: true or properties: {}
schema:
                type: object
                properties:
                  attributes:
                    additionalProperties: true
                    allOf:
                      - type: object
                        properties:
                          tags:
                            type: array
                            items:
                              type: string
                      - $ref: '#/components/schemas/anotherSchema'

schema:
                type: object
                properties:
                  attributes:
                    properties: {}
                    allOf:
                      - type: object
                        properties:
                          tags:
                            type: array
                            items:
                              type: string
                      - $ref: '#/components/schemas/anotherSchema'

BUT there is one caveat mentioned by Roman which is they force unevaluatedProperties/additionalProperties: false on their example evaluation because they feel a lot of users don't understand some of these JSON Schema intricacies. That means they have introduced some default behavior that is unexpected by advanced JSON Schema users. Again, this is an opinionated linting tool to help the majority so it's understandable why they've made that decision.
By adding properties: {}, they are triggering some Ajv validation code to run against that schema, and then applying their own forced additional properties logic against the schema, which is outputting the errors. Again, using properties: {} by itself, should not produce any errors from JSON Schema perspective.
The work around of providing properties: {} still remains the fix IF you want to see errors in that location. This is non-conformant behavior for JSON Schema.
If you want to disable the behavior of properties: {} incorrectly outputting an error, modify your config file:
rules:
  no-invalid-media-type-examples:
    disallowAdditionalProperties: false

https://github.com/Redocly/redocly-cli/issues/552#issuecomment-1072948715

Thank you for your thorough response! I am indeed new to JSON Schema and OpenAPI, so I guess I appreciate this unexpected behavior 😅  This makes sense now. I will continue to keep the properties: {} logic because it's the best way to ensure my schema and examples are kept accurate.


96boards-tools: update srcrev

Update the recipe to pull latest version

Riku Voipio commented:
Well after some testing that didn't fix the issue. Still investigating.

No other changes to 96boards-tools is expected, so feel free to merge this.

Updated version and srcrev


autocomplete list for child resources should be filtered based on the parent

If I'm adding a child resource to the vnet, the autocomplete list should be restricted to child resources of the parent.

Example:

Just FYI, we've had some issues with the Azure schema CDN and some other things that have delayed this release.  Will have it out as soon as possible, hopefully next week.

Fix went out today with this functionality, thanks.


MB-15611: add factory for OriginDutyLocationGBLOC and use in BuildOrders

Jira ticket for this change
Summary
Orders now have OriginDutyLocationGBLOC, set it and add factory
Setup to Run Your Code

💻 You will need to use one terminalsto test this locally.
Terminal 1
Run the factory tests
go test -count 1 ./pkg/factory/...

Messages

:book:
:link: MB-15611

  Generated by :no_entry_sign: dangerJS against 69bcd7fc6e77f17be1ef01ba3f50a902635e7a3d


Issues with closed ports inside containers

I am trying to understand how does port forwarding work with vctl as none of my apps within containers report having their ports open to the outside world via NAT.

Click on a 🔧  icon to access Settings
Click on the Network tab, wait a few seconds
Check status of a default port

Expected result
The following ports should be open:

51413/tcp
51413/udp

Actual result
The port is closed
Additional info

The same ports (more than one) have been tested on the host using another instance of Transmission, and the port appears open, so the NAT is definitely working.
There is no "double NAT" issue on the tested network
Port forwarding isn't active on the router

I just realised that vm containers are behind a different network, which makes it impossible for NAT to work, and this a manual port forwarding needs to be setup on the router to achieve desired outcome.
Closing this issue, but feel free to provide a bit more insight (or point me to the right page) into how networking inside vctl in general.

So the short answer is 'it depends on how you're accessing it'.
For instance, running the container creates a new container host ('CRX VM' appliance), and that VM gets an IP on the vmnet that it is connected to. In this case, the VM gets it's IP from the vmnet and port forwarding is not required when accessing it from the local host. (i.e. you just go to "172.x.x.x:51413" or whatever the IP is).
When you want to share the service outside of the computer it's running on, that's where the -p flag applies.
What it does in this context is create a NAT rule on the vmnet directly. So you can access it from outside the local host by hitting the IP of the physical device and the port you're exposing. That gets forwarded to the "172.x.x.x" or whatever IP the VM gets.
That said, the port forwarding feature of the vmnet is historically a Pro only feature.
So while Fusion and Workstation Player can deploy a container and access it from the local host, it can't be shared in the same way (so you'd need to create some rule at your router to do the port forwarding)


WEBSITE_CONTENTSHARE and WEBSITE_CONTENTAZUREFILECONNECTIONSTRING settings are not allowed on linux consumption plan.

Right now documentation says that "On Windows, a Consumption plan requires two additional settings in the site configuration: WEBSITE_CONTENTAZUREFILECONNECTIONSTRING and WEBSITE_CONTENTSHARE." But it doesn't mention that these settings are not allowed on Linux and will break the application if used.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a1ba8817-3e01-7e84-6c71-b98b131f2927
Version Independent ID: a7703386-9e83-1075-72ef-f9e63f3dc4de
Content: Automate function app resource deployment to Azure
Content Source: articles/azure-functions/functions-infrastructure-as-code.md
Service: azure-functions
GitHub Login: @ggailey777
Microsoft Alias: glenga

@older Thank you for the question. We will review and get back to you.

@older Thank you for the question. We will review and get back to you.

@older Functions on windows and Linux work differently. Those settings are not needed/ expected for Linux consumption function apps. So you can continue without those. Also, for Linux it is recommended to use the run from package instead and/or use a container

@older Functions on windows and Linux work differently. Those settings are not needed/ expected for Linux consumption function apps. So you can continue without those. Also, for Linux it is recommended to use the run from package instead and/or use a container

@older Functions on windows and Linux work differently. Those settings are not needed/ expected for Linux consumption function apps. So you can continue without those. Also, for Linux it is recommended to use the run from package instead and/or use a container.

@older Functions on windows and Linux work differently. Those settings are not needed/ expected for Linux consumption function apps. So you can continue without those. Also, for Linux it is recommended to use the run from package instead and/or use a container.

@JayaChatterjee-MSFT Yes, I realise that. The reason I suggested to change the documentation is that I actually had my function app conditionally deployed from ARM template on Windows or Linux consumption plan depending on region. And I had these 2 settings both on Linux and Windows. But my application often caused functions runtime errors when running on Linux so I opened support request with Microsoft and got a response that these 2 settings are not allowed on Linux and are the reason for functions runtime errors in my app.
So I thought that the documentation has to be updated to reflect this.

@JayaChatterjee-MSFT Yes, I realise that. The reason I suggested to change the documentation is that I actually had my function app conditionally deployed from ARM template on Windows or Linux consumption plan depending on region. And I had these 2 settings both on Linux and Windows. But my application often caused functions runtime errors when running on Linux so I opened support request with Microsoft and got a response that these 2 settings are not allowed on Linux and are the reason for functions runtime errors in my app.
So I thought that the documentation has to be updated to reflect this.

Hello @older, in this document https://docs.microsoft.com/en-us/azure/azure-functions/functions-infrastructure-as-code#linux - it is mentioned that these two settings are required only for the windows. Does that help?

Hello @older, in this document https://docs.microsoft.com/en-us/azure/azure-functions/functions-infrastructure-as-code#linux - it is mentioned that these two settings are required only for the windows. Does that help?

@JayaChatterjee-MSFT No, it doesn't. There's no mention that these settings are not allowed on Linux.

@JayaChatterjee-MSFT No, it doesn't. There's no mention that these settings are not allowed on Linux.

@ggailey777 Could you please review the request and confirm if we need to add it separately for Linux as well?

@ggailey777 Could you please review the request and confirm if we need to add it separately for Linux as well?


[bug] Inconsistent Suggestions in Command Input Box

Issue Description:
The command input box is not providing consistent suggestions based on user input. When typing a partial query, such as "se," the input box should display relevant suggestions. However, when additional characters are added or removed, the suggestions do not consistently reflect the available options, causing confusion for users.
Steps to Reproduce:

Open the link https://ui.shadcn.com/docs/components/command and navigate to the command input box.
Start typing a partial query, such as "se," to trigger suggestions.

Expected Behavior:

The input box should show relevant suggestions based on the partial query.

Continue typing or modifying the query and the press backspace.

Expected Behavior:

The input box should update and display relevant suggestions based on the current query.

Actual Behavior:

The suggestions in the input box are not consistently updated to reflect the available options as the query is modified.

Additional Information:

This issue impacts the user experience and may lead to confusion as users expect the input box to provide accurate and up-to-date suggestions.
It's important to ensure that the command input box dynamically adjusts its suggestions as the user types or modifies their query to provide a smooth and intuitive user experience.
This issue should be addressed to improve the overall usability of the command input box.

I would like to work on this issue and contribute to its resolution. Please assign it to me.

i had the same issue and what i can contribute to solve this issue right now is that under the hood cmdk uses the commandItem dom element's textContent to search the commands

Bumping this.
Is there a configuration to change this behaviour in the  component? It's a bit frustrating that it only updates the result when the input becomes longer, but nothing happens when characters are deleted from it (except when removing everything).
Also, pressing "enter" doesn't trigger the first element in the list.

The result update when removing characters has been fixed as part of cmdk 0.2.1 🎉