The Response object's default field is confusing and inconsistently interpreted

According both 3.0 and 3.1, in theResponse object there MUST be at least one response code and it SHOULD be the response for a successful operation call. In addition the default describes “responses other than the ones declared for specific HTTP response codes”.
This means that if we had a 200 response and default, the latter would cover everything except for 200. Many seem to interpret default to mean something like response categories not covered by explicit response codes. Consider this example from the OAI org:
responses:
        '200':
          description: pet response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Pet'
        default:
          description: unexpected error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Error'

https://github.com/OAI/OpenAPI-Specification/blob/main/examples/v3.0/petstore-expanded.yaml
This implies that default covers error types when in fact it covers non-error responses as well. In fact, the consumer wouldn't necessarily know if the API might reply with another success code such as a 204. A more precise example might look like this:
responses:
        '200':
          description: pet response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Pet'
        4xx:
          description: unexpected error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Error'
        5xx:
          description: unexpected error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Error'

I would suggest the following actions:

Update the example to interpret the spec more literally
Clarify default since this confusion is not limited to this example, but extends to APIs in the wild
Consider in 3.2 or 4.0 eliminating default
Consider in 3.2 or 4.0 adding a new named error category that means "both 4xx and 5xx". This could be used in the way that default is often misused today

@ahl looking over this to see where it might fit in terms of releases and SemVer, I have a few questions:

Why not simply use a 2XX to exclude the other 200-series codes from default?  The spec makes it clear that the 200 response would take precedence over 2XX when an actual 200 response is returned.
Since Response Objects can be referenced, why not just $ref the same response from both 4XX and 5XX?  That is two additional lines of spec, but that's not much in the grand scheme of OAS versbosity.

I agree that regardless, the wording of the examples is misleading, so I will tag this for 3.0.4 and 3.1.1 as those aspects should be fixed in those patch versions, regardless of what might happen in 3.2.

I'm also going to tag this as "3.0 needs documentation" because we could explain it on the learn.openapis.org site and I think that's what that label is for (@MikeRalphson @webron is that correct?)

Why not simply use a 2XX to exclude the other 200-series codes from default?  The spec makes it clear that the 200 response would take precedence over 2XX when an actual 200 response is returned.

I want to describe an operation that

responds with a 200 upon success and a particular response body
does not respond with any other 2xx responses
responds with a different response body with all 4xx and 5xx responses

In other words, if the API were to erroneously respond with a 204 there would be no guarantee that the response body conformed to the same schema as would, say, a 400 response.

Since Response Objects can be referenced, why not just $ref the same response from both 4XX and 5XX?  That is two additional lines of spec, but that's not much in the grand scheme of OAS versbosity.

I think that's the accurate way of describing the example. Why don't others just do that? I'd imagine they're being less precise, are confused, or are blithely following examples... from the OAI org.
What would you say an intended or well-formed use of default? How would it be used on its own? How would it be used in concert with other, discrete status codes?

I agree that regardless, the wording of the examples is misleading, so I will tag this for 3.0.4 and 3.1.1 as those aspects should be fixed in those patch versions, regardless of what might happen in 3.2.

Again, consider this example: https://github.com/OAI/OpenAPI-Specification/blob/main/examples/v3.0/petstore-expanded.yaml
Do the operations return a single success code or might there be other ones? If the operation above responded with a 204 would its response body conform to #/components/schemas/Error? Seems unlikely, but the definition of default suggests that's the case.

@handrews the documentation label definitely predates the learn.openapis.org site, however it is a good fit to indicate a PR against learn would be appropriate.

I want to describe an operation that

responds with a 200 upon success and a particular response body
does not respond with any other 2xx responses
responds with a different response body with all 4xx and 5xx responses

In other words, if the API were to erroneously respond with a 204 there would be no guarantee that the response body conformed to the same schema as would, say, a 400 response.

In my opinion, the onus is on you to document the known responses. When you say "if the API were to erroneously respond with a 204 there would be no guarantee that the response body conformed to the same schema as would, say, a 400 response", that's not an OpenAPI problem. If your API can return a 204, you should document it as such unless it happens to have the same structure as default...if you provide it. default is only useful when you have a collection of status codes that have the same response structure, which in many/most cases error responses would apply. So to me, OpenAPI can't be expected to handle "erroneous [responses]" as that is a failure on the API to adhere to the contract.

In my opinion, the onus is on you to document the known responses.

On the API description author... sure! I would merely observe that mistakes have occurred in the past, and are likely to occur in the future.

So to me, OpenAPI can't be expected to handle "erroneous [responses]" as that is a failure on the API to adhere to the contract.

Perhaps the hypothetical was confusing. How about this:
Consider https://github.com/OAI/OpenAPI-Specification/blob/main/examples/v3.0/petstore-expanded.yaml
What response codes does, say, the addPets operation respond with? One could infer that 200 is the only success response, but what in the OpenAPI spec tells us that this is so? If default actually, de facto, is meant to describe "responses in http status code categories not covered by explicit or ranged response descriptions", then perhaps the spec should say that.

One could infer that 200 is the only success response, but what in the OpenAPI spec tells us that this is so? If default actually, de facto, is meant to describe "responses in http status code categories not covered by explicit or ranged response descriptions", then perhaps the spec should say that.

It does say that. It says, for the default keyword:

The documentation of responses other than the ones declared for specific HTTP response codes.

So if you only document 200 and default, your document is specifically saying that anything but 200 must conform to the schema specified for default. If that references an Error schema, then you may conclude that there are no other 2xx (non-error) responses to be returned, and if the application does so it may be considered a violation of the schema.

One could infer that 200 is the only success response, but what in the OpenAPI spec tells us that this is so? If default actually, de facto, is meant to describe "responses in http status code categories not covered by explicit or ranged response descriptions", then perhaps the spec should say that.

It does say that. It says, for the default keyword:

The documentation of responses other than the ones declared for specific HTTP response codes.

The additional detail I suggested was intentionally to distinguish it from the existing definition of "default". In particular the language "in categories not covered by explicit or ranged responses". This may be another case where more words would be clearer than fewer. My suggestion is that default would apply to status code ranges that are otherwise not mentioned. For example, if there is an entry for a 200, then default would not apply to the 2xx range. If there was a 404, default would similarly not apply to the 400 range.
Put perhaps another way. A common use of default is to cover the 4xx and 5xx range with an error response type. I think there would be utility in a spec being able to say "this is what a 200 response looks like and there will be no other 2xx responses".
Or perhaps "default" is insufficiently specific, and an "error" response that covers the 4xx and 5xx range would be more useful and intelligible.

if there is an entry for a 200, then default would not apply to the 2xx range. If there was a 404, default would similarly not apply to the 400 range.

This would be a breaking change, so it couldn't be done until OAS 4 "Moonwalk" where things are being re-imagined on a larger scale, including API-level responses.  You might want to follow those discussions to see if there's an analogous concern with the new format.
On the one hand, I kind of like the error idea, but since it's already possible to $ref Response Objects, you can accomplish it by just $ref-ing the same response from 4XX and 5XX.  It doesn't seem worth adding more work for implementations just to save one reference.


Fix allow optional attributes on decodeAttributes

Empty events emitted will result in missing attributes field in events object, which is a legit scenario.
{
   "type": "cosmos.module.EmittedEvent"
}

Which causes decodeAttributes assertArray throw an exception UnhandledPromiseRejectionWarning: Error: Value must not be undefined. I'm proposing to make this field optional.

Thank you for bringing this up. Could you please have a look in https://github.com/cosmos/cosmjs/pull/1199 for an alternative implementation?

Shipped as part of CosmJS 0.29.0


added scope for finding mutual friends with test cases

Coverage increased (+0.005%) to 99.787% when pulling f27596769c9b74ed6bb852c41f6e463df1a8351e on whatz-admin:extend-has-friendship into 6613f22aacea4ff7ed7812b085f61771f92e90c8 on sungwoncho:master.

This is a useful feature. But we already have friends_with? here and the method name friend_with is too similar/ambiguous.
Also any reason to use instance_eval rather than defining an instance method?


Added Password with Reveal Eye option

Hi,
I've added a Reveal Password option to the RadzenPassword class.
A couple things to note:
I could not figure out the CSS structure so I re-used some classes from rz-calendar. I would be happy to fix/update this if I knew how.
I tried to follow Radzen's naming conventions.
I added a test as well as updated the components demo page.
Also, if you don't like this or the way it was done, I will not take it personally. Just trying to contribute.
Thanks for any consideration!

Hi @codepend,
Thanks for the pull request!
There is a problem when width is defined

@enchev  I see. I will try and figure out.

I have implemented some changes which I believe resolve previously mentioned issues as well as added ability to allow password reveal while the component is Disabled. Happy to work on any issues you see with it.

please rebase your branch against the master in order to check your changes

I rebased this, believe it is up to date now using the new "config" components for demo pages

The latest implementation is problematic as well. The Style is rendered twice which is wrong. Also there is another wrapper which seems unnecessary. I am closing this PR as this feature needs to be addressed by a more robust rendering implementation.

Thanks again for the consideration. I will try and work on something more appropriate.

The biggest problem is the rendering. Adding a wrapper element is a breaking change and should be fully tested that it doesn't break too many cases. This is why we prefer to implement it ourselves. Thank you for your efforts though! We appreciate the good will.
If you need this for your application I would implement it in a custom component which renders RadzenPassword or RadzenTextBox depending on a property value which is toggled by the icon.


Use relative paths to cwd for reading config/data in train entrypoint

The previous approach to specifying the paths to data/config was confusing me because the files were read in as relative to the src/train.py module rather than being relative to the cwd that the user is in when they do modal run.
As part of this changed I moved the default config and dataset out of src/ asexample_configs/codellama/config.yml and datasets/sqlqa.jsonl.
I wonder if we should have these as the default values for the train CLI or if users should always have to pass a config / data. I think not having defaults might make sense since anything beyond the simplest quickstart example would need to supply both values (since dataset information is also part of the config, you couldn't just hold the config constant and iterate on the dataset, e.g., so the default config isn't really a good "base config" for experimentation on users' own data).
(Extremely fussy side note: if we expect users to build on this it might make more sense to have config than example_config).

Do you think it makes sense to flatten configs to make it more accessible?

I don't think I feel strongly here! The nested structure maybe makes it a little bit more obvious what models we "support".

Maybe a point in favor of a flat structure is that we could avoid --config=config/codellama/config.yml which is too many repetitions of "config" imo.


Unification of additional tag keys

Hi,
is it currently to unify multiple keys into one column? 'mappings' is only for alternative entity classes if I understood correctly. For example there is wild use to tag colors for Simple-3D-Buildings at the moment which would be nice to be put into one column (of course it would be better to clean up the data in OSM):
building:colour (94 662)
building:color (6 926)
colour in combination with building (3000)
color in combination with building (2500)
roof:colour (~150000)
building:roof:colour (~10000)
roof:color (~10000)
building:roof:color (~100)

The other option is to use coalesce in SQL: map both column, use the colour tag if possible, otherwise color. As you said, "of course it would be better to clean up the data in OSM".

Closing this as it's unlikely that this feature will be supported directly in the future. This could be solved with scripting support. However, there is no time frame for scripting. Right now HStore would be the best solution.


Is LittleProxy competable with Android?

Dear LittleProxy team,
I just wonder: would LittleProxy be working fine within Android application?
I tried to create http Proxy server using LittleProxy:
compile group: 'org.littleshoot', name: 'littleproxy', version: '1.0.0-beta7'

public class MyHttpProxyServer {
    org.littleshoot.proxy.HttpProxyServer server;
    public MyHttpProxyServer(){
        server = DefaultHttpProxyServer.bootstrap().withAddress(InetSocketAddress.createUnresolved("<IP_ADDRESS>", 8080)).start();
    }
}

I added this lines to gradle.build file, bc gradle didn't want build app otherwise:
packagingOptions {
    exclude 'META-INF/DEPENDENCIES.txt'
    exclude 'META-INF/LICENSE.txt'
    exclude 'META-INF/NOTICE.txt'
    exclude 'META-INF/NOTICE'
    exclude 'META-INF/LICENSE'
    exclude 'META-INF/DEPENDENCIES'
    exclude 'META-INF/notice.txt'
    exclude 'META-INF/license.txt'
    exclude 'META-INF/dependencies.txt'
    exclude 'META-INF/LGPL2.1'
}

And here my emulator refused to start app:
Failure [INSTALL_FAILED_NO_MATCHING_ABIS]
When I comment all LittleProxy import - emulator again works fine. So I doubt: is LittleProxy compatible with Android?

Hi, I have it in my daily use with Android, but I use the Eclipse and
Maven to do the build, no Gradle. It's not in production state :-) , so
it's not public this time (but working for years). I use a more recent
version of Netty 4.1.0.Beta5 which claims to support Android, but some
older versions works too. Some Netty versions (4.1.0.Beta3,
4.0.25.Final) failed with my requirements. It's NIO, very close to the
system, this means you have to test every supported platform!
I've included for Android additionally to the Java version:
 <dependency>
   <groupId>com.github.tony19</groupId>
   <artifactId>logback-android-classic</artifactId>
   <version>1.1.1-2</version>
   <scope>compile</scope>
 </dependency>

And probably more important I've excluded this:
     <exclusion>
       <artifactId>slf4j-log4j12</artifactId>
       <groupId>org.slf4j</groupId>
     </exclusion>
     <exclusion>
       <artifactId>log4j</artifactId>
       <groupId>log4j</groupId>
     </exclusion>
     <exclusion>
       <artifactId>barchart-udt-bundle</artifactId>
       <groupId>com.barchart.udt</groupId>
     </exclusion>
     <exclusion>
       <artifactId>bcprov-jdk15on</artifactId>
       <groupId>org.bouncycastle</groupId>
     </exclusion>
     <exclusion>
       <artifactId>bcmail-jdk15on</artifactId>
       <groupId>org.bouncycastle</groupId>
     </exclusion>

You have to try it. Please consider to create an example...
Am 03.07.2015 um 17:18 schrieb Sersz Rudziankoŭ:

Dear LittleProxy team,
I just wonder: would LittleProxy be working fine within Android application?
I tried to create http Proxy server using LittleProxy:
compile group: 'org.littleshoot', name: 'littleproxy', version:
'1.0.0-beta7'
public class MyHttpProxyServer {
org.littleshoot.proxy.HttpProxyServer server;
public MyHttpProxyServer(){
server =
DefaultHttpProxyServer.bootstrap().withAddress(InetSocketAddress.createUnresolved("<IP_ADDRESS>",
8080)).start();
}
}
I added this lines to gradle.build file, bc gradle didn't want build app
otherwise:
packagingOptions {
exclude 'META-INF/DEPENDENCIES.txt'
exclude 'META-INF/LICENSE.txt'
exclude 'META-INF/NOTICE.txt'
exclude 'META-INF/NOTICE'
exclude 'META-INF/LICENSE'
exclude 'META-INF/DEPENDENCIES'
exclude 'META-INF/notice.txt'
exclude 'META-INF/license.txt'
exclude 'META-INF/dependencies.txt'
exclude 'META-INF/LGPL2.1'
}
And here my emulator refused to start app:
Failure [INSTALL_FAILED_NO_MATCHING_ABIS]
When I comment all LittleProxy import - emulator again works fine. So I
doubt: is LittleProxy compatible with Android?
—
Reply to this email directly or view it on GitHub
https://github.com/adamfisk/LittleProxy/issues/214.

Exclude only barchart-udt-bundle will solve your problem, because the jar contains native libraries but not include arm build. I suggest author should provide an example for android too.

I like to have log files on my device. Therefore I use logback-android-classic and exclude log4j and slf4j-log4j12 used by my applications core module. A LogConfiguration class enables a LogcatAppender and a FileAppender in my Android app.
I use BouncyCastle and sqlite4java in my application, but BouncyCastle won't work for me with Android. Since using sqlite4java, I have to exclude bc to build the Android app. Of course, this is my experience only.

instead of BouncyCastle on android, use SpongyCastle

Hi Stephen,
I'm successfully using LittleProxy and BouncyCastle on Android. My main
problem using SSL on Android is fixed by using a recent Netty version.
I've seen the same problems using SpongyCastle before.
You could try a binary of my application from here:
https://ganskef.github.io/MoCuishle/
Please send me a hint if it fails on your platform.
Regards Frank
Am 11.03.2016 um 18:52 schrieb Stephen Gutknecht:

instead of BouncyCastle on android, use SpongyCastle
—
Reply to this email directly or view it on GitHub
https://github.com/adamfisk/LittleProxy/issues/214#issuecomment-195477537.

Hi Frank - your app worked fine on my Android M beta 1 system, I was able to use the Android Firefox app and see it logging the various sites I visited.
I'm hoping some of us can put together a simple Github app that shows how to use LittleProxy with gradle and Android Studio.  I see you have your own fork and some fixes of LittleProxy?

@RoundSparrow Is there a sample app yet on using it with gradle?

Sorry, I don't have anything current.

What worked for me was to add this to my app's build.gradle:
configurations {
    all*.exclude group: 'com.barchart.udt', module: 'barchart-udt-bundle'
}

Sorry to everyone I've pinged and doesn't care anymore about this but it seemed like the most logic place to post.

implementation ('org.littleshoot:littleproxy:1.1.2'){
    exclude group: 'com.barchart.udt', module: 'barchart-udt-bundle'
}

There are a lot of CVE in Spongy Castle.
You need to use Bouncy Castle:

https://www.bouncycastle.org/releasenotes.html


Add webhook logging system

I think the webhook should be in my test server so we don't clog up the channels on the Geyser Discord but it could be anywhere

I'm not sure if the polling system with setInterval is clean but it works and it groups messages together


Git head broken

Just FYI:  The file scripts/greaseweazle/version.py is missing from latest head revision.

This file is generated as part of the build. make pysetup will generate it for you.

Thanks, Keir.  That does work, but the step you describe does not seem to be mentioned anywhere.


Update availability-set-overview.md

The following fact should be clearly mentioned for better clarity:
"Note that when you deploy VMs in an availability set, the VMs are not deployed into different datacenters."

@mimckitt
Can you review the proposed changes?
When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

@mimckitt
Can you review this old PR and determine whether it needs to be closed or merged?
@MicrosoftDocs/public-repo-pr-review-team

Thank you for your interest in contributing to our documentation. Unfortunately, the changes in this pull-request cannot be accepted because our Content group is migrating content for the "virtual-machines", "virtual-machines-scale-set", "container-instances", and "service-fabric" folders to a new repository: azure-compute-docs.
Short-term, if there are changes in this PR for other folders, you can move forward with those by removing the change(s) to all the above mentioned folders' content.
In the meantime, we are closing this PR and tagging it so that when the content is available in the new repo, we can follow up with the new location information.
#please-close
#label:"migrate-followup"

@azarboon  We’re happy to share that contributions to virtual-machines are once again possible at azure-compute-docs. Please open a PR in the new repo to submit your changes. We appreciated your patience during this migration.


Detect dynamic Gradle property access via Project object

Follow-up to #18946

@bot-gradle test QFL

OK, I've already triggered QuickFeedbackLinux build for you.

@bot-gradle test ACC

OK, I've already triggered AllConfigCacheTestsReadyForMerge build for you.


metadata-added-provenance

Phases
MP
MM
MB
MR
JM
Total

ETA
0
0
0
0
0
0

Developing
30
0
0
0
0
0

Review
0
0
0
0
0
0

Total
-
-
-
-
-
0

ETA est.

0

ETA cust.
-
-
-
-
-
0

Problem description
After adding, deleting, or editing the metadata of an item, a new record should be created in the metadata as dc.description.provenance.

Problem: the page is loading data from cache?

Checks are not green


Update styles in SQFormScrollableCard

Description:
As a developer, I need to update the SQForm components, so that they match design library patterns.
Updates needed on SQFormScrollableCard:
Follow specifications per component design: SC3 DESIGN LIBRARY
Changes needed:
Header = 49px height
(Text style H5 of 24pt weight 400 + 12px padding above below header + 1px divider line)
Footer =,47px height
(Button height = 30px + 8px padding above below header + 1px divider line)
Padding around content should be 24px top, 16px left and right, and 32px bottom

PR covering both the updates to the SQFormScrollableCard styles and updates to SQFormHelperText styles: https://github.com/SelectQuoteLabs/SQForm/pull/822


add transformers api

Continuing from this comment: https://github.com/ai/keyux/pull/15#discussion_r1523891374
Implementing the transformers API and switching overrides to transformers.

Looks great. Thanks.


TypeScript type definition

I wrote it and using locally..!
I'm afraid which one is better to send PR to:

This repo (so when interface changed type definition file should be modified at the same time).
DefinitelyTyped (repo for user-contributed type definition files).

Thanks..!

I think it's better to keep the "type" stuff off this project.
No everyone use TypeScript
Adding it might cause un-sync state of implementation and declaration, especially when accepting PR.

Agreed, send it to DefinitelyTyped

Let's go for DefinitelyTyped

@ypresto did you end up pushing this to DefinitelyTyped? I couldn't seem to find it :)

@ypresto could you push it to DefinitelyTyped or just post it here please.

Sent PR to DefinitelyTyped..! https://github.com/DefinitelyTyped/DefinitelyTyped/pull/21693

Seems merged already
Feel free to update the doc here

Can anybody share some example of how to use it with typescript? I have difficulties trying to use it.

Can someone help me? I don't get it how to use typescript with this lib
import I18n from 'i18n-js';
import * as RNLocalize from 'react-native-localize';

import en from './locales/en';
import pt from './locales/pt';

const locales = RNLocalize.getLocales();

if (Array.isArray(locales)) {
  I18n.locale = locales[0].languageTag;
}

I18n.fallbacks = true;
I18n.translations = {
  en,
  pt,
};

export default I18n;

    <Text>{I18n.t('noAutoCompleteHereWhy?')}</Text>

@Rodrigo816 I suggest you open another issue to ask your question
with more details on what you encountered and what you try to achieve

@Rodrigo816 This library is just a JS lib and not responsible for autocompletion. If you use vscode you can contribute i18n-ally to support this library.

For me, the solution was similar to what I wrote here. I installed the types for i18n-js, and then I prepend global. before I18n.


sec gas mask hailer

About the PR
hailer action with 5s cooldown to say a message depending on combat mode or obscenities when emagged
Technical details
HailerComponent with the logic and requires Speech to say things
Media
combat mode off/on/on

emag sus

[X] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Breaking changes
no
Changelog
:cl:

add: Security gas masks now have hailers to shout warnings during a chase.

it is time....


add password options + fix password prompts

This fixes password prompts, which were did not appear on the console, and adds options for btc-hot to provide passwords directly from the command line (for unattended usage).

Very good improvement, thank you! Let me know what do you think about my comment, otherwise LGTM


Fix old JIT linkage when lowering DUFunc calls.

Moved DUFunc._lower_me() into a callable object that keeps track of libraries needed for linkage during lowering.
Thanks to @sklam for figuring this out and presenting a proof-of-concept fix.

Apart from the question, looks good to me.

We've crossed an important threshold where we are discussing readability for a longer time than it would take to try to make improvements. Patch committed.

Indeed. Style guides generally exist to avoid this kind of back and forth. Or knowledge of what is generally considered idiomatic or not :-)


replace deprecated terraform stanza

as part of the terraform upgrade performed in #2054, we received a deprecation warning covering a statement in aws_s3_bucket.logs. This is not preventing upgrades right now, but will become a blocking error for some future terraform upgrade
│ Warning: Argument is deprecated
│
│   with aws_s3_bucket.logs,
│   on storage.tf line 7, in resource "aws_s3_bucket" "logs":
│    7: resource "aws_s3_bucket" "logs" {
│
│ Use the aws_s3_bucket_acl resource instead
│
│ (and 3 more similar warnings elsewhere)

It would also be useful to consider replacing the cert or ecr modules (or others) as part of the same cleanup task.


Databases are not created like storage schema dictates

Hi!
I installed Graphite at a customers of mine and the installation has a problem.
When start or end of the interval to show in a graph is outside the first interval of the storage schema, the graph is empty. This happens with Graphite-Web and Graphana (which uses Graphite-Web to fetch data from carbon)
In the log I see the possible reason for this problem.
This storage schema is:
 [carbon]
pattern = ^carbon\.
retentions = 60:90d

[icinga_internals]
pattern = ^icinga\..*\.(max_check_attempts|reachable|current_attempt|execution_time|latency|state|state_type)
retentions = 5m:7d

[icinga_default]
pattern = ^icinga\.
retentions = 1m:2d,5m:10d,30m:90d,360m:4y

And the creates.log reads:
09/02/2015 12:37:39 :: new metric [metricname] matched schema icinga_default
09/02/2015 12:37:39 :: new metric [metricname] matched aggregation schema default
09/02/2015 12:37:39 :: creating database file [databasename] (archive=[(60, 2880), (300, 2880), (1800, 4320), (21600, 5840)] xff=None agg=None)

The 5m interval is not set to 10 days like the schema says, but 2 days exact like the 1m interval.

@widhalmt Actually the 5m archive is correct: 2880 points at 300s resolution = 10 days. Something else must be causing your problem. Have you tried using whisper-dump on a whisper file to see if it actually contains the archives and data you expect?

A whisper-dump shows, that no archive except the first one contains any data.

@eyesonlyone @widhalmt are you having the same issue or working together on the same installation?  Can you confirm which version you are running (master, 0.9.12, 0.9.x)?  I haven't seen this happen, but it does look like a bug.

Yes we are working together on the same Installation. The used versions are

python-carbon-0.9.12-3.el6.noarch
graphite-web-0.9.12-5.el6.noarch


Fix ECDSA algo names in LcobucciJWSProvider

Hi,
While trying to use ECDSA signature, I encountered The algorithm "ES256" is not supported by Lexik\Bundle\JWTAuthenticationBundle\Services\JWSProvider\LcobucciJWSProvider.
This name mismatch is pretty clear observing the GenerateKeyPairCommand command, which defines ES256, ES384 and ES512  in its ACCEPTED_ALGORITHMS constant.
This PR fixes this issue and allows usage of ESxxx algorithms.

Thank you @ControlAltBen.


Control path opacity

Hi, thanks for the useful tool!
I'm wondering if there's an handy way to control the opacity of a map region (so an SVG path).
I'm building an heatmap with an opacity scale based on data incoming from an API, so is not really possible to do that as in the Heatmap example.
Or rather, it's not possible without defining a CSS class for each possible opacity value, that's not something I really wanna do :)
Any suggestions?
Thanks a lot

Hi,
It's currently not possible (without defining a CSS class for each possible opacity value), but I think adding the feature to set the style attribute of each location (with a new locationStyle prop for example) would solve your problem.
I'll try to work on it in the following weeks, or feel free to create a pull request!


docs: changelog++ for v1.4.0

Codecov Report

Merging #690 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master    #690   +/-   ##
======================================
  Coverage    93.2%   93.2%
======================================
  Files          65      65
  Lines        5738    5738
======================================
  Hits         5348    5348
  Misses        233     233
  Partials      157     157


Fix typos in documentation

1 misspelling.
The Precedence examples kept using keypairs where they meant buckets.

Closing as the referenced code blocks were removed and this is no longer needed.


World multi goes insane after using a custom ign

Unsure what causes this, will resolve later

The issue seems to occur when the instructions
movss xmm0,worldMulti mulss xmm0,worldMulti addss xmm1,xmm0 movss [0207D794],xmm1
stop running

I have implemented a countermeasure which stops the game from completely locking up when this happens. The issue is not resolved, it is simply much less of an issue.

I think this still happens. The countermeasure is still in place.


WIP: Adds a basic TCP server - DO NOT MERGE

This is just a WIP to review as we work on it. @TimonPost @LucioFranco

@fhaynes I like this, but Im not sure about exposing two completely different API's to the end user. I want to take a look at it this weekend. Maybe come up with a trait that can express both the UDP and TCP api together.

Yup, I just haven't gotten that far yet. =) Gimme time.
On Fri, Sep 28, 2018 at 3:46 PM Lucio Franco<EMAIL_ADDRESS>wrote:

@fhaynes https://github.com/fhaynes I like this, but Im not sure about
exposing two completely different API's to the end user. I want to take a
look at it this weekend. Maybe come up with a trait that can express both
the UDP and TCP api together.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/LucioFranco/amethyst-protocol/pull/18#issuecomment-425586782,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEL2FbQA0FmyWf4jUh7c4Pq7VdwfSXGMks5ufqbEgaJpZM4W-8Z3
.

Yeah, once #16 is merged I'll rebase and handle the unwraps


Sorry! Something went wrong. This page didn't load Google Maps correctly

same issue with me as well.
Someone please help

copy the above script tag and paste it into the index.html file in the public folder. It will work

I'm having this same issue too. I'm guessing it something to do with billings

I'm having this same issue too. I'm guessing it something to do with billings

I don't think so, i just used the script tag mentioned above in index.html and it worked for me.

another solution is to reclone the repo , this worked for  ME  , if any problem can i PR your repo ? pwease

it worked for me.

nope..it is not working for me

@kiritocode1 yess please..

Line 6:1:  Expected indentation of 2 space characters but found 0  react/jsx-indent
i am getting this error when i copy pasted it in index.js

copy the above script tag and paste it into the index.html file in the public folder. It will work

go to either the console or network tab to see the exact error message you're receiving. Also check index.js to see if React being in strict mode is causing issues.

copy the above script tag and paste it into the index.html file in the public folder. It will work

Thank you! It worked for me


Fixed repository mode link

Closing as it is a duplicate of #741.

Closing as it is a duplicate of #741.


Operations on NamedTuples do not work with singleton types due to match type refusing to reduce

Compiler version
7d559ad06bdfe448450341c7fdf12cc0513456f3
Minimized code
import scala.language.experimental.namedTuples

object Test:
  val f: (name: String, age: Int) = ???
  val x: NamedTupleDecomposition.Names[f.type] = ("name", "age")

Output
5 |  val x: NamedTupleDecomposition.Names[f.type] = ("name", "age")
  |                                                 ^^^^^^^^^^^^^^^
  |Found:    (String, String)
  |Required: NamedTupleDecomposition.Names[(Test.f : (name : String, age : Int))]
  |
  |Note: a match type could not be fully reduced:
  |
  |  trying to reduce  NamedTupleDecomposition.Names[(Test.f : (name : String, age : Int))]
  |  failed since selector (Test.f : (name : String, age : Int))
  |  does not match  case NamedTuple.NamedTuple[n, _] => n
  |  and cannot be shown to be disjoint from it either.

Expectation
I'd expect this to reduce, otherwise all NamedTuple operations that use NamedTupleDecomposition like NamedTuple.Map will fail to reduce on singleton types. NamedTupleDecomposition is defined as:
/** Separate from NamedTuple object so that we can match on the opaque type NamedTuple. */
@experimental
object NamedTupleDecomposition:
  import NamedTuple.*

  /** The names of a named tuple, represented as a tuple of literal string values. */
  type Names[X <: AnyNamedTuple] <: Tuple = X match
    case NamedTuple[n, _] => n

So NamedTuple is treated as an abstract type constructor and we end up in https://github.com/scala/scala3/blob/7d559ad06bdfe448450341c7fdf12cc0513456f3/compiler/src/dotty/tools/dotc/core/TypeComparer.scala#L3506-L3511
Which fails since the scrutinee isn't widened. @sjrd : do you anticipate a soundness issue if we added a .widenSingleton on the scrutinee here? Or is the whole design of NamedTupleDecomposition doomed?

Yes, widening singleton types of paths that can be substituted later creates established unsoundness issues. See https://github.com/scala/scala3/issues/19746

Thanks for the reference. A possibly viable workaround is to use a wrapper type:
import scala.language.experimental.namedTuples

class Wrapper[T](x: T)
object Wrapper:
  type Extract[W <: Wrapper[?]] = W match
    case Wrapper[t] => t

object Test:
  val f: (name: String, age: Int) = ???
  val f2 = Wrapper(f)
  val x: NamedTupleDecomposition.Names[Wrapper.Extract[f2.type]] = ("name", "age")

Not sure if there's a better way or if that can help suggest some more systematic fix.

@smarter what if we explicitly create a Widen type operation? So Names will be implemented as Widen[X] match ...

That would reopen the soundness hole of https://github.com/scala/scala3/issues/19746 if Widen[this.type] can reduce to different types depending on the context.

I'm not sure that will happen, because of the lazy nature of singleton ops operations.

Though note that the fix for #19746 (https://github.com/scala/scala3/commit/0a3497bf7f0ac032f98d36cf1713e8926b5d176c) specifically target singleton types coming from parameters (since those can be substituted), so maybe something like a widenSingletonOfNonParamRef operation? It's not clear to me that this is enough if the underlying type of a non-param singleton is a GADT, in which case maybe the original hole isn't fixed.

in which case maybe the original hole isn't fixed.

Indeed: #20515.


video widget local file

Is there a way to get the video widget to play mp4 files located locally? I've tried every format for the url I could think of but the video isn't playing. I'm not too familiar with browser tech so I'm probably trying to break some rule.
Ultimately I'd like to be able to dump mp4 files into a folder locally and present a list of files in that folder to the user and when the user clicked on a video it would play in a video widget. Any help on getting me moving would be greatly appreciated.

Apparently, browsers won't let you play local files directly, unless they were defined by the user using a file selector: http://stackoverflow.com/questions/8885701/play-local-hard-drive-video-file-with-html5-video-tag
You can make Flexx serve the video file though:
from flexx import app, ui
url = app.assets.add_shared_data('myvideo.mp4', 'file://' + fname)
ui.VideoWidget(source=url)

A full example:
import os
from flexx import app, ui

fname = 'd:/videos/ice age 4 trailer.mp4'
assert os.path.isfile(fname)
url =app.assets.add_shared_data('myvideo.mp4', 'file://' + fname)
app.App(ui.VideoWidget, source=url).launch()
app.run()

The downside of this approach is that Flexx loads the complete video file in memory, which is ok from small videos, but don't try that with the full Lord of the Rings trilogy ;)
I can see this as a perfectly fine use-case though, so maybe Flexx should support serving files via the filesystem; as a proxy rather than reading it into memory.

Any "static file server" can provide you with URLs though. Try nginx!

True, and Tornado too, but would be nice if Flexx can make this a no-brainer.

Thank you for your responses, you mentioned that Tornado can do it, can this Flexx example of adding handlers be modified and added too the example above to include the static local content without fully loading it into the browser first?
I ask because tornado has a tornado.web.StaticFileHandler class, but I can't figure out exactly how it would fit into Flexx.

Oh nice, that turns out to be pretty easy!

import os
from flexx import app, ui

tornado_app = app.current_server().app
from tornado.web import StaticFileHandler

tornado_app.add_handlers(r".*", [
    (r"/videos/(.*)", StaticFileHandler, {"path": "d:/videos"}),
    ])

url = '/videos/ice age 4 trailer.mp4'
app.App(ui.VideoWidget, source=url).launch()
app.run()

Maybe we could add app.assets.add_data_dir(local_dir, url_path) to streamline the above.

Beautiful! Now the video shows up immediately, streaming in the normal fashion.

Additionally the streamlined pattern you provided would be welcome, I think it adds enormous value to Flexx.
Just a note, this is from the Tornado docs:
StaticFileHandler is intended primarily for use in development and light-duty file serving; for heavy traffic it will be more efficient to use a dedicated static file server (such as nginx or Apache).

It might be worth adding that to the docs or the docstring for figure add_data_dir method.

Getting this in #313, including a small example to list and view videos from your local machine and the web.

So eventually I decided against having a static file handler in Flexx by default for secutity reasons. There is a niew video_viewer.py example which demonstrates how to setup a static file server with tornado.


Add HardSourceWebpackPlugin for better build times

Easy 3x faster build times (even more on slow machines, I got 5x improvements at the default boilerplate and 16x in a large project of mine)
Some numbers here: https://github.com/mzgoddard/hard-source-webpack-plugin/issues/251
Note: cant be used for production builds right now cause of an issue with OfflinePlugin used for the Service Worker in this project

@williamoliveira Curious why you closed it? Didn't work out too well?


🐛 organizeImports causes check to fail but does not print errors with --diagnostic-level=error

Environment information
><EMAIL_ADDRESS>rage
> biome rage

CLI:
  Version:                      1.6.4
  Color support:                true

Platform:
  CPU Architecture:             x86_64
  OS:                           linux

Environment:
  BIOME_LOG_DIR:                unset
  NO_COLOR:                     unset
  TERM:                         "xterm-256color"
  JS_RUNTIME_VERSION:           "v20.11.1"
  JS_RUNTIME_NAME:              "node"
  NODE_PACKAGE_MANAGER:         "npm/10.2.4"

Biome Configuration:
  Status:                       Loaded successfully
  Formatter disabled:           false
  Linter disabled:              false
  Organize imports disabled:    false
  VCS disabled:                 true

Workspace:
  Open Documents:               0

What happened?
Repro is available here https://codesandbox.io/p/devbox/practical-snowflake-dh8pwq
Run npm i && npm run check in the terminal.
See that the check command failed but does not print the organizeImports errors
> biome check ./src --diagnostic-level=error

Checked 1 file in 1445µs. No fixes needed.
Found 1 error.
check ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  ✖ Some errors were emitted while running checks.

Expected result
Run npm run check-working which prints the error causing organizeImports output as green warnings.
> biome check ./src

./src/index.js organizeImports ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  ℹ Import statements could be sorted:

    1   │ - import·{·graphql,·useFragment,·useMutation·}·from·"react-relay";
    2   │ - import·{·FC,·memo,·useCallback·}·from·"react";
      1 │ + import·{·FC,·memo,·useCallback·}·from·"react";
      2 │ + import·{·graphql,·useFragment,·useMutation·}·from·"react-relay";
    3 3 │
    4 4 │   console.log("Hello CodeSandbox");

Checked 1 file in 1960µs. No fixes needed.
Found 1 error.
check ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  ✖ Some errors were emitted while running checks.

Code of Conduct

[X] I agree to follow Biome's Code of Conduct

Thanks for the very fast fix 🥇


Added Support For .Net Framework 2.0

Added Support For .Net Framework 2.0 because lets face it, its still being used.
Todo:
-Convert to vb.net
Why Do i need vb.net while i can just add it as a reference?
well some developers like me do not like using other dll files to avoid file not found or using it as embedded reference and load it in runtime.

2.0 is no longer supported by Microsoft.  3.5 SP1 is though.
As for VB.NET support, I don't know what you're saying is needed and can't see in this PR.


Doesn`t work in general

Hi,
I installed this via HACS with the goal to be able to analyze the energy-consumption of our new Panasonic Aquarea Heatpump. I logged in to Panasonic CC, but get only errors in Homeassistant. Is this still working and maintained?

Settings of Energy Dashboard:

Regards
Martin

Same Issue hear. I've Panasonic Aquera T-Cap Heatpump and after integration setup, all device controls and sensor are wrong and in energy configuration same issue like on @Trike27 screens

The "Nanoe" is an extra unit that also has been added for me by the nitergration, while I know for a fact I don't have this option buildin into my AC unit (CS-TZ25WKEW).
I believe this is an air cleaning add-on. There might be a bug that "adds" this feature by default
Also (At least in the latest updated version for HA en the integration and such) the power feature does seem to work for me

What is the issue here? You would like to use the Energy sensors on the Energy dashboard? Is this still happening for you?

This issue has been inactive for a while, so we’re closing it for now. If you believe this issue is still relevant, please feel free to reopen it or create a new one.


Change metadata while upload is in progress

I would like to create an interface similar to the Vimeo/Youtube upload process.
While a large file is being uploaded, allow the user to fill in related metadata.
Is it possible to change metadata after an upload has started?

Ok, thanks @Acconut


support azure keyvault provider

hey peeps, we are thinking about cutting a v0.1 release soon!
any chances of including this work as well? 🤔

hey @ahmedmus219, could you rebase your branch from main?

Hello, I'm a newcomer in @ahmedmus219's team, and I will work with him on the project. It's my first open source contribution and I'm discovering go, so it might not be perfect.
I did a rebase as requested and mainly reworked the unit tests.

Welcome @ncourbet-1A ! Awesome! I just enabled CI for you, so you can have feedback here. Feel free to ask any questions if you need :)
Just out of curiosity, you mentioned that you are joining @ahmedmus219 team and will be working on the project, what do you mean by that? Is your company letting you folks contribute on work time? Would love to hear about it, and your use case for External Secrets Operator :)

mea culpa, I ran a make fmt && make lint but did not realize that the make fmt updated the file after my commit... And we bypassed the PR checks on the 1a-mj fork due to the merge conflict after rebase.
This will be fixed once @ahmedmus219 merge https://github.com/1A-mj/external-secrets/pull/2 (I do not have write access myself)
And yes, our company allowed us to work on this project, as we are migrating to Azure but still want some capability to run on AWS or GCP. So this external-secret allow us to keep the same architecture.
If you want more details, we can move this thread elsewhere, as it is not directly to the pull request.

@knelasevero indeed, @ncourbet-1A , @ahmedmus21 & myself are from Amadeus & we have been internally allowed to contribute code to this project to have a better K8s-Azure integration, following our company's public partnership with Microsoft. As discussed with @riccardomc at KubeCon EU 2019, the ExternalSecret approach is way lighter than the CSI-based one (no DaemonSet, no node-level permissions, use of old & small & stable Secret K8s objects) and it relies on the otherway-enforced etcd encryption to protect Secrets (eg. using a KmsProvider).

That's awesome! Thanks for sharing.
I will try to test this out this weekend. If everything is ok, I think we can merge, if you are all happy with it.
Would be super cool if we could get some help with implementing some e2e tests for azure in the furute. We already got com infra donated for that, so we just need to write the tests :smile:

@ncourbet-1A / @ahmedmus21 seems like the is some conflict to catch-up with.

After a rebase, is this ready? I plan to test it again :)

I think the only thing left is some docs at docs/provider-azure-key-vault.md. We can merge this one and add docs on a later PR if you want

I think the only thing left is some docs at docs/provider-azure-key-vault.md. We can merge this one and add docs on a later PR if you want

Thanks @knelasevero , yes we can add some more docs (and snippets) in a later PR

/approve

/merge


Support for 3.x syntax

I was wondering if it'd be possible to support the 3.x syntax. For example, _append instead of :append.

Here is the documentation for 3.1 syntax.
Here is the vimscript implementation of the syntax, which has been unchanged for quite a long time and seems to work nicely.
I am not sure if the syntaxes would play nicely together, or if there is some sort of configuration that can be applied to load one versus the other - I am not familiar enough with treesitter.

hmm, I was hoping for a slightly simpler/cleaner page with examples I can easily peruse - that link is lengthy
I see that what I referenced - https://docs.yoctoproject.org/bitbake/2.4/index.html - is for BitBake 2.4 I assume, but I couldn't seem to navigate to 3.1's version on this site, would be nice if there was something similar

I tried to link the appropriate heading, but it appears some browsers or the site itself doesn't respect the anchor declaration in the url. There should also be an "individual web page" version. I just use the mega manual since I find it easier to search through.

Note that the 3.1 link above refers to the Yocto Project Release 3.1, which is actually bitbake 1.46.  The bitbake 2.4 link above refers to bitbake 2.4, which was part of the Yocto Project 4.2 release.
The overload operator was changed from _ to : somewhere around there because _ was also used as a word separator in variable names.
So the requested syntax is the older syntax, though still valid in some of the LTS branches.
The releases are listed here https://wiki.yoctoproject.org/wiki/Releases


UISearchBar Doesn't Show with UINavigationController + Inject.ViewControllerHost

class ViewController: UIViewController, UISearchResultsUpdating {

    let searchController: UISearchController

    init() {
        self.searchController = UISearchController(searchResultsController: nil)

        super.init(nibName: nil, bundle: nil)

        searchController.searchResultsUpdater = self
        searchController.obscuresBackgroundDuringPresentation = false
        searchController.searchBar.placeholder = NSLocalizedString("Search", comment: "")
        searchController.searchBar.searchBarStyle = .minimal

        navigationItem.searchController = searchController
        navigationItem.title = "Hello"
        navigationItem.hidesSearchBarWhenScrolling = false

        self.definesPresentationContext = true
    }

    required init?(coder: NSCoder) {
        fatalError("init(coder:) has not been implemented")
    }

    func updateSearchResults(for searchController: UISearchController) {

    }

}

// elsewhere

let injectVC = Inject.ViewControllerHost(ViewController())
let root = UINavigationController(rootViewController: injectVC)
window.rootViewController = root

self.window = window
window.makeKeyAndVisible()

Doing something like the following results in the search bar never showing with iOS 16 (I haven't tried iOS 15 yet). Removing the Inject.ViewControllerHost and just using ViewController directly works just fine, so I'm not sure if  ViewControllerHost is messing with UIKit's ability to detect where to put the search bar or forwarding or something ¯_(ツ)_/¯
Any idea for a workaround? For now I've just decided to not use Inject. Tried it with and without Injection app running
Sorry if you aren't working on iOS 16 support yet, disregard for now if so!

hey @b3ll, the way this works at host level is that it wraps the real VC as a child view controller and acts as a proxy so that the injected VC version can be swapped out.
That being said I think we'd just need to forward those properties you are setting in here, would you mind trying that?


LEGO: Pull request from lego/hb_a12a4630-4852-4e7d-9cbc-c0e1117da1ab_20210412204204074 to main

LEGO: Pull request from lego/hb_a12a4630-4852-4e7d-9cbc-c0e1117da1ab_20210412204204074 to main with localized lcls
This is part of the localisation onboarding, and needs to be merged.

This needs to be merged for the localisation onboarding. Will have followup discussion on teams.


[Feature] Dockerfile for terminusdb-python

The Dockerfile has an accompanying README.md file to show how to use it.
A provisional image of it is available at hoijnet/terminusdb. Should you accept the PR and an official image would be created, it would be awesome and make the terminusdb client more available to be used even without installing python directly.
The client can be started without arguments, launching python3 where the client can be loaded, or with the regular arguments to load the schema and such.
Hope it's helpful :-)

Hello again, @hoijnet!
We're thinking this would be better as a separate project (e.g. another GitHub repository). Would you want to create something like that to share with others? We'd be happy to direct others to that repository in the README. What do you think?

The maintenance burden of keeping the image up to date is something I need to consider actively. The worst is stale projects just laying around. I'll close this for now and I'll check in later with you if I use it enough that it makes sense to maintain more broadly 🎯

Thanks! Maintenance is indeed a concern with us, too, and I don't think anybody on the team would use it frequently. At the very least, it's nice to have a PR like this, so people can find it, take it, and adapt it if they need it. 😁


Testing Dnd with Cypress

Hi everyone,
Someone has tried testing the feature "drag and drop" with Cypress?.
I cannot move an event. I have tried several options but not success.
For example:
cy.get('.rbc-addons-dnd-resizable > .rbc-event-content')       .trigger('mousedown', { which: 1 })       .trigger('mousemove', { clientX: 200, clientY: 300 })       .trigger('mouseup', { force: true });
Thanks in advance.
Greetings

Any luck?

@rockandbit I am testing it using something like this:
  cy.get('.rbc-event').trigger('dragstart');   cy.get('[find the timeslot where you want to drop something like .rbc-time-slot]').trigger('drop', {           force: true,         });
So for your specific case you need to adjust the selectors but I think you will get the point how to test using dragstart and drop events.

I can't create an event with DnD. I'm only able to create an event by triggering a 'click' event
what I tried:

https://youtu.be/hvwYBlKLz6U?t=232
https://github.com/4teamwork/cypress-drag-drop
.trigger('dragstart') and .trigger('drop', { force: true, })

found a solution, pretty obvious, the trick was: use 'mousemove' on the target selector instead of the source selector
here a simple example: a 3hs event
cy.get('.rbc-day-slot .rbc-time-slot').first().trigger('mousedown', { force: true });
cy.get('.rbc-day-slot .rbc-timeslot-group:nth-child(3) .rbc-time-slot:nth-child(2)')
  .last()
  .trigger('mousemove', { force: true })
  .trigger('mouseup', { force: true });

.rbc-timeslot-group is the hour block and .rbc-time-slot:nth-child the half hour block inside that hour block


lbrycrdd should have a logroll if its printing console output to logs.

Since it prints every line about every bit of indexing that it does it should be able to roll the log file a bit so that you don't end up with a full blockchain sized log file.

I think we should also audit what is currently being written to debug.log.

I removed the biggest culprit here, this would dump about 200mb worth of stuff on every claimtrie flush:
https://github.com/lbryio/lbrycrd/commit/7eca4f00a7a6d24d8386dab733e136940ef457f2
Did an audit of the other logs, they don't cause too much of a problem in terms of the volume of logs.
These definitely should be removed, as they serve no useful purpose other than debugging for devs:
https://github.com/lbryio/lbrycrd/blob/master/src/claimtrie.cpp#L608
https://github.com/lbryio/lbrycrd/blob/master/src/claimtrie.cpp#L614
https://github.com/lbryio/lbrycrd/blob/master/src/claimtrie.cpp#L1598
https://github.com/lbryio/lbrycrd/blob/master/src/claimtrie.cpp#L1629
https://github.com/lbryio/lbrycrd/blob/master/src/claimtrie.cpp#L1945
https://github.com/lbryio/lbrycrd/blob/master/src/claimtrie.cpp#L2111
https://github.com/lbryio/lbrycrd/blob/master/src/claimtrie.cpp#L2384
Below could be removed they do show some useful claim information (for example, a new claim gets created), but not sure if that's something that should go into the logs.
https://github.com/lbryio/lbrycrd/blob/master/src/claimtrie.cpp#L58
https://github.com/lbryio/lbrycrd/blob/master/src/claimtrie.cpp#L65
https://github.com/lbryio/lbrycrd/blob/master/src/claimtrie.cpp#L1592
https://github.com/lbryio/lbrycrd/blob/master/src/claimtrie.cpp#L1611
https://github.com/lbryio/lbrycrd/blob/master/src/claimtrie.cpp#L1697
https://github.com/lbryio/lbrycrd/blob/master/src/claimtrie.cpp#L2001
https://github.com/lbryio/lbrycrd/blob/master/src/claimtrie.cpp#L2020
https://github.com/lbryio/lbrycrd/blob/master/src/claimtrie.cpp#L2098
https://github.com/lbryio/lbrycrd/blob/master/src/claimtrie.cpp#L2105
https://github.com/lbryio/lbrycrd/blob/master/src/claimtrie.cpp#L2163
https://github.com/lbryio/lbrycrd/blob/master/src/claimtrie.cpp#L2215

Your assessment looks good. I think we should also remove those 7 you suggest. I think we should keep a log entry for each user addition/subtraction, so I think most on the bottom list should stay. It looks like we may be logging added claims twice, though -- both add and insert?

I'm worried of this line too -> https://github.com/lbryio/lbrycrd/pull/159/commits/80940714a1015dd8dcc22de8d53ae99bff65328f#diff-1f647c4b5527c3d3e86cfb735380b479R60

@bvbfan , I do have that line removed in my normalization work.

The fixes in this release https://github.com/lbryio/lbrycrd/releases/tag/v<IP_ADDRESS> will remove much of the unnecessary logging. lbrycrdd has a function that will automatically shrink large log files upon startup, so we will not implement a log roll/rotation.
Users that want a log roll/rotation should implement their own (the automatic shrinking of log files can be disabled through option -shrinkdebugfile )

I propose that we start a new file with each run, and that we backup the old file. Hence, I should always have this run and the previous one available. That's not quite a "roll" of logs, in that it won't fix the maximum size while the daemon is running, but I think it would help.

@BrannonKing can I work on this to implement what you described in your last comment?

@gahag , yes you can work on this.

I've explored the code, and I'm thinking of implementing in BCLog::Logger::OpenDebugLog. The code would move the debug file if it exists to a .old file. This way, there would be one file for each execution of the entire application. Another option would be to implement such code in the default constructor for BCLog::Logger. What do you guys think?

Look at the top of AppInitMain(); the "shrink" code there would also need to change.

Sure. What are we expecting to be the new behavior of the -shrinkdebugfile flag? Should it be removed, or should it shrink the old log file?

I think the flag should be removed.

This looks like it should've been closed by #344 I don't have time to really validate this functionality but if people see this PR didn't hit the notes that this thread evolved to hit then please feel free to re-open it.


Changes to support STAC on multiple Azure clouds - Part 1 of 2

These changes have been successfully tested against the commercial cloud as well.

Thanks for looking into supporting the solution for multiple Azure clouds

We will have to update https://github.com/Azure/Azure-Orbital-STAC/blob/main/docs/deploying-infra.md to document steps for switching to a different cloud az cloud set --name
Also trivial but this PR is to support (and is supporting with this change) multiple Azure clouds and not just usgov - so we may want to rename accordingly

The document has been updated to cover the instructions for multiple Azure clouds. Thanks @mandarinamdar


Update development dependencies

With this PR, pip install '.[dev]' should install all the dependencies needed for development. I dropped black because pre-commit deals with that, gitpython because I don't think it's needed (?), and coverage because it's included in pytest-cov.

Fab, I've put those two back.


WIP: Work on getting CI targets working with the build-tools container

I can successfully run make docker test-cni from macos with the following patch:
diff --git a/Makefile b/Makefile
index 94e9d57e..93a604ad 100644
--- a/Makefile
+++ b/Makefile
@@ -29,7 +29,7 @@
 export BUILD_WITH_CONTAINER ?= 0

 ifeq ($(BUILD_WITH_CONTAINER),1)
-IMG = gcr.io/istio-testing/build-tools:2019-08-28T11-32-19
+IMG = gcr.io/istio-testing/build-tools:2019-08-29T13-57-48
 UID = $(shell id -u)
 PWD = $(shell pwd)
 GOBIN_SOURCE ?= $(GOPATH)/bin
diff --git a/Makefile.core.mk b/Makefile.core.mk
index f8715e9e..dcce87a2 100644
--- a/Makefile.core.mk
+++ b/Makefile.core.mk
@@ -264,9 +264,9 @@ else
 endif
 test-cni: | $(JUNIT_REPORT)
        mkdir -p $(dir $(JUNIT_UNIT_TEST_XML))
-       set -o pipefail; \
+       set -o pipefail; unset GOOS && unset GOARCH && CGO_ENABLED=1 && \
        $(MAKE) -f Makefile.core.mk --keep-going $(TEST_OBJ) \
-       2>&1 | tee >($(JUNIT_REPORT) > $(JUNIT_UNIT_TEST_XML))
+               2>&1 | tee >($(JUNIT_REPORT) > $(JUNIT_UNIT_TEST_XML))

 .PHONY: install-test
 # May want to make this depend on push but it will always push at the moment:  install-test: docker.push
diff --git a/tools/istio-cni-docker.mk b/tools/istio-cni-docker.mk
index f3e0bf09..b9b5d414 100644
--- a/tools/istio-cni-docker.mk
+++ b/tools/istio-cni-docker.mk
@@ -43,8 +43,9 @@ docker.install-cni: $(ISTIO_OUT)/istio-cni tools/packaging/common/istio-iptables
                deployments/kubernetes/install/scripts/filter.jq
        mkdir -p $(ISTIO_DOCKER)/install-cni
        cp $^ $(ISTIO_DOCKER)/install-cni
-       time (cd $(ISTIO_DOCKER)/install-cni && \
-               docker build -t $(HUB)/install-cni:$(TAG) -f Dockerfile.install-cni .)
+       time docker build -t $(HUB)/install-cni:$(TAG) \
+               -f $(ISTIO_DOCKER)/install-cni/Dockerfile.install-cni \
+               $(ISTIO_DOCKER)/install-cni

 DOCKER_TARGETS:=docker.install-cni

Trying make prow-e2e now.

Hmmm actually the tests still fail in make test-cni.

I almost got it to work.
Now I'm blocked by the fact that the install_cni.go test is itself running Docker containers, into which it mounts 3 temporary subdirectories from its own /tmp, which of course don't exist in Docker's host (they exist only in the container running install_cni.go), so those mounts are empty.
I'll now try mounting /tmp' from the host into the test container's /tmp`...

OK, this patch makes make docker and make test-cni run for real:
diff --git a/Makefile b/Makefile
index 94e9d57e..7280406f 100644
--- a/Makefile
+++ b/Makefile
@@ -29,7 +29,7 @@
 export BUILD_WITH_CONTAINER ?= 0

 ifeq ($(BUILD_WITH_CONTAINER),1)
-IMG = gcr.io/istio-testing/build-tools:2019-08-28T11-32-19
+IMG = gcr.io/istio-testing/build-tools:2019-08-29T13-57-48
 UID = $(shell id -u)
 PWD = $(shell pwd)
 GOBIN_SOURCE ?= $(GOPATH)/bin
@@ -67,11 +67,13 @@ RUN = docker run -t -i --sig-proxy=true -u $(UID) --rm \
        -v /etc/passwd:/etc/passwd:ro \
        -v $(readlink /etc/localtime):/etc/localtime:ro \
        -v /var/run/docker.sock:/var/run/docker.sock \
+       -v /tmp:/tmp \
        $(CONTAINER_OPTIONS) \
        --mount type=bind,source="$(PWD)",destination="/work" \
        --mount type=volume,source=istio-go-mod,destination="/go/pkg/mod" \
        --mount type=volume,source=istio-go-cache,destination="/gocache" \
        --mount type=bind,source="$(GOBIN_SOURCE)",destination="/go/out/bin" \
+       --mount type=tmpfs,destination=/run/secrets/kubernetes.io/serviceaccount \
        -w /work $(IMG)
 else
 export GOBIN ?= ./out/bin
diff --git a/Makefile.core.mk b/Makefile.core.mk
index f8715e9e..77c28133 100644
--- a/Makefile.core.mk
+++ b/Makefile.core.mk
@@ -262,9 +262,12 @@ ifeq ($(WHAT),)
 else
        TEST_OBJ = selected-pkg-test
 endif
-test-cni: | $(JUNIT_REPORT)
+/var/run/secrets/kubernetes.io/serviceaccount/token:
+       echo "dummy-token" > /var/run/secrets/kubernetes.io/serviceaccount/token
+
+test-cni: /var/run/secrets/kubernetes.io/serviceaccount/token | $(JUNIT_REPORT)
        mkdir -p $(dir $(JUNIT_UNIT_TEST_XML))
-       set -o pipefail; \
+       set -o pipefail; unset GOOS && unset GOARCH && CGO_ENABLED=1 && \
        $(MAKE) -f Makefile.core.mk --keep-going $(TEST_OBJ) \
        2>&1 | tee >($(JUNIT_REPORT) > $(JUNIT_UNIT_TEST_XML))

diff --git a/tools/istio-cni-docker.mk b/tools/istio-cni-docker.mk
index f3e0bf09..b9b5d414 100644
--- a/tools/istio-cni-docker.mk
+++ b/tools/istio-cni-docker.mk
@@ -43,8 +43,9 @@ docker.install-cni: $(ISTIO_OUT)/istio-cni tools/packaging/common/istio-iptables
                deployments/kubernetes/install/scripts/filter.jq
        mkdir -p $(ISTIO_DOCKER)/install-cni
        cp $^ $(ISTIO_DOCKER)/install-cni
-       time (cd $(ISTIO_DOCKER)/install-cni && \
-               docker build -t $(HUB)/install-cni:$(TAG) -f Dockerfile.install-cni .)
+       time docker build -t $(HUB)/install-cni:$(TAG) \
+               -f $(ISTIO_DOCKER)/install-cni/Dockerfile.install-cni \
+               $(ISTIO_DOCKER)/install-cni

 DOCKER_TARGETS:=docker.install-cni

And make prow-e2e cannot run because the build image is missing kind.

And make prow-e2e cannot run because the build image is missing kind.

Could we use the same image as prow presubmits to get KinD (and build)?  (admittedly, I'm way out-of-date from when I really understood the details of istio/test-infra/prow)
https://github.com/istio/test-infra/blob/master/prow/cluster/jobs/istio/cni/istio.cni.master.yaml#L19

@rlenglet Wow, thanks Romain for making such progress.
I'll update my PR with your changes, and I'll get Kind in the build-tools image so we can go to the next steps.

@tiswanso We're trying to converge all repos to be building with the same build-tools container image. We've been gradually expanding what is in the container as we bring up the different repos. So hitting a wall where a tool is needed in the container is normal. I'll go add what is needed in the container shortly and the iterative loop to get this repo converted to the container can resume...

@rlenglet Wow, thanks Romain for making such progress.

+1 ... great progress

Note that the tests will write some (small) temp files into the host's /tmp, and that can't be fixed without being more invasive, e.g. mouting tmpfs instead of /tmp in the nested containers.

Note that the tests will write some (small) temp files into the host's /tmp, and that can't be fixed without being more invasive, e.g. mouting tmpfs instead of /tmp in the nested containers.

Right... that's existing behavior so I don't think it needs to be changed in this PR.  IIRC, we originally did that so we could debug failures.

@rlenglet I've updated my PR with the latest common files, which brings in the latest build-tools image that includes KinD. I tried applying your patch, but git complains that it is corrupt. So could you try again, given this updated PR please?

@geeknoid: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

e2e_cni
946b2325e5774e2639ddff61d0cc2bd352be4796
link
/test e2e_cni

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@geeknoid: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

lintmodern_cni
9a84d8a2fc7e3262029a60975d1bc3c74791d016
link
/test lintmodern_cni

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@geeknoid: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

lintmodern_cni
9a84d8a2fc7e3262029a60975d1bc3c74791d016
link
/test lintmodern_cni

buildmodern_cni
9a84d8a2fc7e3262029a60975d1bc3c74791d016
link
/test buildmodern_cni

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@geeknoid: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

buildmodern_cni
9a84d8a2fc7e3262029a60975d1bc3c74791d016
link
/test buildmodern_cni

lintmodern_cni
61ad591e196669ab1325185d5fcdb49c722362e8
link
/test lintmodern

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@geeknoid: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

lintmodern_cni
61ad591e196669ab1325185d5fcdb49c722362e8
link
/test lintmodern

buildmodern_cni
61ad591e196669ab1325185d5fcdb49c722362e8
link
/test buildmodern

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@geeknoid: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

lintmodern_cni
61ad591e196669ab1325185d5fcdb49c722362e8
link
/test lintmodern

buildmodern_cni
5810e093ab82ce24248f36a8113b3a9a6b1ad935
link
/test buildmodern

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@geeknoid: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

lintmodern_cni
61ad591e196669ab1325185d5fcdb49c722362e8
link
/test lintmodern

buildmodern_cni
5810e093ab82ce24248f36a8113b3a9a6b1ad935
link
/test buildmodern

build_cni
5810e093ab82ce24248f36a8113b3a9a6b1ad935
link
/test build

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@geeknoid: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

lintmodern_cni
61ad591e196669ab1325185d5fcdb49c722362e8
link
/test lintmodern

buildmodern_cni
5810e093ab82ce24248f36a8113b3a9a6b1ad935
link
/test buildmodern

build_cni
5810e093ab82ce24248f36a8113b3a9a6b1ad935
link
/test build

install_cni
5810e093ab82ce24248f36a8113b3a9a6b1ad935
link
/test install

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@geeknoid: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

buildmodern_cni
5810e093ab82ce24248f36a8113b3a9a6b1ad935
link
/test buildmodern

build_cni
5810e093ab82ce24248f36a8113b3a9a6b1ad935
link
/test build

install_cni
5810e093ab82ce24248f36a8113b3a9a6b1ad935
link
/test install

e2e_cni
5810e093ab82ce24248f36a8113b3a9a6b1ad935
link
/test e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@geeknoid: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

build_cni
5810e093ab82ce24248f36a8113b3a9a6b1ad935
link
/test build

install_cni
5810e093ab82ce24248f36a8113b3a9a6b1ad935
link
/test install

e2e_cni
5810e093ab82ce24248f36a8113b3a9a6b1ad935
link
/test e2e

buildmodern_cni
5a631ba02278e129a699c143a274c125c763cefa
link
/test buildmodern

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@geeknoid: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

buildmodern_cni
5a631ba02278e129a699c143a274c125c763cefa
link
/test buildmodern

e2e_cni
5a631ba02278e129a699c143a274c125c763cefa
link
/test e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

from the check:
+ pushd ../istio
/home/prow/go/src/istio.io/istio /home/prow/go/src/istio.io/cni
+ make istioctl
STATIC=0 GOOS=linux GOARCH=amd64 LDFLAGS='-extldflags -static -s -w' bin/gobuild.sh /home/prow/go/out/linux_amd64/release/istioctl ./istioctl/cmd/istioctl
flag provided but not defined: -trimpath
usage: go build [-o output] [-i] [build flags] [packages]
Run 'go help build' for details.
Makefile.core.mk:393: recipe for target 'istioctl' failed

Stevens-MacBook-Pro:pilot sdake$ go build -trimpath
flag provided but not defined: -trimpath
usage: go build [-o output] [-i] [build flags] [packages]
Stevens-MacBook-Pro:pilot sdake$ go version
go version go1.12.7 darwin/amd64

The internet says -trimpath was added in golang 1.13.

From buildmodern check:
time docker build -t istio/install-cni:fc050a1a908b229997ed303fa5de5187c6e549c4 \
	-f /home/prow/go/out/linux_amd64/release/docker_temp/install-cni/Dockerfile.install-cni \
	/home/prow/go/out/linux_amd64/release/docker_temp/install-cni
time="2019-09-19T18:46:00Z" level=error msg="failed to dial gRPC: cannot connect to the Docker daemon. Is 'docker daemon' running on this host?: dial unix /var/run/docker.sock: connect: no such file or directory"
context canceled```
this implies Kind does not expose /var/run/docker.sock of the host K8s system.  This is not unexpected.  Using /var/run/docker.sock iin K8s from docker is dangerous.  Especially so with K8s in docker in docker.

Looking at the buildtools container, it clearly indicates 1.13.0 of golang is in use.

/test e2e

@geeknoid: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

buildmodern_cni
5a631ba02278e129a699c143a274c125c763cefa
link
/test buildmodern

e2e_cni
5a631ba02278e129a699c143a274c125c763cefa
link
/test e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

go version is 1.12.9 because istio-builder is being used.
/home/prow/go/src/istio.io/istio /home/prow/go/src/istio.io/cni
+ make istioctl
STATIC=0 GOOS=linux GOARCH=amd64 LDFLAGS='-extldflags -static -s -w' bin/gobuild.sh /home/prow/go/out/linux_amd64/release/istioctl ./istioctl/cmd/istioctl
go version go1.12.9 linux/amd64

See PR: https://github.com/istio/test-infra/pull/1829 which updates istio-builder.
Note istio-builder is the wrong container entirely for this purpose - the goal is to be using buildtools for a consistent toolchain.

/test buildmodern

/test e2e

@geeknoid: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

e2e_cni
5a631ba02278e129a699c143a274c125c763cefa
link
/test e2e

buildmodern_cni
5a631ba02278e129a699c143a274c125c763cefa
link
/test buildmodern

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@tiswanso I assume the circleci job is desireable because ipv6 doesn't work in kind?
Could you verify?  If the job is redundant, we are trying to limit our use of circleci and I'll remove it from the list of jobs.  PLMK.
Cheers
-steve

buildmodern_cni is using the build-tools container (which contains no entrypoint to launch docker: https://github.com/istio/test-infra/blob/master/prow/cluster/jobs/istio/cni/istio.cni.release-1.3.gen.yaml#L279-L288

e2e (which was previously failing in this PR) is using istiobulider with an entrypoint to start docker: https://github.com/istio/test-infra/blob/master/prow/cluster/jobs/istio/cni/istio.cni.release-1.3.gen.yaml#L235-L244
The conclusion is we need to subsume the istiobuilder docker entrypoint work into build-tools.  THis may take multiple revisions to test-infra and tools/docker/build-tools to obtain a working check that functions well on prow.

I was unable to get ipv6 working on kind, did to the underlying prow nodes not supporting ipv6. I agree you need the entry point but it will require changes to the docker files since we blow away some of the files required to service docker start

@howardjohn I understand.  I will be taking on making docker run within the build-tools container today and hopefully finish.  At latest, my target is this week.  As for the dind-ipv6 job, it was already failing prior to this pr.  See #180, #181, or #182.
Cheers
-steve

Sounds great, thanks!!
On Wed, Sep 25, 2019 at 8:03 AM Steven Dake<EMAIL_ADDRESS>wrote:

@howardjohn https://github.com/howardjohn I understand. I will be
taking on making docker run within the build-tools container today and
hopefully finish. At latest, my target is this week. As for the dind-ipv6
job, it was already failing prior to this pr. See #180
https://github.com/istio/cni/pull/180, #181
https://github.com/istio/cni/pull/181, or #182
https://github.com/istio/cni/pull/182.
Cheers
-steve
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/istio/cni/pull/167?email_source=notifications&email_token=AAEYGXOFA7PRYWS2SS4ERWDQLN4UTA5CNFSM4ISGLKY2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7SHOYY#issuecomment-535066467,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEYGXJ4PBOQQG2OJFBKQGDQLN4UTANCNFSM4ISGLKYQ
.

@geeknoid: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

buildmodern_cni
4b76dbbbff7ee9b6838d1504122b8078f31c55ee
link
/test buildmodern

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


Move out observable logic

Simple custom event is enough to implement observable in userland. Basic Observable example can simply:
fromEvent(el, 'propchange').map(props => props)

Two points though.

Event observable doesn't emit initial value
Setting up attribute observer lazily in subscribe is cheaper and rather ad-hoc
Due to this, keeping observable in-place.

It makes sense though reduce duplication by delegating observable part to 0b

element-props/observable

Needs merging. Now repeats code too much ugh


Javascript interop documentation

Hi, where can I find that ? To use JS libraries for components (data table tools like tabulator.info, datatables.net or whatever)
Thanks

@guardrex

According to our current plan, JS/TS interop documentation will appear soon after the release of the preview docs (we don't have an issue yet; the issue will appear in a couple of weeks). In the meantime, I'll drop the section in here that we have in the draft introduction topic, which was derived from the @SteveSandersonMS blog post.

For apps that require third-party JavaScript libraries and browser APIs, WebAssembly is designed to interoperate with JavaScript. Blazor is capable of using any library or API that JavaScript is able to use.

To call JavaScript libraries or your own custom JavaScript/TypeScript code from .NET, the current approach is to register a named function in a JavaScript/TypeScript file:

Blazor.registerFunction('doPrompt', message => {
  return prompt(message);
});

Wrap the named function for calls from .NET:

public static bool DoPrompt(string message)
{
    return RegisteredFunction.Invoke<bool>("doPrompt", message);
}

We now have a chapter on JS interop with an example of interacting with a UI library (Kendo UI) online: https://learn-blazor.com/architecture/interop/ Hope it helps getting started until Microsoft has more in-depth documentation available.

Thanks for the answers, @rstropek and @guardrex!

How does use a large third party API with a large number of methods? For example GoogleMap JS API?
Would there be better interop?
Or one should also rely on community made wrapper?

@superlloyd The thinking is that we will have tooling that can generate the JS interop .NET wrapper for you from the library's typescript definitions. The community can then ship these JS interop APIs as NuGet packages. See https://github.com/mono/TsToCSharp for additional details.

Sounds like great thinking, awesome, thanks! :)


Optimize DefaultChannelId.equals

Motivation:
A DefaultChannelId has final hashCode field calculated in the constructor. We can use it in equals to the fast return for different objects.
Modifications:
Use hashCode field in DefaultChannelId.equals().
Result:
Fast equals on negative scenarios.

@fenik17 can you split this into two prs as these are different issues ?

@normanmaurer sure. and one #7193

Cherry-picked into 4.1 (e40469057483c6b3ce9cf69af12dd19a68392cfd). @fenik17 thanks!


Errors are cached for a long period

Hello,
First of all I want to say that I really like your service. 3 days ago we started to use it for serving Gravatar images and it made it faster and more reliable.
But here is the problem. When Gravatar responses with 503 Service Unavailable then this service will respond with 404 Not Found -  The requested URL returned error: 503 message and it serves the same 404 for 3 days now in some cases.
If a new parameter is added, accessed from a different device is used or from a different zone with VPN the image served "normally" so it might be a Cloudflare configuration issue I am not sure at the current state.
Probably not helpful at all but here is an example:
https://www.gravatar.com/avatar/82a206ae2834d1f3ce0affb688ab83d1?s=100&d=wavatar
https://images.weserv.nl/?url=www.gravatar.com%2Favatar%2F82a206ae2834d1f3ce0affb688ab83d1%3Fs%3D100%26d%3Dwavatar
{"status":"error","code":404,"message":"The requested URL returned error: 503"}

I'm aware that there is some caching on 404's, but only for a short amount of time (eg. 15 minutes). It could be that the browser caches this response longer, or it may be that Cloudflare is holding longer on the cached response than anticipated. Will look into this, to check if there is some official statement on this behavior. In the meantime you could play with the &maxage= setting: https://images.weserv.nl/docs/format.html#cache-control

When I look at the response for https://images.weserv.nl/?url=weserv.nl/doesnotexist.jpg
I don't see any clear indication that it should be cached by anyone in the headers (no Expires:, no Cache-Control:). But I do see that the request is being cached by Cloudflare (the Age: header):
    HTTP/1.1 404 Not Found
    Date: Mon, 23 Dec 2019 17:42:28 GMT
    Content-Type: application/json
    Transfer-Encoding: chunked
    Connection: keep-alive
    Set-Cookie: __cfduid=dd9739330ad24b3c947766187bd21c6b81577122948;
        expires=Wed, 22-Jan-20 17:42:28 GMT; path=/;
        domain=.weserv.nl; HttpOnly; SameSite=Lax
    Vary: Accept-Encoding
    Access-Control-Allow-Origin: *
    X-Images-Api: 5
    CF-Cache-Status: HIT
    Age: 370
    Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
    Expect-CT: max-age=604800, report-uri="https://report-
        uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
    Alt-Svc: h3-24=":443"; ma=86400, h3-23=":443"; ma=86400
    Server: cloudflare
    CF-RAY: 549c21ddad9d6e48-SJC

Which is somewhat strange behaviour..

Proposal @kleisauke : can we do a test with sending Cache-Control: max-age=900 to Cloudflare for all errors? Probably we need to check this behaviour in our test-enviroment first.

We've changed a setting regarding Edge-Cache TTL on Cloudflare's side. Errors should be cached for a shorter time now. @nerg4l : Could you test/confirm if this is working?

For now I used curl with i.
When using for the linked URL I get cf-cache-status: HIT and age: 174847 (around 2 days). However if I test it with the given https://images.weserv.nl/?url=weserv.nl/doesnotexist.jpg I get cf-cache-status: HIT and an age value below 200 and then cf-cache-status: EXPIRED and no age header as expected. Probably the first one was cached before the modification.
Details below.
Command 1:
curl https://images.weserv.nl/\?url\=www.gravatar.com%2Favatar%2F82a206ae2834d1f3ce0affb688ab83d1%3Fs%3D100%26d%3Dwavatar -i

Response 1:
HTTP/2 404
date: Mon, 23 Dec 2019 19:07:35 GMT
content-type: application/json
set-cookie: __cfduid=d1b64c5491999ca8a5232d841a308e3841577128055; expires=Wed, 22-Jan-20 19:07:35 GMT; path=/; domain=.weserv.nl; HttpOnly; SameSite=Lax
vary: Accept-Encoding
access-control-allow-origin: *
x-images-api: 5
cf-cache-status: HIT
age: 174847
strict-transport-security: max-age=31536000; includeSubDomains; preload
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
alt-svc: h3-24=":443"; ma=86400, h3-23=":443"; ma=86400
server: cloudflare
cf-ray: 549c9e8aeafed105-TXL

{"status":"error","code":404,"message":"The requested URL returned error: 503"}%

Command 2:
curl https://images.weserv.nl/\?url\=weserv.nl/doesnotexist.jpg -i

Result 2 Cache HIT:
HTTP/2 404
date: Mon, 23 Dec 2019 19:09:47 GMT
content-type: application/json
set-cookie: __cfduid=d81c53835fd7ff439a16946eb4f2650001577128187; expires=Wed, 22-Jan-20 19:09:47 GMT; path=/; domain=.weserv.nl; HttpOnly; SameSite=Lax
vary: Accept-Encoding
access-control-allow-origin: *
x-images-api: 5
cf-cache-status: HIT
age: 11
strict-transport-security: max-age=31536000; includeSubDomains; preload
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
alt-svc: h3-24=":443"; ma=86400, h3-23=":443"; ma=86400
server: cloudflare
cf-ray: 549ca1c21e10d109-TXL

{"status":"error","code":404,"message":"Will not follow a redirection to itself"}%

Result 2 Cache Expire:
HTTP/2 404
date: Mon, 23 Dec 2019 19:16:40 GMT
content-type: application/json
set-cookie: __cfduid=d5ca9cb3c6bf39cc600b8ee03697559be1577128600; expires=Wed, 22-Jan-20 19:16:40 GMT; path=/; domain=.weserv.nl; HttpOnly; SameSite=Lax
vary: Accept-Encoding
access-control-allow-origin: *
x-images-api: 5
cf-cache-status: EXPIRED
strict-transport-security: max-age=31536000; includeSubDomains; preload
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
alt-svc: h3-24=":443"; ma=86400, h3-23=":443"; ma=86400
server: cloudflare
cf-ray: 549cabd7ed5ad121-TXL

{"status":"error","code":404,"message":"Will not follow a redirection to itself"}%

Cache has been invalidated for the particular image. If you want, you can send us an email with the images that are affected by this bug and need to be purged from cache.

We have around 35000 users with Gravatar images so it would be a bit of a hassle to check all of them, probably I will just modify the URL script by adding 1 as a query parameter.
Thank you for the quick reaction.


Consider adding a More Info button to the modal dialog prompt requiring users to install ipykernel

Probably consider adding a more info button here.

Validation.
Create a clean venv and select it as your notebook kernel.
Run a cell to get the missing ipykernel message
See if More Info shows and that it links to the wiki.

I do see the More Info button, but if I click it, the modal dialog reappears, not sure if that's expected?


Is it possible to call a proc on throttled? == true

I'd like to do something simple like Airbrake.notify whenever jobs are throttled. Is there a way to add a callback type behavior? I've looked through the code and don't see anything native off hand. TIA!

It's not available ATM, but I will think on adding this feature - I like it.

@ixti I'd be happy to submit a PR with a little guidance on how you'd like to see this implemented. Thanks!

Idea is pretty same to :key option, so that if it's on top-level it's passed to both strategies.

Or in fact it might be even simplier - and remain top-level only. So that in here:
https://github.com/sensortower/sidekiq-throttled/blob/master/lib/sidekiq/throttled/strategy.rb#L57-L66
you will just add something like:
if @concurrency && @concurrency.throttled?(...)
  @observer&.call(@concurrency)
  return true
end

 # ...

In fact it might be implemented per-strategy - will be even easier.

I think it's better to add callback process which will be called after throttled per threshold with :after_throttled option:
class MyWorker
  include Sidekiq::Worker
  include Sidekiq::Throttled::Worker

  sidekiq_options :queue => :my_queue

  # Define a callback process
  AFTER_THROTTLED = -> do
    puts "MyJob was throttled"
  end

  sidekiq_throttle({
    :concurrency => { :limit => 10 },
    # Apply a callback process
    :threshold => { :limit => 1_000, :period => 1.hour, after_throttled: AFTER_THROTTLED }
  })

  def perform
    # ...
  end
end

There are several points:

The callback process will be basically called ONLY ONCE after throttled in terms of impact on performance.
The callback process doesn't have any arguments.
Doesn't provide callback function in concurrency because some settings will be throttled so frequently. (e.g. concurrency { limit: 1 } will cause throttling on every sidekiq polling )

As for No.2, I first assumed that it is better to process *job_args or somthing like key_suffix, however, given No.1 (called ONLY ONCE) and hence it can't handle every time throttled, I made the conclusion not to have any arguments.

The callback process will be basically called ONLY ONCE after throttled in terms of impact on performance.

This description was not clear, I meant that frequently processing should be prevented in terms of impact on performance and don't mean will be never processed in the future. I think it is better to prevent during a period of ttl from the throttling started, like below.
def throttled?(*job_args)
  return false unless (job_limit = limit(job_args))

  key = key(job_args)
  period = period(job_args)

  kwargs = {
    :keys => [key],
    :argv => [job_limit, period, Time.now.to_f]
  }

  throttled = Sidekiq.redis { |redis| 1 == SCRIPT.eval(redis, kwargs) }
  throttled ? callback_once("#{key}:callback", period)
  throttled
end

def callback_once(key, ttl)
  MUTEX.synchronize do
    return if callback_paused?(key)

    @callback&.call
    pause_callback(key, ttl)
  end
end

def pause_callback(key, ttl)
  Sidekiq.redis { |conn| conn.setex(key, ttl.to_i, "pause") }
end

def callback_paused?(key)
  Sidekiq.redis { |conn| conn.exists(key) }
end

I think it's better to add callback process which will be called after throttled per threshold with :after_throttled option:

The idea is to provide a generic observer. For both types of throttling. Also I really don't like after_* name. It sounds as if there's corresponding before_* and around_*. While in our case we just need to notify of some event happened.

The callback process doesn't have any arguments.

No. We want to pass arguments that caused throttling. That's essential. For example, let's say there's a job that pings different hosts based on given URL, on throttling we would really be curious which host throttles us most.

Doesn't provide callback function in concurrency because some settings will be throttled so frequently. (e.g. concurrency { limit: 1 } will cause throttling on every sidekiq polling )

I don't understand reasoning. Configuration of concurrency is NOT fixed, and in different cases it might be different. And we definitely want to know when concurrency occurs if we don't expect that. Your reason of not observing concurrency throttling can be easily applied to threshold as well - user can set VERY low limit of threshold just as easy as he can do that with concurrency - both strategies are configurable and can occur on every sidekiq polling.

The callback process will be basically called ONLY ONCE after throttled in terms of impact on performance. I meant that frequently processing should be prevented in terms of impact on performance and don't mean will be never processed in the future. I think it is better to prevent during a period of ttl from the throttling started, like below.

That is totally out of the scope of this feature or sidekiq-throttled in general. It's consumer's responsibility to throttle/skip notifications not our. Sidekiq-throttled should be simple and straight forward in this aspects. Skipping notifications on any conditions is not what we should do. Observer should do that on it's own.

API design
I see 2 ways of this feature to be implemented. First is straight forward and I want this iteration of the feature to focus exactly on this version:
class MyWorker
  # ...

  MY_OBSERVER = lambda do |strategy, url|
    return if :concurrency  == strategy # skip concurrency if we want that

    App.statsd.increment "my_worker:#{URI(url).host}"
  end

  sidekiq_throttled **some_config, :observe => MY_OBSERVER

  def perform(url)
    # ...
  end
end

In other words, observer is an object that responds to #call message, and accepts 1+ arguments (first argument is strategy name, rest is what was passed to #perform.
Second way is to provide Sidekiq::Observable mixin that will enhance child classes with .on and .notify methods. But conceptually under the hood both ways will look pretty much the same.
See Also: https://en.wikipedia.org/wiki/Observer_pattern
Performance
If you want to make this more performant, we can discuss this after the first iteration of this feature. As I see how we can make notifications to be handled in a dedicated thread. But most likely this is totally out of the sidekiq-throttled scope.

Thank you for sharing your opinion. I totally agree with you.

Doesn't provide callback function in concurrency because some settings will be throttled so frequently. (e.g. concurrency { limit: 1 } will cause throttling on every sidekiq polling )

I first thought that there was :period option in threshold which means 'threshold is controllable' unlike concurrency, but depending on the configuration is the same as concurrency and as you said there will be definitely the case we want to know when concurrency occurs. Thanks!

Performance

Aside from performance, if neither using thresholds' period as TTL nor applying option such as a :observe_duration, MY_OBSERVER will be called the same number of sidekiq workers times on every sidekiq polling once after reached to limit. I think it seems a bit annoying but I understand that it's out of the sidekiq-throttled scope, so I won't care it in the first iteration of this feature.

Implemented as of v0.11.0