ManagedLedger should streamline the read requests

Problems
Currently managed ledger read entries in a very large batch requests - 100 entries by default. This is an inefficient approach. We should streamline the read requests like what dlog is doing.

I have seen in a production deployment the consumer can never catch up if bookie's avg read latency is 10+ms bad due to the disk and other workloads running on same machine. the problem can potentially be mitigated if we can have a proper readahead mechanism in managed like what dlog is doing.

@sijie
I agree that pushing that mechanism to the low level API will be useful

This is a also blocking issue for practical use of tiered storage as historical retention, since replay from tiered storage (at least s3) is too slow.
It'd be great if number of read-ahead threads and/or outstanding requests is tunable, as many blob/object store scale throughput proportional to number of connections.

@vicaya thank you for your feedback. @MarvinCai are you willing to give it a try?

@sijie sorry just saw the replies, how about I start with doc with problem statement and try propose a solution. If everything looks good then we can proceed from there.

@MarvinCai Are you working on this issue already?

@sijie  I was new to BK code base and was reading the some LeadferHandler and DL codes tog figure out what should be change, have a simple doc about what I think may need to change.
I haven't start writing any code yet. If someone with more experience with BK plan to work on it then I'm fine. Else I'm also glad to help.

Is there any progress on this issue? Whenever people ask me about why we don't use tiered storage, I have to point them to this issue for why it's too slow for us (readers cannot read fast enough from tiered storage and backlogs would build up even at moderate throughput (<1000msgs/s with small messages (<100 bytes), which is <100KB/s).

@nicoloboschi you could be interested in working on a fix for this issue if @MarvinCai doesn't have time to work on this topic

@MarvinCai, @sijie, @eolivelli - It looks like the feature to improve read throughput for blob storage is still outstanding. I'm very interested in helping to contribute this feature. Do you think it deserves its own issue to start a dedicated discussion for offloading?

@michaeljmarshall you can start a discussion with a proposal.
you can start on dev@pulsar in order to have more people in the discussion.

@eolivelli - thanks, I'll do that.


Updating README

Updating README with a small note on how the security advisories project can be used in build plans.

Based on the work happening in https://github.com/symfony/diversity/issues/9, I suggest avoiding the word simply in the new note.

Closing in favor of #405


:bug: Do not exclude scheduling.k8s.io group

Summary
The resource group scheduling.k8s.io is currently include 1 resource called PriorityClass that allow users to set scheduling priority for deployments/pods. We should allow down syncing for PriorityClass.
Related issue(s)
Fixes #

/lgtm
/approve


[stable/rabbitmq-ha] docker-entrypoint.sh: line 196: /var/lib/rabbitmq/.erlang.cookie: Permission denied

Is this a request for help?:
NO
Is this a BUG REPORT or FEATURE REQUEST? (choose one):
Bug report

Version of Helm and Kubernetes:
Helm 2.12.3
Kubernetes 1.13.3 (on DigitalOcean)
Which chart:
stable/rabbitmq-ha
What happened:
When installing the chart and using a customized values.yaml file with persistentVolume enabled, the first pod loops in in error without ever becoming ready and "Running". By checking the pod's logs one can see the following error:

$ kubectl -n workspace logs -f rabbitmq-0
/usr/local/bin/docker-entrypoint.sh: line 196: /var/lib/rabbitmq/.erlang.cookie: Permission denied

What you expected to happen:
To have the rabbitmq pods starting and running as usual.
How to reproduce it (as minimally and precisely as possible):
helm install --namespace workspace --name rabbitmq -f rabbitmq-helm-config.yaml stable/rabbitmq-ha
Anything else we need to know:
Our config file worked at the beginning of last week and the previous weeks, we notice that recently the rabbitmq and rabbitmq-ha added a securityContext section that was non-existing a few days ago. Also, we are using this with DigitalOcean to provide dynamic provisioning with Persistent Volume Claims and someone pointed that this error manifests in the stable/rabbitmq for NFS file systems on stable/rabbitmq in #11926

We have a few options to try. If you can try each of these fixes in turn to see which solves your problem:

Via helm setsecurityContext.fsGroup=101
kubectl edit the statefulset and add into the init container: /bin/chown -R 100:101 /var/lib/rabbitmq
Via helm set securityContext=null

The last one should definitely work since it's reverting to the previous functionality of running as root, but please try the others first so we can find a better approach.

@steven-sheehy Thanks, the 1st one worked immediately.
The new securityContext.runAsGroup: 101 setting in #11863 doesn't seem to work as intended (at least for me in DigitalOcean), that config doesn't even show up in the pod's yaml config as soon as you install it, whereas fsGroup shows up immediately.
When I execute the command below for both versions, that's what I get on the working and non-working YAML's securityContext sections:
$ kubectl -n master get statefulset rabbitmq -o yaml
NOT WORKING:
securityContext:
runAsNonRoot: true
runAsUser: 100
WORKING:
securityContext:
fsGroup: 101
runAsNonRoot: true
runAsUser: 100

Thanks for verifying. runAsGroup and fsGroup have separate uses with the former being used for the process and the latter for the filesystem. runAsGroup is just not enabled by default in clusters and has to be enabled via the feature gate RunAsGroup=true. So sounds like we just need to add fsGroup: 101. Do you think you would be able to create a PR adding that one line and bump the bugfix version in the Chart.yaml?

@steven-sheehy Sure, I can do it, but do I need you or the team to give me privileges for that?

Nope, anyone can create PRs. I can approve PRs for this chart once you do. I can't however approve my own though, so your help would be appreciated. 👍

How is fsGroup=101 supposed to universally resolve this issue? When I mount a persistent volume /var/lib/rabbitmq is mounted with root:root ownership and as the init container runs in the security context I can't chown the folder from there. Please could you explain how to get the root:101 permissons of /var/lib/rabbitmq in the first place in order to proceed with fsGroup=101.


Roadmap for V2 of the ML Model Extension

:rocket: Feature Request
originally posted on https://github.com/crim-ca/dlm-extension/issues/7
Below are acceptance criteria for #2 the new version of the ML Model Extension. After all these items are done I think #2 is ready to merge and we can publish a new version 2 release to https://github.com/stac-extensions/ml-model/issues . I'm working on most of these, let me know if you think other acceptance criteria should be included cc @fmigneault

[x] Incorporate feedback from @fmariv and others at @earthpulse

I had a meeting with @fmariv, they're keen to review the extension, provide feedback and see if we can align it more with general ML use cases. They're working on supporting the STAC extension ecosystem for ML as a part of the EOTDL initiative
https://hackmd.io/DBRF1sQCS1WmSqygJNKQJQ?view
feel free to comment first and then submit a PR against https://github.com/rbavery/dlm-extension/tree/validate

[x] address #5

this will involve segregating flat fields from objects that are not meant to be searched on and are instead used for inference model loading, model input and output processing, and documenting in detail the accelerator and runtime details needed to run the model

[x] #4 by creating output objects for common detection tasks. At a minimum, start with single label classification, semantic segmentation, object detection. This will help resolve common ambiguities when interpreting model outputs, like the bbox coordinate ordering for object detection. EDIT this can be left for after the v2 release
[x] resolve #3, deciding how to refer to model extension metadata. within it's own STAC item/collection json or are these fields composed with common metadata in STAC json representing a spatiotemporal asset?
[x] https://github.com/crim-ca/dlm-extension/pull/2#discussion_r1446651049 create an example and schema for the final spec
[x] reorganize and update the stac_model extension validation and serialization package according to the final spec: https://github.com/crim-ca/dlm-extension/pull/2#issuecomment-1882002335
[x] https://github.com/rbavery/dlm-extension/pull/2
[x] consider inputs from user feedback (https://github.com/stac-extensions/ml-model/issues/13)
[x] consider community examples applying DLM (https://github.com/sentinel-hub/stac-ml-example)
[x] #8
[x] #9
[x] #12
[x] #13
[x] integrate JSON schema representing new properties/fields (mix of https://github.com/crim-ca/dlm-extension/tree/main/json-schema and https://github.com/rbavery/dlm-extension/pull/1)
[x] #14
[x] https://github.com/stac-extensions/classification/issues/48
[ ] https://github.com/stac-extensions/example-links/issues/4
[ ] https://github.com/stac-extensions/processing/issues/31
[ ] https://github.com/stac-extensions/processing/pull/33
[ ] consideration of end_datetime: null not supported by STAC Core spec
(https://github.com/radiantearth/stac-spec/issues/1268)

[ ] new issue to track for the specific use case: https://github.com/radiantearth/stac-spec/issues/1161

Remaining issues moved to https://github.com/stac-extensions/mlm/issues/28 and https://github.com/stac-extensions/mlm/issues/29.


Add broad range of inputs for leap benchmark?

Right now the benchmark for the leap uses the actual test cases. This is not a very representative sample of years, and could skew the benchmark. How about using a range of numbers like 1000-2000 to get a good spread of values?

Or maybe benchmark single numbers of the different cases -- not divisible by 4, 4 but not 100, and so on.    I think it's okay for comparing one implementation to another, but less meaningful for understanding the time of an individual operation.

Ah, I was thinking of the benchmark as comparing implementations.
Yeah, we could separate them out by case.

So how about single number tests plus a test of a 400 year period.  The 400 year period would be good for comparing implementations but representing the frequency of occurrence of the cases.  The individual cases might help understanding the 400 year results.
You know this pattern of benchmarking all the test cases together is just something that's easy as a comparable time.  (I think the old pattern of starting and stopping the timer doesn't really work.)  More interesting or more meaningful benchmarks might be devised for a number of the exercises.

So how about single number tests plus a test of a 400 year period.

That sounds very good.

leap $ go test -tags example -bench .
PASS
Benchmark2015Once-8<PHONE_NUMBER>	         2.24 ns/op
Benchmark2014Once-8<PHONE_NUMBER>	         2.26 ns/op
Benchmark1900Once-8	500000000	         3.54 ns/op
Benchmark1600Once-8	500000000	         3.54 ns/op
Benchmark2015x400-8	 2000000	       924 ns/op
Benchmark2014x400-8	 2000000	       929 ns/op
Benchmark1900x400-8	 1000000	      1401 ns/op
Benchmark1600x400-8	 1000000	      1405 ns/op
BenchmarkRange400-8	 1000000	      1049 ns/op
ok  	github.com/exercism/xgo/leap	18.751s

I'm not sure what it means.  I would guess complexities of the CPU architecture and the the compiler code generator make this hard to understand.  Maybe it's best to just use the 400 year period and not include the other tests.

Yeah, agreed. The 400 year range will at least give people a number that would simulate a varied set of inputs.


when i do ctrl+S to overwrite existing file - it fails....

when i do ctrl+S to overwrite existing file - it fails....
thanks

This is the expected behavior. Please use a unique name for the file, or just remove the file before running the script.

thanks abiyani, love your script ....I was wondering if you can add the option to save html page as complete page and/or html only. (sometimes switching between the two is required)   thanks


Binary dependency is not correctly handled

Xmake Version
2.7.1
Operating System Version and Architecture
Windows 10 Pro 20H2
Describe Bug
When target b depends on a where a is a binary target, xmake does not build a before b
Expected Behavior
Always build a before b if b depends on a
Project Configuration
xmake.lua:
target('a')
  set_kind('binary')
  add_files('a.c')

rule('rulea')
  on_load(function(t)
    local o=t:autogendir()
    t:add('deps', 'a')
    t:add('includedirs', o)
  end)
  before_buildcmd_file(function(t, b, s)
    import('core.project.project')
    local o=t:autogendir()
    if not os.isdir(o) then os.mkdir(o) end
    local n=s:gsub('%\\','/'):match('([^/]+)%.[^%.]+$')
    b:vrunv(project.target('a'):targetfile(), {
      path.join(o, n..'.h')
    })
  end)

target('b')
  set_kind('binary')
  add_rules('rulea')
  add_deps('a') -- to be or not to be, xmake behaves the same
  add_files('b.c')
  add_files('b.c', {rule='rulea'})

a.c:
#include <stdio.h>

int main(int c, char** v)
{
  if (c!=2) return -1;
  FILE* f=fopen(v[1], "wb");
  if (!f) return -1;
  fprintf(f, "enum { HELLO = 1024 };\n");
  fclose(f);
  return 0;
}

b.c:
#include "b.h"
#include <stdio.h>

int main()
{
  printf("%d\n", HELLO);
  return 0;
}

Additional Information and Error Logs
you will need manually call xmake -b a before xmake to build above example

see https://xmake.io/#/manual/project_target?id=buildacross_targets_in_parallel

Thanks, that works!
but I'm fairly sure that depending on a binary target semantically implies requiring that binary to be executable (fully linked) - or what else can the dependency statement mean? Please consider making it automatic

By default, all target source files have no dependencies on each other, so even if add_deps is set, they can be compiled in full parallel, which can significantly improve compilation speed. This has nothing to do with whether or not they depend on binary at the moment.
Their final linking dependencies will still be linked according to add_deps, ensuring that no linking errors occur.
But you are artificially creating dependencies between sources by relying on a target in the before_build_file script, breaking the default parallelism policy. So you can only disable cross-target parallelism for a given target according to the set_policy I mentioned.
This is not automatically recognised because xmake has no way of knowing the dependencies between the script configured by the user in the before_build_file script and other targets.

Well, I've digged into the xmake code base a little just now, and IMHO changing this line in build.lua from
if dep:policy("build.across_targets_in_parallel") == false then

into
if dep:policy("build.across_targets_in_parallel") == false or dep:kind() == 'binary' then

is sufficient, and reasonable.

You're still missing my point, even the source files of the binary target and the source files of the other targets can be compiled in full parallel with each other without relying on add_deps, it's just that you're making extra dependencies in the before_build_file that make the source files dependent on target A.

Well, I've digged into the xmake code base a little just now, and IMHO changing this line in build.lua from
if dep:policy("build.across_targets_in_parallel") == false then

into
if dep:policy("build.across_targets_in_parallel") == false or dep:kind() == 'binary' then

is sufficient, and reasonable.

If you do this, the source files in the binary target will not be compiled in parallel with the source files of other targets, which will affect the compilation speed.

You're still missing my point, even the source files of the binary target and the source files of the other targets can be compiled in full parallel with each other without relying on add_deps, it's just that you're making extra dependencies in the before_build_file that make the source files dependent on target A.

I got the point, that add_deps join depencies at linking stage.
But I think when you depend on an binary, you'd always want that binary to be executable before compiling, as it just makes no sense at link stage afterall. So the binary being depended on shouldn't be compiled in parallel with other projects anyway.

Well, I've digged into the xmake code base a little just now, and IMHO changing this line in build.lua from
if dep:policy("build.across_targets_in_parallel") == false then

into
if dep:policy("build.across_targets_in_parallel") == false or dep:kind() == 'binary' then

is sufficient, and reasonable.

If you do this, the source files in the binary target will not be compiled in parallel with the source files of other targets, which will affect the compilation speed.

So be it, if not so, I will have to add set_policy('build.across_targets_in_parallel', false) in the dependent project manually to make it build, ended up all the same.

It makes sense for compilation speedup, if you have 100 binary targets and only one source file per binart target, then I can compile 100 source files in parallel and then link them one by one. Compile a.c, b.c, c.c in parallel ...
But as you say, the 100 source files can only be compiled one by one. compile a.c, link a, compile b.c, link b ...
It will be very slow.

But as you say, the 100 source files can only be compiled one by one. compile a.c, link a, compile b.c, link b ...

When b depends on a, c depends on b, ... that is the case, and that's what we want, wasn't it?
If c doesn't depend on a or b, nothing stops it from being compiled in parallel

If b depends on a and c depends on b, but I can still compile their source files in parallel, but now I can't.

If b depends on a and c depends on b, but I can still compile their source files in parallel,

If a is a binary, it's definitely not something to link to, in what way would b depends on a then?

Or let's see if it's clearer if I put it this way:
What I need is something like add_deps_thats_ready_to_execute_before_compiling(...), and I thought add_deps(binary) should imply that, as binary dependencies only make sense before compiling.

If b depends on a and c depends on b, but I can still compile their source files in parallel,

If a is a binary, it's definitely not something to link to, in what way would b depends on a then?

Run other binary programs in scripts like before_link, after_build, etc. As you are doing now, b also depends on a. and run a in b script

Or let's see if it's clearer if I put it this way:
What I need is something like add_deps_thats_ready_to_execute_before_compiling(...), and I thought add_deps(binary) should imply that, as binary dependencies only make sense before compiling.

so you just need set_policy for this target.

If b depends on a and c depends on b, but I can still compile their source files in parallel,

If a is a binary, it's definitely not something to link to, in what way would b depends on a then?

Run other binary programs (a) in b scripts like before_link, after_build, etc. As you are doing now, b also depends on a. and run a in b script

Okay.. that makes sense. Still think something like add_deps(..., {deadline='before_build'}) would be better, but I'm good with the set_policy solution


Update env as separate action.

This Pull Request was opened by Chef Automate user Jon Morrow
Signed-off-by: Jon Morrow<EMAIL_ADDRESS>Ready to merge? View this change in Chef Automate and click the Approve button: https://delivery.chef.co/e/chef/#/organizations/Delivery-Build-Cookbooks/projects/habitat-build/changes/252faadf-ef13-497c-8590-d92e70d8005f

It would also be nice to have some context in the commit message that explains what the outcomes of this fix are. Perhaps writing tests and adding them along with the commit will make that more clear, but I'd lean towards a more verbose commit message as well to help our future selves.


JaCoCo docs don't work with new android project

I am trying to set up JaCoCo plugin according to the docs.
Expected Behavior

Following the setup steps on a new Android project should cause a build using ./gradlew test or ./gradlew jacocoTestReport
Current Behavior

Running test produces this output
Executing tasks: [jacocoTestReport] in project /Users/tony/Documents/MSM/MyApplication

FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* Where:
Build file '/Users/tony/Documents/MSM/MyApplication/app/build.gradle' line: 7

* What went wrong:
A problem occurred evaluating project ':app'.
> Could not find method test() for arguments [build_7vdwupgycm5b9wgi2lrj6s14x$_run_closure1@a3d81ac] on project ':app' of type org.gradle.api.Project.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
A problem occurred configuring project ':app'.
> com.android.builder.errors.EvalIssueException: compileSdkVersion is not specified. Please add it to build.gradle

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

BUILD FAILED in 260ms

Context

I am trying to follow docs in order to simplify an issue I am having with test coverage, but I haven't gotten that far yet. Side note: The docs aren't clear about WHERE in the build.gradle file to place each code section - A sample build.gradle file would be ideal.
Steps to Reproduce

Start a new empty project in Android Studio
Make sure everything is up to date
Add code from example 1 and 2, found here, to build.gradle file, which now looks like this (Example 3 and 4 can be added as well, but it changes nothing:

plugins {
    id 'com.android.application'
    id 'org.jetbrains.kotlin.android'
    id 'jacoco'
}

test {
    finalizedBy jacocoTestReport // report is always generated after tests run
}
jacocoTestReport {
    dependsOn test // tests are required to run before generating the report
}

android {
    compileSdk 31

    defaultConfig {
        applicationId "com.example.myapplication"
        minSdk 23
        targetSdk 31
        versionCode 1
        versionName "1.0"

        testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
    }

    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
        }
        debug {
            testCoverageEnabled true
        }
    }
    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }
    kotlinOptions {
        jvmTarget = '1.8'
    }
}

dependencies {

    implementation 'androidx.core:core-ktx:1.7.0'
    implementation 'androidx.appcompat:appcompat:1.4.1'
    implementation 'com.google.android.material:material:1.5.0'
    implementation 'androidx.constraintlayout:constraintlayout:2.1.3'
    testImplementation 'junit:junit:4.13.2'
    androidTestImplementation 'androidx.test.ext:junit:1.1.3'
    androidTestImplementation 'androidx.test.espresso:espresso-core:3.4.0'
}

Run ./gradlew test

Your Environment

I'm using MacOS, gradle 7.4, Java 11, and the latest version of Android Studio. And the latest version of the docs ( https://docs.gradle.org/current/userguide/jacoco_plugin.html )

Thank you for your interest in Gradle!
This is an issue with the Android Gradle Plugin which is developed by Google and cannot be addressed in the core of the Gradle Build Tool.
Please open such Android-related issues on the Android Issue Tracker at https://source.android.com/source/report-bugs


Agent container stopped, device offline

Description
I've recently had a few instances where the client docker container has stopped, this obviously results in being unable to connect.
After regaining access via local means and inspecting the docker logs there's no obvious cause for the agent container to have stopped. I've checked the restart policy, and it's correctly set to on-failure
Had anyone else experienced this? Any ideas for how to debug? Since remote connection is lost when this issue occurs is's a big problem and seems tricky to diagnose.
The effected clients are running Debian 10 Buster arm.
Edition
Community
Version
v0.10.2

I understand that you encountered an unexpected issue that seems unrelated to the ShellHub agent. However, it could be related to the version or settings of Docker that you are using. To help me better assist you, please provide the following information:

docker version
docker inspect shellhub

Thanks for your reply, here's the info requested:
docker version
docker version
Client: Docker Engine - Community
 Version:           20.10.18
 API version:       1.41
 Go version:        go1.18.6
 Git commit:        b40c2f6
 Built:             Thu Sep  8 23:11:38 2022
 OS/Arch:           linux/arm
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          24.0.2
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.4
  Git commit:       659604f
  Built:            Thu May 25 21:51:03 2023
  OS/Arch:          linux/arm
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker inspect shellhub
pi@emonhp:~ $ docker inspect shellhub
[
    {
        "Id": "xxxxxxxxxxxxxxxxxxxxxx",
        "Created": "2022-12-17T13:55:04.119703817Z",
        "Path": "/bin/agent",
        "Args": [],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 8213,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2023-08-03T01:56:20.801180212Z",
            "FinishedAt": "2023-07-04T08:17:55.572590156Z"
        },
        "Image": "sha256:9de8da09b7e8e5ab2eed4d45a7fe9015fbcd60222527c2ddfc582a26297a2a04",
        "ResolvConfPath": "/etc/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/709e6d40eb2cb83b6411d75b84f073ab8658a596c33fa6b48c6a3b8ff10290c3/hostname",
        "HostsPath": "/var/lib/docker/containers/709e6d40eb2cb83b6411d75b84f073ab8658a596c33fa6b48c6a3b8ff10290c3/hosts",
        "LogPath": "/var/lib/docker/containers/709e6d40eb2cb83b6411d75b84f073ab8658a596c33fa6b48c6a3b8ff10290c3/709e6d40eb2cb83b6411d75b84f073ab8658a596c33fa6b48c6a3b8ff10290c3-json.log",
        "Name": "/shellhub",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "/var/run:/var/run",
                "/var/log:/var/log",
                "/:/host",
                "/dev:/dev",
                "/var/run/docker.sock:/var/run/docker.sock",
                "/etc/passwd:/etc/passwd",
                "/etc/group:/etc/group",
                "/etc/resolv.conf:/etc/resolv.conf"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "host",
            "PortBindings": {},
            "RestartPolicy": {
                "Name": "on-failure",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "ConsoleSize": [
                0,
                0
            ],
            "CapAdd": null,
            "CapDrop": null,
            "CgroupnsMode": "host",
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "private",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "host",
            "Privileged": true,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": [
                "label=disable"
            ],
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": [],
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": null,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": null,
            "ReadonlyPaths": null
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/e28e99641b3c4ca0ec44a0f8e631bcfc353d21d66afb7afb6182ed138e9b92e4-init/diff:/var/lib/docker/overlay2/f34b6e22759951bc21f083f6b45dcb50c4ebb115743e6605597edd7bae9322e8/diff:/var/lib/docker/overlay2/65c5de48724303d6a34c1b0d1885d046ca59cedc7cf1184d619e97718b3da35d/diff:/var/lib/docker/overlay2/4107f0d6c0a866945f23d80f776de965f9d28e02c7531d37c7948f5485af6879/diff:/var/lib/docker/overlay2/200afc4cc4bc9b8929da327ea14002d72354b00d477ff374804e46ee6369ff88/diff:/var/lib/docker/overlay2/c69ec8b67f9c15eda24ed53734007b2fb98a13cc3b9191cfc3f6f0075291502a/diff:/var/lib/docker/overlay2/746e8e3ccc0547a110a68c1090c772ca87b80ceb1b7b2226cbcd2882455ff94d/diff:/var/lib/docker/overlay2/e4ebe1bd52e0c29337e0156595f29204afebb85ed946f259e2cc0a0c7a8042a4/diff",
                "MergedDir": "/var/lib/docker/overlay2/e28e99641b3c4ca0ec44a0f8e631bcfc353d21d66afb7afb6182ed138e9b92e4/merged",
                "UpperDir": "/var/lib/docker/overlay2/e28e99641b3c4ca0ec44a0f8e631bcfc353d21d66afb7afb6182ed138e9b92e4/diff",
                "WorkDir": "/var/lib/docker/overlay2/e28e99641b3c4ca0ec44a0f8e631bcfc353d21d66afb7afb6182ed138e9b92e4/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [
            {
                "Type": "bind",
                "Source": "/var/run",
                "Destination": "/var/run",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/var/run/docker.sock",
                "Destination": "/var/run/docker.sock",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/dev",
                "Destination": "/dev",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/etc/group",
                "Destination": "/etc/group",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/etc/passwd",
                "Destination": "/etc/passwd",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/etc/resolv.conf",
                "Destination": "/etc/resolv.conf",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/",
                "Destination": "/host",
                "Mode": "",
                "RW": true,
                "Propagation": "rslave"
            },
            {
                "Type": "bind",
                "Source": "/var/log",
                "Destination": "/var/log",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            }
        ],
        "Config": {
            "Hostname": "xxxxxxxxx",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "SHELLHUB_SERVER_ADDRESS=xxxxxxxxxxxxxxxxxx",
                "SHELLHUB_PRIVATE_KEY=/host/etc/shellhub.key",
                "SHELLHUB_TENANT_ID=xxxxxxxxxxxxxx",
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": null,
            "Image": "shellhubio/agent:v0.10.2",
            "Volumes": null,
            "WorkingDir": "/",
            "Entrypoint": [
                "/bin/agent"
            ],
            "OnBuild": null,
            "Labels": {}
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "xxxxxxxxxxxxxx",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/var/run/docker/netns/default",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "host": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "xxxxxxxxxxxxxxxxx",
                    "EndpointID": "xxxxxxxxxxxxxxxxx",
                    "Gateway": "",
                    "IPAddress": "",
                    "IPPrefixLen": 0,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "",
                    "DriverOpts": null
                }
            }
        }
    }
]

I've noticed this happening again to several more of my devices, looking at docker logs here are some errors which may be related:
2023/03/30 13:34:30.728357 ERROR RESTY Get "https://xxxxxxxx:443/info%3Fagent_version=v0.10.2": x509: certificate has expired or is not yet valid: current time 2023-03-30T13:34:30Z is before 2023-04-17T00:00:00Z, Attempt 1
2023/08/08 21:46:11 revdial.Listener: error writing message to server: write tcp <IP_ADDRESS>:45182-><IP_ADDRESS>:443: write: broken pipe
The https certificate has not changed and is still valid. I reboot did not restore the container, I had to connect via alternative means and manually start the container docker start shellhub to restore the shellhub connection.
Around the time the device disconnected, I had to delete some logs on the shellhub server to clean up some space since there as 10's of Gb of logs, could this have effected the devices?
 truncate -s 0 /var/lib/docker/containers/**/*-json.log
The log size is an ongoing issue, I never managed to figure out how to use docker compose to limit the log size as mentioned in https://github.com/shellhub-io/shellhub/issues/1929

@glynhudson Could you please confirm if this issue still persists in the latest version?


Separate staging from download.

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[x] Have you written new tests for your changes? Here's an example.

[x] Have you successfully run brew tests with your changes locally?

Separate unpacking the download from download strategies. These unpack strategies should then also used in the Container classes used for staging casks.

As a general comment it's not clear due to the file change what's been moved as-is and what's been changed. I'd suggest moving to unpack strategy in one PR (or at least one commit) and then making the relevant changes in another.

Nothing has been moved as-is, because previously everything was just unpacked in PWD. So every command had to be adapted to support unpacking into a specified directory.

Nothing has been moved as-is, because previously everything was just unpacked in PWD. So every command had to be adapted to support unpacking into a specified directory.

Sure; I'm suggesting this adaption happen in a future PR (or even separate commits in this one) and this one focuses on trying to just move things as-is. It's very difficult to review what's changed and what hasn't, as-is.

@JCount and @MikeMcQuaid, one last look?

For kubernetes-cli,
Before:
bash-3.2$ git status
Not currently on any branch.
Untracked files:
  (use "git add <file>..." to include in what will be committed)

	.brew_home/

nothing added to commit but untracked files present (use "git add" to track)

After:
bash-3.2$ git status
Not currently on any branch.
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	typechange: .bazelrc
	typechange: .kazelcfg.json
	typechange: BUILD.bazel
	typechange: Makefile
	typechange: Makefile.generated_files
	typechange: WORKSPACE

Untracked files:
  (use "git add <file>..." to include in what will be committed)

	.brew_home/

no changes added to commit (use "git add" and/or "git commit -a")

This is now causing CI failure for anything that checks that its build info reports a clean not dirty tag.

iMac-TMP:Homebrew joe$ git bisect bad
5b3bbb76c9d224c08ae498677868b69aab71ff7c is the first bad commit
commit 5b3bbb76c9d224c08ae498677868b69aab71ff7c
Author: Markus Reiter<EMAIL_ADDRESS>Date:   Sun Jul 1 23:35:29 2018 +0200

    Separate staging from download.

:040000 040000 6d5d52263444dbca74d030b9714bd87afff1d386 183e6d00db7777ddcadf1ce9017e320bed2d45c2 M	Library
iMac-TMP:Homebrew joe$

This has broken the ability to install cromwell on El Capitan.
See https://jenkins.brew.sh/job/Homebrew Testing/1023/ and https://jenkins.brew.sh/job/Homebrew Core Pull Requests/27643/
==> brew install --verbose cromwell
==> FAILED
/usr/bin/sandbox-exec -f /private/tmp/homebrew20180720-32126-1i16pn6.sb nice /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.7/bin/ruby -W0 -I /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/gems/2.3.0/gems/did_you_mean-1.0.0/lib:/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/site_ruby/2.3.0:/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/site_ruby/2.3.0/x86_64-darwin9.0:/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/site_ruby/2.3.0/universal-darwin9.0:/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/site_ruby:/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/vendor_ruby/2.3.0:/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/vendor_ruby/2.3.0/x86_64-darwin9.0:/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/vendor_ruby/2.3.0/universal-darwin9.0:/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/vendor_ruby:/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/2.3.0:/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/2.3.0/x86_64-darwin9.0:/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.7/lib/ruby/2.3.0/universal-darwin9.0:/usr/local/Homebrew/Library/Homebrew/cask/lib:/usr/local/Homebrew/Library/Homebrew -- /usr/local/Homebrew/Library/Homebrew/build.rb /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/cromwell.rb --verbose
==> Downloading https://github.com/broadinstitute/cromwell/releases/download/33.1/cromwell-33.1.jar
Already downloaded: /Users/brew/Library/Caches/Homebrew/cromwell-33.1.jar
==> Verifying cromwell-33.1.jar checksum
unzip -qq /Users/brew/Library/Caches/Homebrew/cromwell-33.1.jar -d /private/tmp/d20180720-32128-18ejwwt
warning [/Users/brew/Library/Caches/Homebrew/cromwell-33.1.jar]:  76 extra bytes at beginning or within zipfile
  (attempting to process anyway)
error [/Users/brew/Library/Caches/Homebrew/cromwell-33.1.jar]:  reported length of central directory is
  -76 bytes too long (Atari STZip zipfile?  J.H.Holm ZIPSPLIT 1.1
  zipfile?).  Compensating...
error:  expected central file header signature not found (file #133751).
  (please check that you have transferred or created the zipfile in the
  appropriate BINARY mode and that you have compiled UnZip properly)
Error: Failure while executing: unzip -qq /Users/brew/Library/Caches/Homebrew/cromwell-33.1.jar -d /private/tmp/d20180720-32128-18ejwwt


List of tweets for a twitter wall

Provide a list of tweets from our own database (updated elsewhere once a
minute). The newest tweet will be first.
Closes #123.

Pull Request Test Coverage Report for Build 412

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+8.0e**-05%**) to 99.975%

Totals

Change from base Build 410:
8.0e-05%

Covered Lines:
3382

Relevant Lines:
3382

💛  - Coveralls


Improvement for background.js

First things first: Thank you.
Since I'm not a Chrome user, I'll just share my thoughts about some bits of your code.

https://github.com/newfolder0/chrome-clacks/blob/03020050e7107bce48aaf8604bc7f5f7a153d194/src/background.js#L21
The variable does not describe a match, but a pattern.

https://github.com/newfolder0/chrome-clacks/blob/03020050e7107bce48aaf8604bc7f5f7a153d194/src/background.js#L29
You may can solve this with a ternary operator.

https://github.com/newfolder0/chrome-clacks/blob/03020050e7107bce48aaf8604bc7f5f7a153d194/src/background.js#L39
In case you're just about to merge an Array here, why don't use Array.prototype.join()?

https://github.com/newfolder0/chrome-clacks/blob/03020050e7107bce48aaf8604bc7f5f7a153d194/src/background.js#L69
Since there is only one expression, why don't merge the if condition via &&? (else if will take care of handling the else-branch).

Thanks for the input :).
1.You're not a match, but a pattern. Take that (alright then, I'll change it). I originally called it 'expression' but that seemed too generic.

I'm working backwards (nope, I don't know why either), see number 3.

Hmm, newClacks is actually an array of objects so newClacks.join("\n") doesn't work right out of the box. Here: http://stackoverflow.com/questions/16607557/javascript-perform-join-on-value-in-object-array are a lot of different suggestions but I don't think any of them are really better. They all require confusing one-liners or extra libraries and I think a straightforward for loop is just easier to read in this case. Actually scratch all of that, it was just fixed by the guy sitting next to me, now it looks like this: http://pastebin.com/ejG13tBw

True, I think that was originally there for readability but I like your suggestion. Done!


[MM-28222][MM-28215] Payment Warnings

Summary
Banners to display when the credit card is about to expire, is expired and the most recent invoice failed.
NOTES:

Used <10 days as the 'about to expire' metric, can be easily changed.
Using the last invoice displayed on the Subscription screen to show if the most recent payment failed, not sure if it needs other logic.

Will need UX/QA review after merge.
Ticket Link
https://mattermost.atlassian.net/browse/MM-28222
https://mattermost.atlassian.net/browse/MM-28215

/update-branch

/cherry-pick cloud-ga

Cherry pick is scheduled.


rm old super-linter action for editorconfig

guess this also needs changes to required checks

My intuition is telling me the same. However, the configuration says, all actions are allowed 😕

allowed, yes
issues is these two where the required part must be removed in branch protection(repo settings)

Ok, I got it down to one

What happens if we remove the required from the other linter? Would GitHub allow to merge, if linting action fails?


History UX Preview LightBox

Added a Modal Box that does the work, has previous and next buttons as well.

I nitpicked the pull request (comments inline). Once you address those I'll run the code locally and comment on using the functionality. Make sure you leave a comment here when you push new commits, otherwise I am unlikely to take another look.

@smcgregor I've made the changes. Kindly run the code locally. I'm waiting for a feedback.

@smcgregor I've made that variable local now. Check it out now, once.

This is excellent :)
A few comments:

Content is variable length which means putting the buttons only at the bottom will make it difficult to page through many different posts. I recommend moving the buttons to the side or the top since that doesn't move when the content is longer.
Future versions may include previews of the preceding/following links, but that does not need to be included here.
The "destroy" button will be clicked accidentally here so it probably needs a confirmation before destroying the content.

@smcgregor I've made the changes you described above. Maybe test it again locally once, and let me know if it seems fine now.

I'd missed committing the css file. Now, check once.

Yes, we can use the $ sign for jquery variables. I had read it on some page and followed the same here.
Secondly, I've fixed the nitpicks above @smcgregor.

OK, I'll run it locally one more time then likely merge. Have you made any progress on unit testing this?

I've made progress, you can say. Not too much. But, no need to worry, I'm
gonna give all my time to this, so it might just be a few hours away before
I've written a test file for it.
On 13 January 2015 at 01:37, Sean McGregor<EMAIL_ADDRESS>wrote:

OK, I'll run it locally one more time then likely merge. Have you made any
progress on unit testing this?
—
Reply to this email directly or view it on GitHub
https://github.com/privly/privly-applications/pull/141#issuecomment-69636351
.

--
Bhavul Gauri
Student
Msc.(Hons.) Mathematics
B.E.(Hons.) Computer Science and Engineering
Birla Institute of Technology & Science, Pilani, Goa Campus
NH 17B, Zuarinagar
GOA 403 726
Contact: +91-839-0352362

Destroying content gives a console error: Uncaught TypeError: Cannot read property 'closest' of undefined

Fixed the console error on destroying content. Now, the row also hides properly.

Ready for review?

Yes.


 This would be nice to include under the unnecessary_* set of hints.
(If this should be a lint instead of a hint, feel free to move it!)

This looks like just plain old dead code. The rest of the expression continuation after the throw is definitely dead.

Yeah good point!


Plot Generator Rework

https://github.com/AlpsBTE/Plot-System/issues/47

Ready for review / merge


CI test, DO NOT MERGE

Issue Number. Is this pull request related to any outstanding issues? If so, list the issue number.
Describe the changes made. A clear and concise description of what the problem is and what you did to fix it. E.g. [...] was happening and I've changed [...] to fix it.
Goals and Outstanding Issues. A clear and concise list of goals (to be) accomplished.

[ ]

Major files changed.

[ ]

Status.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] I have read the CONTRIBUTING document.
[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

Additional context. Add any other context or screenshots about the pull request here.

Codecov Report

Merging #7 (3e43824) into develop (bcce34d) will decrease coverage by 0.79%.
The diff coverage is 0.00%.

@@             Coverage Diff             @@
##           develop       #7      +/-   ##
===========================================
- Coverage    41.95%   41.16%   -0.80%
===========================================
  Files          131      131
  Lines        16909    16906       -3
===========================================
- Hits          7094     6959     -135
- Misses        9815     9947     +132

Flag
Coverage Δ

unittests
41.16% <0.00%> (-0.80%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

src/westpa/core/sim_manager.py
79.05% <0.00%> (ø)

src/westpa/mclib/__init__.py
6.17% <0.00%> (-50.62%)
:arrow_down:

src/westpa/cli/tools/w_pdist.py
72.03% <0.00%> (-12.80%)
:arrow_down:

src/westpa/work_managers/zeromq/worker.py
82.62% <0.00%> (-12.68%)
:arrow_down:

src/westpa/work_managers/processes.py
77.86% <0.00%> (-12.30%)
:arrow_down:

src/westpa/cli/tools/w_direct.py
79.01% <0.00%> (-6.18%)
:arrow_down:

src/westpa/cli/tools/w_assign.py
77.64% <0.00%> (-3.14%)
:arrow_down:

src/westpa/work_managers/zeromq/core.py
80.90% <0.00%> (-0.76%)
:arrow_down:

src/westpa/core/h5io.py
40.97% <0.00%> (-0.53%)
:arrow_down:

src/westpa/tools/core.py
87.69% <0.00%> (-0.52%)
:arrow_down:

... and 5 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Added CreateOnUpdate and applied formating

Allow for creating accounts when they don't exist when running on update mode.
Checks to see if current secret matches the secret being supplied
Implemented Changes
Updated to allow for creation of accounts when you run in update.
Checks to see if current secret matches the secret to be changed to to prevent excessive versions and reduce load on CPM
Connected Issue/Story
None
Definition of Done
At least 1 todo must be completed in the sections below for the PR to be
merged.
Changelog
[x ] The CHANGELOG has been updated, or
This PR does not include user-facing changes and doesn't require a
CHANGELOG update
Test coverage
This PR includes new unit and integration tests to go with the code
changes, or
[ x] The changes in this PR do not require tests
Documentation
[ x] Docs (e.g. READMEs) were updated in this PR
A follow-up issue to update official docs has been filed here: insert issue ID
This PR does not require updating any documentation
Behavior
[ x] This PR changes product behavior and has been reviewed by a PO, or
These changes are part of a larger initiative that will be reviewed later, or
No behavior was changed with this PR
Security
Security architect has reviewed the changes in this PR,
These changes are part of a larger initiative with a separate securit

Looks good :)
Thanks


docker compose build : Unable to locate package build-essential

Hello,
I am running through the tutorial, but my docker compos build keeps on failing.
Command I run
docker compose up --build

output
[+] Building 240.1s (20/50)                                                                docker:default
 => [web internal] load build definition from Dockerfile                                             0.0s
 => => transferring dockerfile: 2.20kB                                                               0.0s
 => [web internal] load .dockerignore                                                                0.0s
 => => transferring context: 216B                                                                    0.0s
 => [css internal] load .dockerignore                                                                0.0s
 => => transferring context: 216B                                                                    0.0s
 => [css internal] load build definition from Dockerfile                                             0.0s
 => => transferring dockerfile: 2.20kB                                                               0.0s
 => [worker internal] load metadata for docker.io/library/python:3.12.1-slim-bookworm                0.6s
 => [js internal] load metadata for docker.io/library/node:20.6.1-bookworm-slim                      0.6s
 => [js internal] load .dockerignore                                                                 0.0s
 => => transferring context: 216B                                                                    0.0s
 => [js internal] load build definition from Dockerfile                                              0.0s
 => => transferring dockerfile: 2.20kB                                                               0.0s
 => [worker internal] load build definition from Dockerfile                                          0.0s
 => => transferring dockerfile: 2.20kB                                                               0.0s
 => [worker internal] load .dockerignore                                                             0.0s
 => => transferring context: 216B                                                                    0.0s
 => [worker assets 1/7] FROM docker.io/library/node:20.6.1-bookworm-slim@sha256:2dab2d0e8813ee1601f  0.0s
 => [css internal] load build context                                                                0.0s
 => => transferring context: 7.77kB                                                                  0.0s
 => [js internal] load build context                                                                 0.0s
 => => transferring context: 7.77kB                                                                  0.0s
 => CACHED [worker assets 2/7] WORKDIR /app/assets                                                   0.0s
 => ERROR [css assets 3/7] RUN apt-get update   && apt-get install -y --no-install-recommends bui  239.5s
 => [worker app  1/10] FROM docker.io/library/python:3.12.1-slim-bookworm@sha256:123229cfb27c384ee1  0.0s
 => [web internal] load build context                                                                0.0s
 => => transferring context: 7.77kB                                                                  0.0s
 => CACHED [web app  2/10] WORKDIR /app                                                              0.0s
 => [worker internal] load build context                                                             0.0s
 => => transferring context: 7.77kB                                                                  0.0s
 => CANCELED [web app  3/10] RUN apt-get update   && apt-get install -y --no-install-recommends b  239.5s
------
 > [css assets 3/7] RUN apt-get update   && apt-get install -y --no-install-recommends build-essential   && rm -rf /var/lib/apt/lists/* /usr/share/doc /usr/share/man   && apt-get clean   && groupmod -g "1001" node && usermod -u "1001" -g "1001" node   && mkdir -p /node_modules && chown node:node -R /node_modules /app:
20.39 Ign:1 http://deb.debian.org/debian bookworm InRelease
40.41 Ign:2 http://deb.debian.org/debian bookworm-updates InRelease
60.43 Ign:3 http://deb.debian.org/debian-security bookworm-security InRelease
80.46 Ign:1 http://deb.debian.org/debian bookworm InRelease
100.5 Ign:2 http://deb.debian.org/debian bookworm-updates InRelease
120.5 Ign:3 http://deb.debian.org/debian-security bookworm-security InRelease
140.5 Ign:1 http://deb.debian.org/debian bookworm InRelease
160.5 Ign:2 http://deb.debian.org/debian bookworm-updates InRelease
179.4 Ign:3 http://deb.debian.org/debian-security bookworm-security InRelease
199.4 Err:1 http://deb.debian.org/debian bookworm InRelease
199.4   Temporary failure resolving 'deb.debian.org'
219.4 Err:2 http://deb.debian.org/debian bookworm-updates InRelease
219.4   Temporary failure resolving 'deb.debian.org'
239.4 Err:3 http://deb.debian.org/debian-security bookworm-security InRelease
239.4   Temporary failure resolving 'deb.debian.org'
239.4 Reading package lists...
239.4 W: Failed to fetch http://deb.debian.org/debian/dists/bookworm/InRelease  Temporary failure resolving 'deb.debian.org'
239.4 W: Failed to fetch http://deb.debian.org/debian/dists/bookworm-updates/InRelease  Temporary failure resolving 'deb.debian.org'
239.4 W: Failed to fetch http://deb.debian.org/debian-security/dists/bookworm-security/InRelease  Temporary failure resolving 'deb.debian.org'
239.4 W: Some index files failed to download. They have been ignored, or old ones used instead.
239.4 Reading package lists...
239.5 Building dependency tree...
239.5 Reading state information...
239.5 E: Unable to locate package build-essential
------
failed to solve: process "/bin/sh -c apt-get update   && apt-get install -y --no-install-recommends build-essential   && rm -rf /var/lib/apt/lists/* /usr/share/doc /usr/share/man   && apt-get clean   && groupmod -g \"${GID}\" node && usermod -u \"${UID}\" -g \"${GID}\" node   && mkdir -p /node_modules && chown node:node -R /node_modules /app" did not complete successfully: exit code: 100

Docker version 24.0.7, build afdd53b
Docker Compose version v2.21.0

Thanks for the help

Hi,
That looks like the internet is not working inside of your container based on Temporary failure resolving 'deb.debian.org'.
How do you have Docker installed?

I installed docker/docker compose per the instructions here https://docs.docker.com/engine/install/ubuntu/
I tried restarting the services like this but it did not work
sudo service docker restart
sudo /etc/init.d/docker restart

The solution here did it for me https://stackoverflow.com/a/51037151
Thanks for your help


Support iOS 7.0 with cocoapods.

Hi guys,
I really like your work.
But I had a problem, my project still supports iOS 7.0. I hope you can down version of Bond in Cocoapods from 8.0 to 7.0.
Huy.

it wont work with iOS 7 since it is written in Swift. It requires compiling the po as Dynamic Framework which can be embedded into iOS 8 deployment targets

It works fine if you include Bond as Dynamic Framework with optional option. I did a lot of research about using Dynamic Framework in Swift with iOS7. :dancer:

interesting, could you post some link or anything that refers to this?

Here bro :dancer:
http://quellish.tumblr.com/post/103107323582/dynamic-frameworks-on-ios-7

But there are no Dynamic Frameworking linking in Cocoapods. Cocoapods imports Bond source code directly to your project, so you don't need to worry about that issue.

Huy,
Check out v3.4.2. I lowered deployment target to 7.0. Give us your feedback if it works. Thanks

Cocoapods adds Swift source code to the project? Didnt know about that :O

@zdenektopic  sorry about that, in switf it's must be framework.
@srdanrasic thank you, I'll give a try right now.

Hi zdenektopic,
It works as expected, but I don't know Apple accept or reject if my project use Dynamic Framework target to 7.0

Well it is weak-linked, it builds with stable release version of Xcode so I suppose they should accept it? Specially if Cocoapods supports dynamic frameworks with deployment target 7.0

Many thanks :dancer:

I get ld: embedded dylibs/frameworks are only supported on iOS 8.0 and later (@rpath/Bond.framework/Bond) for architecture arm64 when trying to compile for iOS 7.

It's an error or warning?

It was an error. Solved, see my edit.

however, i am kinda sure this will only work if the application is running on iOS 8. If you run it on iOS 7 it will crash. Right?

It won't crash if in your framework does not contain "App extension". App extension only supports iOS > 8.0.

@huyphams How to write the Podfile to specify Bond should be weak linked? I'm eager to know, thanks.


pyasn1/type/univ.py:711: DeprecationWarning: int returned non-int (type SizedInteger)

Testing reports following warnings:
=============================== warnings summary ===============================
tests/test_rfc5913.py::ClearanceTestCase::testOpenTypes
tests/test_rfc5913.py::ClearanceTestCase::testOpenTypes
tests/test_rfc5913.py::ClearanceTestCase::testOpenTypes
tests/test_rfc5913.py::ClearanceTestCase::testDerCodec
tests/test_rfc5913.py::ClearanceTestCase::testDerCodec
tests/test_rfc5913.py::ClearanceTestCase::testDerCodec
tests/test_rfc5917.py::ClearanceSponsorTestCase::testOpenTypes
tests/test_rfc5755.py::CertificateWithClearanceTestCase::testOpenTypes
  $(PYTHON_DIR)/vendor-packages/pyasn1/type/univ.py:711: DeprecationWarning: __int__ returned non-int (type SizedInteger).  The ability to return an instance of a strict subclass of int is deprecated, and may be removed in a future version of Python.
    return SizedInteger(value).setBitLength(len(value))

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

Oops, this is caused by pyasn1, not by pyasn1-modules.  And it was already fixed via https://github.com/etingof/pyasn1/commit/db8f1a7930c6b5826357646746337dafc983f953.


tf.contrib.quantize bug: errors within weights/activations quantization

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes, provided
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 16.04
TensorFlow installed from (source or binary): Binary
TensorFlow version (use command below): 1.8.0
Python version:  3.6
Bazel version (if compiling from source): -
GCC/Compiler version (if compiling from source): -
CUDA/cuDNN version: -
GPU model and memory: -
Exact command to reproduce: python quantization_test.py (file provided)

Describe the problem
tf.contrib.quantize (experimental and non-experimental) automatic quantization produce close to but not quantized results as when you manually introduce fakequant operations.
Weights and/or activations are close to the quantized values, but they are not correct.
Source code / logs
import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt

D_SIZE = 2048
BATCH_SIZE = 128
LR = 1e-2
EPOCHS = 2000

SHOW_INTERVAL = 20
OUT_CH = 8

def create_dummy_model(is_training=True):
    # prepare dataset
    np_features = np.arange(-int(D_SIZE) / 2, int(D_SIZE / 2))
    features_arr = tf.constant(np_features, dtype=tf.float32)
    np_labels = np.zeros((D_SIZE, OUT_CH), dtype=float)
    print(np_labels.shape)
    for i in range(OUT_CH):
        np_labels[:, i] = 2 * (i + 1) * np_features
    print('np_features:')
    print(np_features[0:8])
    print('np_labels:')
    print(np_labels[0:8])

    labels_arr = tf.constant(np_labels, dtype=tf.float32)
    trn_data = tf.data.Dataset.zip(
        (tf.data.Dataset.from_tensor_slices(features_arr),
         tf.data.Dataset.from_tensor_slices(labels_arr))
    )
    trn_data = trn_data.shuffle(D_SIZE)
    trn_data = trn_data.batch(BATCH_SIZE)
    trn_data = trn_data.repeat(1)
    trn_data = trn_data.prefetch(1)

    iterator = tf.data.Iterator.from_structure(trn_data.output_types,
                                               trn_data.output_shapes)
    features, labels = iterator.get_next()
    data_init_op = iterator.make_initializer(trn_data)

    # global step
    global_step = tf.train.get_or_create_global_step()

    # simple model dense, WITH ACTIVATION TO TEST QUANTIFICATION
    input = tf.reshape(features, (BATCH_SIZE, 1))
    flow = tf.identity(input, name='input_tensor')
    # batch_size x INPUT_CH
    flow = tf.reshape(flow, (-1, 1))
    flow = tf.layers.dense(flow, units=OUT_CH, name='dense')
    flow = tf.reshape(flow, (-1, OUT_CH), name='out')

    # Build forward pass of model.
    # Make the loss op
    with tf.variable_scope('loss'):
        tf.logging.debug('[loss] size of out: %s',
                         flow.get_shape().as_list())
        power = tf.pow(labels - flow, 2)
        power = tf.reduce_sum(power, 1)
        # power = tf.reduce_sum(power)
        loss = tf.reduce_mean(power, name='m_loss')
    tf.summary.scalar('loss', loss)

    # Optimizer
    if is_training:
        opt_op = tf.train.AdamOptimizer(LR).minimize(loss, global_step)
        # opt_op = tf.train.GradientDescentOptimizer(1e-4).minimize(loss, global_step)
    else:
        opt_op = False
    # Merge all the summaries
    merged = tf.summary.merge_all()

    return {
        'iterator': iterator,
        'features': features,
        'labels': labels,
        'data_init_op': data_init_op,
        'global_step': global_step,
        'flow': flow,
        'loss': loss,
        'opt_op': opt_op,
        'merged': merged
    }

def quantize_tensor(np_array, n_bits):
    """Quantizes an np_array"""
    min_wt = np.min(np_array)
    max_wt = np.max(np_array)
    # clap values if required
    if min_wt < -2**(n_bits-1):
        min_wt = -2**(n_bits-1)
    if max_wt > (2**(n_bits-1) - 1) :
        max_wt = (2**(n_bits-1) - 1)
    # find number of integer bits to represent this range
    int_bits = int(np.ceil(np.log2(max(abs(min_wt), abs(max_wt)))))
    print('int_bits: ', int_bits)
    # remaining bits are fractional bits (1-bit for sign)
    frac_bits = n_bits -1 - int_bits
    # floating point weights are scaled and rounded to [-128,127], which are used in
    # the fixed-point operations on the actual hardware (i.e., microcontroller)
    quantized = np.round(np_array * (2**frac_bits))
    # To quantify the impact of quantized weights, scale them back to
    # original range to run inference using quantized weights
    return quantized / (2**frac_bits)

def train_quantization(quantize=True, w_b=3, a_b=3):

    g_tr = tf.Graph()
    with g_tr.as_default():

        model = create_dummy_model(True)

        # Call the training rewrite which rewrites the graph in-place with
        # FakeQuantization nodes and folds batchnorm for training. It is
        # often needed to fine tune a floating point model for quantization
        # with this training tool. When training from scratch, quant_delay
        # can be used to activate quantization after training to converge
        # with the float graph, effectively fine-tuning the model.
        if quantize:
            quant_delay = int((D_SIZE / BATCH_SIZE) * EPOCHS * 3 / 4)
            print('quant_delay: ', quant_delay)
            print('w_b: ', w_b)
            print('a_b: ', a_b)
            tf.contrib.quantize.experimental_create_training_graph(weight_bits=w_b,
                                                                   activation_bits=a_b,
                                                                   quant_delay=quant_delay)

        with tf.Session(graph=g_tr) as sess:
            # global variables Initializing
            sess.run(tf.global_variables_initializer())
            # local variables Initializing
            sess.run(tf.local_variables_initializer())

            if quantize:
                train_writer = tf.summary.FileWriter(
                    './tmp/train_quantized_w' + str(w_b) + '_a_' + str(a_b),
                    sess.graph)
            else:
                train_writer = tf.summary.FileWriter('./tmp/train_normal',
                                                     sess.graph)

            epoch_counter = 0
            nn_in = None
            nn_out = None
            nn_loss = None
            for epoch_counter in range(EPOCHS):

                # re-initialize the dataset iterator
                sess.run(model['data_init_op'])

                try:
                    while True:
                        _, nn_in, nn_out, nn_labels, nn_loss, gs, summary = sess.run(
                            [model['opt_op'],
                             model['features'],
                             model['flow'],
                             model['labels'],
                             model['loss'],
                             model['global_step'],
                             model['merged']])

                        # tensorboard and  statistics
                        train_writer.add_summary(summary, gs)
                except tf.errors.OutOfRangeError:
                    # do nothing
                    if epoch_counter % SHOW_INTERVAL == 0:
                        print(epoch_counter,  '/', EPOCHS,
                              ': loss ', nn_loss,
                              '. Global step: ', gs)

            weights = sess.run(tf.get_default_graph().get_tensor_by_name(
                'dense/kernel:0'))
            saver = tf.train.Saver()
            if quantize:
                trained_model_path = './tmp/quantized_w' + \
                    str(w_b) + '_a_' + str(a_b) + '.ckpt'
            else:
                trained_model_path = './tmp/normal.ckpt'

            saver.save(sess, trained_model_path)

        # debug
        print('nn_out shape: ', nn_out.shape)
        print('nn_labels shape: ', nn_labels.shape)
        print('nn_out different values in OUT_CH (', OUT_CH, ') elements:',
              len(np.unique(nn_out)))
        print('weights shape: ', weights.shape)
        print('weights: ', weights)
        print('weights different values in OUT_CH (', OUT_CH, ') elements:',
              len(np.unique(weights)))

        plt.plot(nn_in[:], nn_out[:, 0], label='nn_out',
                 marker='o', markersize=5)
        plt.plot(nn_in[:], nn_labels[:, 0], label='labels',
                 marker='*', markersize=5)
        plt.legend(loc='lower right')
        plt.show()
        plt.plot(nn_in[:], nn_out[:, 5], label='nn_out',
                 marker='o', markersize=5)
        plt.plot(nn_in[:], nn_labels[:, 5], label='labels',
                 marker='*', markersize=5)
        plt.legend(loc='lower right')
        plt.show()

        plt.plot(weights[0, :], label='weights',
                 marker='o', markersize=5)
        plt.legend(loc='lower right')
        plt.show()

        print('[debug] exit training')
        return trained_model_path

def inference_quantization(trained_model_path, quantize=True, w_b=3, a_b=3):

    #################################
    # Reset default graph
    tf.reset_default_graph()
    #################################

    g_inf = tf.Graph()
    with g_inf.as_default():

        model = create_dummy_model(False)

        with tf.Session(graph=g_inf) as sess:
            # Call the eval rewrite which rewrites the graph in-place with
            # FakeQuantization nodes and fold batchnorm for eval.
            if quantize:
                print('w_b: ', w_b)
                print('a_b: ', a_b)
                tf.contrib.quantize.experimental_create_eval_graph(
                    weight_bits=w_b,
                    activation_bits=a_b)

            saver = tf.train.Saver()
            saver.restore(sess, trained_model_path)
            # frozen_graphdef = tf.graph_util.convert_variables_to_constants(
            #       sess, sess.graph_def, ["out"])
            print('[debug] restored')

            # re-initialize the iterator, but this time with training data

            sess.run(model['data_init_op'])

            nn_in, nn_out, nn_labels, nn_loss = sess.run([
                model['features'],
                model['flow'],
                model['labels'],
                model['loss']])
            weights = sess.run(tf.get_default_graph(
            ).get_tensor_by_name('dense/kernel:0'))

        # debug
        print('nn_out shape: ', nn_out.shape)
        print('nn_labels shape: ', nn_labels.shape)
        print('nn_out different values in OUT_CH (', OUT_CH, ') elements:',
              len(np.unique(nn_out)))
        print('weights shape: ', weights.shape)
        # print('weights: ', weights)
        print('weights different values in OUT_CH (', OUT_CH, ') elements:',
              len(np.unique(weights)))
        print('weights: ', weights)
        q_weights = quantize_tensor(weights, w_b)
        print('q_weights: ', q_weights)
        plt.plot(nn_in[:], nn_out[:, 0], label='nn_out',
                 marker='o', markersize=5)
        plt.plot(nn_in[:], nn_labels[:, 0], label='labels',
                 marker='*', markersize=5)
        plt.legend(loc='lower right')
        plt.show()
        plt.plot(nn_in[:], nn_out[:, 5], label='nn_out',
                 marker='o', markersize=5)
        plt.plot(nn_in[:], nn_labels[:, 5], label='labels',
                 marker='*', markersize=5)
        plt.legend(loc='lower right')
        plt.show()

        plt.plot(weights[0, :], label='weights',
                 marker='o', markersize=5)
        plt.legend(loc='lower right')
        plt.show()

def test_quantization():

    # train_quantization(False)
    trained_model_path = train_quantization(quantize=True, w_b=2, a_b=2)
    inference_quantization(
        trained_model_path=trained_model_path, quantize=True, w_b=2, a_b=2)

test_quantization()

Nagging Assignees @suharshs, @poxvoculi: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

This is because the FakeQuant operations do more than just quantize. First its very important that floating point 0.0 maps directly to a quantized value, otherwise padding errors can propagate. This nudging can be seen here: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/kernels/fake_quant_ops_functor.h#L41
Also we restrict the weight values to being quantized from [1,255] instead of [0,255] to enable a ARM Neon optimization for our kernels.
Thus, this is working as intended, please do reopen and correct me if I am misunderstanding or mistaken.

Are your sure you are running the FakeQuant nodes after the weights. The weights will still retain their full precision but the output fo the following FakeQuant will emulate the correct quantization. So make sure for each weight you are actually checking the 'weight_quant/FakeQuantWithMinMaxVars' operation following it.

You were completely right. only the Bias nodes were not quantized.
Thanks.


Upgrade to v2.28.1 - > v2.29.0 An error was reported

Error message：

Error executing query: expanding series: proxy Series(): rpc error: code = Aborted desc = No StoreAPIs matched for this query

Symptom The same query generates an error after v2.29.0 is upgraded, and v2.28.1 is rolled back to recover the query
An error is as follows

Hmm, maybe something related to https://github.com/thanos-io/thanos/pull/5296. Do you know how we could reproduce this? :thinking:

Do not check the specific problem and error, directly roll back the version, Looking forward to repair
The new 'thanos_frontend_sharding_middleware_queries_total' metrics added to query-fe was not found

I upgrade 0.29.0.rc early and not found problem, I mainly use sidecar mode, so just upgrade sidecar, querier, query-frontend, not include receiver.
today when i upgrade receiver, the 'No StoreAPIs matched for this query' problem occured.
it only error when query the tenant which use receive mode with thanos-query, query other tenant is work ok, query via thanos-query-frontend is work ok.
the receive print this logs
2022-11-07 18:27:56 level=warn ts=2022-11-07T10:27:56.053906788Z caller=proxy.go:282 component=receive component=proxy request="min_time:1667816575801 max_time:1667816875801 matchers:<name:\"app\" value:\"xxx\" > matchers:<name:\"namespace\" value:\"xxx\" > matchers:<name:\"server\" value:\"xxx\" > matchers:<name:\"tenant_id\" value:\"default\" > matchers:<name:\"__name__\" value:\"process_resident_memory_bytes\" > aggregates:COUNT aggregates:SUM partial_response_disabled:true " err="No StoreAPIs matched for this query" stores="store LabelSets: {receive_cluster=\"cluster_xxx\", receive_replica=\"thanos-receiver-ingestor-xxx-0\", tenant_id=\"xxx\"} Mint:<PHONE_NUMBER>098 Maxt:<PHONE_NUMBER>854775807 filtered out: external labels [{receive_cluster=\"cluster_xxx\", receive_replica=\"thanos-receiver-ingestor-xxx-0\", tenant_id=\"xxx\"}] does not match request label matchers: [app=\"xxx\" namespace=\"xxx\" server=\"xx\" tenant_id=\"default\" __name__=\"process_resident_memory_bytes\"];store LabelSets: {receive_cluster=\"cluster_xxx\", receive_replica=\"thanos-receiver-ingestor-xxx-0\", tenant_id=\"default\"} Mint:<PHONE_NUMBER>854775807 Maxt:<PHONE_NUMBER>854775807 filtered out: does not have data within this time period: [1667816575801,1667816875801]. Store time ranges: [9223372036854775807,9223372036854775807]"

I think the PR #4886 may be relevant.

hi @GiedriusS @matej-g
I notice the prometheus code minTime is math.MaxInt64 but in thanos multi tsdb code it is math.MinInt64, so the query is filtered out unexpectedly?
https://github.com/prometheus/prometheus/blob/6dd4e907a31a58d1b738aa5debbfc9c5e1ed32ac/tsdb/head.go#L288-L289
https://github.com/thanos-io/thanos/blob/ef3a3310bf34c179ffaf73d94d0a4f60f0d07a99/pkg/store/tsdb.go#L115-L116

Hey @hanjm, it's possible, although I cannot see how this breaks the time range filtering

@BigbigY do you use Thanos Receive?

my receiver instance have two tenant data, tenant_id=comp with time range is Mint:<PHONE_NUMBER>098 Maxt:<PHONE_NUMBER>854775807
tenant_id=default with time range is Mint:<PHONE_NUMBER>854775807 Maxt:<PHONE_NUMBER>854775807
the receiver may exposed multi tenant label info, but only one time range info.
https://github.com/thanos-io/thanos/blob/ef3a3310bf34c179ffaf73d94d0a4f60f0d07a99/pkg/store/proxy.go#L218-L236
when query receiver in querier, it will be not filtered out.
store Addr: *.*.*.*:10907 LabelSets: {receive_cluster=\"cluster_comp\", receive_replica=\"thanos-receiver-ingestor-comp-0\", tenant_id=\"comp\"},{receive_cluster=\"cluster_comp\", receive_replica=\"thanos-receiver-ingestor-comp-0\", tenant_id=\"default\"} Mint:<PHONE_NUMBER>098 Maxt:<PHONE_NUMBER>854775807 queried

when request arrived the receiver, it will filtered out.
2022-11-07 18:27:56 level=warn ts=2022-11-07T10:27:56.053906788Z caller=proxy.go:282 component=receive component=proxy request="min_time:1667816575801 max_time:1667816875801 matchers:<name:\"app\" value:\"xxx\" > matchers:<name:\"namespace\" value:\"xxx\" > matchers:<name:\"server\" value:\"xxx\" > matchers:<name:\"tenant_id\" value:\"default\" > matchers:<name:\"__name__\" value:\"process_resident_memory_bytes\" > aggregates:COUNT aggregates:SUM partial_response_disabled:true " err="No StoreAPIs matched for this query" stores="store LabelSets: {receive_cluster=\"cluster_xxx\", receive_replica=\"thanos-receiver-ingestor-xxx-0\", tenant_id=\"comp\"} Mint:<PHONE_NUMBER>098 Maxt:<PHONE_NUMBER>854775807 filtered out: external labels [{receive_cluster=\"cluster_xxx\", receive_replica=\"thanos-receiver-ingestor-xxx-0\", tenant_id=\"comp\"}] does not match request label matchers: [app=\"symphony\" namespace=\"Production\" server=\"piano\" tenant_id=\"default\" __name__=\"process_resident_memory_bytes\"];store LabelSets: {receive_cluster=\"cluster_xxx\", receive_replica=\"thanos-receiver-ingestor-comp-0\", tenant_id=\"default\"} Mint:<PHONE_NUMBER>854775807 Maxt:<PHONE_NUMBER>854775807 filtered out: does not have data within this time period: [1667816575801,1667816875801]. Store time ranges: [9223372036854775807,9223372036854775807]"

then the receiver return a Abort error.

I think the receiver should not return abort error event if no client matched in multi-tsdb, right?

Sounds like a valid bug. Help wanted

We are seeing this and are not using Thanos Receive.

@BigbigY do you use Thanos Receive?

Thanos Receive has not been upgraded

It looks like then this might be because of changes in query? Either way thanks @hanjm for the report, I'll try to take a look whether I can reproduce / fix this.

Hey @hanjm , thanks for the pointers, I did not try to reproduce it, just went through the code (although it should not be too hard to create a test case if my thinking is correct) and I think you're right, but it's not only because of time range.
I believe the filtering in the downstream proxy store (in receiver) works correctly - because for one store we cannot match labels (different tenant) and for the other store we cannot match time range, it correctly returns that no stores were found.
The problem seems to be that the upstream proxy store (in querier) does not have enough information to filter out this downstream proxy store. Since when the downstream store returns info to upstream store, it reports labels and time range for both stores (combined) - the upstream store therefore cannot filter it out. However, once we hit the downstream proxy, we find that in fact neither one of the two stores match and we go to no matching store error  and this is what we see in the upstream proxy store (https://github.com/thanos-io/thanos/blob/main/pkg/store/proxy.go#L322).
I'm not sure how this worked before https://github.com/thanos-io/thanos/pull/5296 and if this could have changed, but does this make sense @hanjm @GiedriusS?

This was also brought to us by @SuperQ at Community Hours.
Generally, the problem that Ben is having is that when multiple Queries are chained, no Store matched situation might be not an error. It's was done as error for easy debugging on configuration error, but not it is impacting correct queries.
Sounds like https://github.com/thanos-io/thanos/issues/5862 is the solution, we will take a look, thanks!

In #5937, I wonder if we need to get that fancy with a control flag.


Cannot build on rust 1.46

This is an error I saw on a Linux/amd64 machine while building the rust bindings. My guess is we just need to document the right version of the compiler and toolchain we should be relying upon
➜  rust git:(40443d6) rustc --version
rustc 1.46.0 (04488afe3 2020-08-24)
➜  rust git:(40443d6) cargo test --features azure
   Compiling deltalake v0.2.0 (/home/tyler/source/github/delta-io/delta.rs/rust)
error: no rules expected the token `,`
  --> rust/tests/read_error_test.rs:12:46
   |
12 |         deltalake::DeltaTableError::NotATable,
   |                                              ^ no rules expected this token in macro call

error: aborting due to previous error

error: could not compile `deltalake`.

To learn more, run the command again with --verbose.

That compiler is old enough to where I don't know if there's anything worth documenting.


Fixes missing inner exception information in ScheduledEventException

Description
When a ScheduledEventException was thrown, IAlgorithm.RunTimeError was not set as ScheduledEventException , but as Exception with the orginal exception message and no inner exception.
Related Issue
Closes #1870
Motivation and Context
Improve exception information.
Requires Documentation Change
No.
How Has This Been Tested?
Simple algorithm from issue #1870.
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Non-functional change (xml comments/documentation/etc)

Checklist:

[x] My code follows the code style of this project.
[x] I have read the CONTRIBUTING document.
[x] All new and existing tests passed.
[x] My branch follows the naming convention bug-<issue#>-<description or feature-<issue#>-<description>

Nice work


corrected indentation with filter

Description
macro is not passing all current variables to the include'd template, so metadata dict was always undefined and job definition was missing initContainers section. Replaced that with regular include with filter to indent properly
I could see it includes metadata.yml on both server and client pod definition now,
...
  initContainers:
  - args:
    - |
      python3 stockpile-wrapper.py -s=https://search-perfscale-dev-chmf5l4sh66lvxbnadi4bznl3a.us-west-2.es.amazonaws.com:443 -u=13a31b07-4c90-535f-90cd-37c8bda42c8b -n=${my_node_name} -N=${my_pod_name} --redisip=<IP_ADDRESS> --redisport=6379 --tags=common,k8s,openshift
    command:
    - /bin/sh
    - -c
...

Fixes

@dry923 PTAL, it works for me.


Web- Settings -Back button in the workspace working incorrectly when the page before is room detail

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Create room
Click room avatar to open room detail
Click workspace name in the room detail
Click back button

Expected Result:
When the previous page is room details it must return to this page.
Actual Result:
When the previous page is the room details, but after pressing back, it returns to the list workspaces page
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[x] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number: <IP_ADDRESS>
Reproducible in staging?: yes
Reproducible in production?: yes
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://github.com/Expensify/App/assets/93399543/c421b30e-755e-43b5-b977-9e4ccb174af8
https://github.com/Expensify/App/assets/93399543/d1b201a9-3bb9-41e7-b7c7-0429804c441e
Expensify/Expensify Issue URL:
Issue reported by: @namhihi237
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1685584637543339
View all open jobs on GitHub

Proposal
Please re-state the problem that we are trying to solve in this issue.
When the previous page is room details it must return to this page.
What is the root cause of that problem?
For now, the back button passes props  onBackButtonPress={() => Navigation.navigate(ROUTES.SETTINGS_WORKSPACES)}.
https://github.com/Expensify/App/blob/0813e2ca1476635a8324cbd9f998424663a33128/src/pages/workspace/WorkspaceInitialPage.js#L165-L179
What changes do you think we should make in order to solve the problem?
We use Navigation.goBack() to make sure the back button always back the previous page
 <FullPageNotFoundView
        shouldShow={_.isEmpty(policy)}
         onBackButtonPress={() => Navigation.goBack()} //=> update this line
 >
       <HeaderWithCloseButton
	    title={props.translate(‘workspace.common.workspace’)}
	    shouldShowBackButton
	    onBackButtonPress={() =>  Navigation.goBack()} // update this line
	    onCloseButtonPress={() =>  Navigation.dismissModal()}
	    shouldShowThreeDotsButton
	    shouldShowGetAssistanceButton
	    guidesCallTaskID={CONST.GUIDES_CALL_TASK_IDS.WORKSPACE_INITIAL}
	   threeDotsMenuItems={threeDotsMenuItems}
	   threeDotsAnchorPosition={styles.threeDotsPopoverOffset(props.windowWidth)}
    />

What alternative solutions did you explore? (Optional)
N/A

OMG, The proposal comes so fast

Contributor details
Your Expensify account email<EMAIL_ADDRESS>Upwork Profile Link: https://www.upwork.com/freelancers/~016f52810f2406ff59

This was intentional  for full context - https://github.com/Expensify/App/issues/18895#issuecomment-1549764035

@alitoshmatov - quick question! are you saying here that this issue is a feature and not a bug? just double-checking before i close this one out.

It is not definately a feature. We just had a chance to address this issue and possibly fix it, but as you can see here they didn't want it


Docs for ephemeral production pipelines running (incrementally) with no Postgres backend

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/2788
Originally created by @aaronsteers on 2021-06-08 16:10:06

In creating the pipeline for the Hub, I did not find docs for how to productionalize a pipeline while not requiring a postgres database. I was thinking of adding a short guide in Tutorials or in Productionalization.
Here are the related resources I found:

How to use the state extra: https://meltano.com/docs/plugins.html#state-extra
--dump option with example: https://meltano.com/docs/command-line-interface.html#examples

I think the process we want for the Gitlab pipeline is:

Load state.json from external storage (in our case, either S3 or a Gitlab CI artifact store.)
Ensure the state extra is configured.
To confirm: during execution, state.json should hopefully be continually updated by meltano. (I think we do not need the --dump option, in this case.) During load, state is incremented updated in the system db (only).

State must be exported by running a second invocation with the --dump=state CLI option.

Catch failures and upload the new state regardless, so that incremental processing will continue on the next run even if the operation is interrupted.
Still raise a failure (return non-zero) if the pipeline fails, so the job status reflects the execution result.

View 6 previous comments from the original issue on GitLab


Add on month select

[x] I have changed my target branch to develop :facepunch:

Description
Hi! I added a callback to inform when the current month is changed (in my application I'm fetching an API to update the availability for that month based on recurrence rules). I tried to write tests but they wouldn't pass because the context is breaking, so I'm unsure if I should cover the other changes with tests or if should I wait for the suite to be fixed in the develop branch.

Closed due to inactivity


FTB Modpack updates

Is this capable of downloading an FTB modpack update?
I started with FTB Direwolf 20 1.20 (ID 119) version 11441.
I see that the pack is now up to version 11469.
Is it just a simple job of giving the settings.json file the new version ID, or is there something else I need to do?

Yes, to download a newer version you would change the version id, and run the script again. All the following steps such as creating a new profile in your launcher are the same. The only difference being that after installing the pack, you would have to copy your saves folder and other stateful files like options.txt, resource packs, backups, etc. from your previous profile to the new one, if you care enough about them.