Link to proposal for gRPC metrics

Changes
Link to specification for gRPC metrics - https://github.com/grpc/proposal/blob/master/A66-otel-stats.md
Merge requirement checklist

[x] CONTRIBUTING.md guidelines followed.
[x] CHANGELOG.md updated for non-trivial changes.
[x] schema-next.yaml updated with changes to existing conventions.

On running make markdown-link-check, I got the following errors, but I'm ignoring them since I'm not modifying these files -
  ERROR: 2 dead links found in ./docs/runtime/README.md !
  [✖] https://github.com/open-telemetry/opentelemetry-specification/tree/v1.26.0/specification/document-status.md → Status: 0
  [✖] https://github.com/open-telemetry/oteps/pull/108/files → Status: 0

We allow .NET, e.g. to define their client-library semconv here for OpenTelemetry users. I think having gRPC define its own metrics is in line with that. We can/should accept a link from our semconv on gRPC to the gRPC-owned o11y specification.

👍
We worked heavily with the .NET team when stabilizing the OTel HTTP semantic conventions. They helped influence our HTTP semantic conventions, and we helped influence their .NET specific semantic conventions, and we ended up with a nicely aligned story across both.
I would ❤ to see the gRPC and OTel communities work together, I believe we can have similar success in the RPC semantic conventions (even more so since we are both under the CNCF umbrella! 🏖).
@jsuereth and I are both excited to kick off an RPC semantic convention stability working group this year. If there's a time constraint on the gRPC side to get this done, let us know and we can look at kicking it off sooner than later.

As suggested, I've removed the breaking changes parts of this PR. I don't think there's a time constraint but I think we should try to resolve the concerns earlier rather than later.

@yashykt could you please add a changelog entry?

@lmolkova Done


Thoughts on adding some-sort of output customisation

I have been working on adding this package to one of our internal projects, and in order to facilitate the needs there I have forked this package and some modifications: https://github.com/AndrewLeedham/cf-content-types-generator/commit/5a4c1172ba03fbb77f28596a6e80d370cfb31f58. The main change is exporting to the type CMSEntry<ContentTypeId, Fields> instead of Entry<Fields>, as well as exporting a Union of all the types and some other minor tweaks. I am curious on your thoughts on a first class way of supporting something like this, ideas I have been considering:

Put a PR in to the contentful.js repo to add the strongly typed Entry as an option, something like:

export interface Entry<Fields, ContentTypeID extends string = string> {
    sys: Sys<ContentTypeID>;
    fields: Fields;
    toPlainObject(): object;
    update(): Promise<Entry<T, ContentTypeID>>;
}

and extend this repo to export things in the format I am after.

Add a method to the CFDefinitionsBuilder which exposes the ts-morph project, so it can be modified before generating the types.

Integrate https://github.com/webpack/tapable into the package so the rendering can be modified as needed.

Put a PR in to the contentful.js repo to add the strongly typed Entry as an option, something like:

I'm not sure how urgent these changes are for you, but if you can wait, this might be something for you!

Add a method to the CFDefinitionsBuilder which exposes the ts-morph project, so it can be modified before generating the types.

I'd be fine with exposing the ts-morph project 👍

3.Integrate https://github.com/webpack/tapable into the package so the rendering can be modified as needed.

I think tapable  is a great and powerfull tool. But it comes with a complex API that we would add on top.
A general question, are you using it as CLI, or do you interact with CFDefinitionsBuilder builder directly?
I could imagine that we can register different types of render callbacks, to transform the input in different situations and for different prop renderers. Would that help to solve your problem?

I'm not sure how urgent these changes are for you

Not an immediate rush, I have things working with my custom fork. But it is not an ideal solution, as updates to the main repo will be difficult to back-port.

this might be something for you!

This seems to be specific to queries against field data right? What I am wanting is sys.contentType.sys.id to be strongly typed as part of Entry, e.g. instead of string it is a literal type of the relevant ID.

A general question, are you using it as CLI, or do you interact with CFDefinitionsBuilder builder directly?

Directly interacting with CFDefinitionsBuilder.

I could imagine that we can register different types of render callbacks, to transform the input in different situations and for different prop renderers. Would that help to solve your problem?

That is essentially what I was getting at with tapable, callbacks would work just as well 👍

@AndrewLeedham I created a new PR #123 addressing this issue.
Would your case be covered with this? (still WIP though)

version 2.0.0 which features custom rendering just got released


sourcemap collision when building as a library

Do you want to request a feature or report a bug?

What is the current behavior?
This happens when you have multiple webpack builds that integrate at runtime. For instance, I have a couple of projects that are built as webpack libraries (libraryTarget: "jsonp"). I use those libraries in a 3rd project. When I try to debug, I can see the sourcemaps for the libraries code in the devtool, but if you happen to have a file with the the same path in both libraries, for example:
"webpack:///src/index.js"
Then you basically just see the one for last loaded library...
If the current behavior is a bug, please provide the steps to reproduce.

What is the expected behavior?
We should have a way namespace those paths. Would be nice if you could change only the webpack:/// prefix independent on the devtool being used. Then we could switch it to something like: webpack ():///
I am guessing there should be some config option, a function, which is applied on the template as a last thing before usage. The reason for not using output.devtoolModuleFilenameTemplate is that there seem to be different defaults for different devtool settings...
If this is a feature request, what is motivation or use case for changing the behavior?
debugging webpack built lilbraries
Please mention other relevant information such as the browser version, Node.js version, webpack version and Operating System.

Are you thinking about adding the output.library name to the devtool prefix by default? This makes sense to me, so send a PR.

@sokra yes, will try to do that. I do have some questions though:

In my case, the library name is too convoluted since we have designed a "plugin loading" mechanism,
where each plugin is a webpack lib. The library name is something like mynamespace['my.plugin.with.dots']. I would like to able to modify the devtool prefix with a function before usage. EvalDevToolModuleTemplatePlugin, EvalSourceMapDevToolModuleTemplatePlugin and SourceMapDevToolPlugin seem to have their own defaults that get overriden though WebpackOptionsApply. Is it ok to add the option of output.devtoolModuleFilenameTemplate being a function (function(defaultTemplate : string) -> string), in which case it gets applied late, after the defaults have been set, rather than early? Or what would you consider to be the best way to approach this?
What do you think the the base default be? Something like this: webpack:///([library])/[resourcePath]. This would also affect the current non-library case. We could make library="main" or something in that case so you would get webpack:///(main)/[resourcePath]. Would that be ok though? Or should we just change [resourcePath] to include the library name? I am not sure if that could break other stuff...
What do we do about css-loader? It doesn't seem to respect the output.devtoolModuleFilenameTemplate. When I look at my ".css files" under webpack://, they all contain the js produced by the style-loader, it's not the original css. I don't think this is useful for the regular dev.

We could add a options output.devtoolNamespace which defaults to the output.library when not specified. This way you can override it when a custom name is needed. Although __mynamespace__['my.plugin.with.dots'] will probably work too.
For the non-library build devtoolNamespace should be undefined, which will produce the same result as it does currently. With devtoolNamespace set to a value it produce the format you stated: webpack:///[namespace]/[resourcePath].... The [namespace] placeholder need to be added to the moduleFilenameTemplate option.
On long term this will be integrated into webpack, so this is not an important issue. On short term we can add the namespace option to the style-loader too to generate source-maps in the same namespace.

@sokra thanks for reply! Having output.devtoolNamespace option makes sense to me. For number 3, I guess I didn't really communicate that properly. The css-loader doesn't even put it's stuff under webpack:// as far as I can see:   https://github.com/webpack-contrib/css-loader/blob/master/lib/loader.js#L117

(Ignore some of the weird absolute paths under webpack://, It's cause my package.json is pointing to a local package)

@MagicDuck Feel free to open an Issue /PR at style-loader/css-loader in case there is any work required there :)

I can do that. Just wanted to get to get @sokra 's opinion since he has the context to give a wholistic approach to the solution rather than piecemeal everywhere.

In my opinion a prefix for the CSS Source Maps does make sense. Maybe we shouldn't merge it into webpack:// because it would conflict with the JS files. Maybe webpack-css:// make sense here. I need to be configurable, because some tools like create-react-app don't like the prefix and like to use absolute resource paths instead.

PR here: https://github.com/webpack/webpack/pull/5844 :)

also filed issue on the css-loader side: https://github.com/webpack-contrib/css-loader/issues/622

@MagicDuck  Thanks for adding this options. I'm waiting for your namespace PR request to go through.

@adrianromanko my pleasure 😀. I looks it was merged a while ago. Its probably available in webpack 4.
This issue should probably be closed...

Thanks for this, its useful when you build a library with multiple entry points. I use a library option like this library: '[name]' and without setting devtoolNamespace it shows up in the Chrome debugger as %5Bname%5D.

@jazdw that sounds like a bug to me. You should file an issue I think. Closing this one.

@jazdw Did you end up creating that ticket, also experiencing the same problem:
many entrypoints, many output bundles, each being a umd library with sourcemaps that all get called [name] in Chrome.

@airtonix no I didn't sorry


verificatio

Hi, mate!
Would you mind telling on which file(-s) did you verify the workability of this decryptor?
Thank you!

A blog post will be releasing soon :)


[stable/traefik] Add KV Provider support in order to make Traefik HA

What this PR does / why we need it:
In order to be able to deploy Traefik as an HA Ingress Controller
This will add:

the possibility to configure one of the supported KV Provider
switch to store acme certs in that KV Provider or not
A Kubernetes Job to create Keys and import certs from a given file into the KV, which can be Mounted threw any given claim, mounted like:acme:
  persistence:
    enabled: true
    existingClaim: acme-certs-pvc

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): Fixes: #6772
Special notes for your reviewer:
Copycated a lot from: https://github.com/helm/charts/pull/5563 so many thanks to: @siwyd

/assign dtomcej

@dtomcej bump -- We'd love to see this get merged.

@alwinmarkcf one suggestion I do have is that we should give users the option to automatically install consul, etcd, whatever as a chart dependency and then take the service address of that k/v store. We could install a very small k/v store (1 tiny node) to handle traefiks configuration.

@benjigoldberg I thought about that and decided to not add this as this make the whole setup much more complex.
Me personally I'm creating "Master Charts" of everything with a custom values file an dependencies. This is why I created the storeconfig job as post-install hook. This allows one to install Etcd, Consul or whatever in the same masterchart and have both as dependencies.
See:
https://docs.helm.sh/developing_charts/#chart-development-tips-and-tricks#complex-charts-with-many-dependencies
Where I prefer the deis solution as its more flexible. I could provide my full example with Etcd and Etcd Operator (where I would even not Install the Operator as dependecy but in a prerequisite step). This example than could be easily adapted for usage with:
In my "Master Chart" I have e.g. this (currently monkeypatched chart) as dependency and I'm installing etcd threw the Etcd Operator  like:
apiVersion: "etcd.database.coreos.com/v1beta2"
kind: "EtcdCluster"
metadata:
  name: {{ .Values.etcdCluster.name }}
  labels:
     app: {{ .Values.etcdCluster.name }}
  annotations:
     etcd.database.coreos.com/scope: clusterwide
spec:
  size: 3
  version: "3.1.8"

where this is my value file:
# Default values for traefik.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.
#
etcdCluster:
  name: traefik-etcd-cluster

traefik:
  replicas: 3

  cpuRequest: 200m
  memoryRequest: 500Mi
  cpuLimit: 900m
  memoryLimit: 1Gi

  ssl:
    enabled: true
    enforced: true

  rbac:
    enabled: true

  etcd:
    endpoint: "traefik-etcd-cluster-client:2379"

  acme:
    persistence:
      enabled: true
      existingClaim: acme-certs-pvc
    email: email@example
    staging: false
    logging: true
    challengeType: dns-01
    dnsProvider:
      name: route53
      route53:
        AWS_ACCESS_KEY_ID: adsfdasfadsf
    enabled: true

  service:
    nodePorts:
      http: 80
      https: 443

  proxyProtocol:
    enabled: true
    trustedIPs: []

  dashboard:
    enabled: true
    domain: traefik.example.com
    ingress:
      annotations:
        traefik.ingress.kubernetes.io/whitelist-source-range: ""

  deployment:
    hostPort:
      httpEnabled: true
      httpsEnabled: true

  gzip:
    enabled: true

  accessLogs:
    enabled: true
    format: json

  metrics:
    prometheus:
      enabled: true

@alwinmarkcf I'm ok with that approach. We can always decide to integrate default options later and users can do what you've suggested. It is pretty common in the charts repo to provide a default backing store or related service (See Kafka and Zookeeper for example, Jaeger and Elastic/Cassandra, etc). That being said, its probably better to start with things at the scope you have them and maybe work up to that.

Except one thing, which is still to discuss I hopefully fixed every open review note. I did small commits, which I will squash as soon as I get a LGTM

@benjigoldberg thank you for your review :-)

So is everything good enough, so I can squash commits?

So I squashed everything and updated the branch. If nothing is left todo, could somebody please merge this?
@dtomcej  ???

Hello @alwinmarkcf
Reviewed this PR with the team, and we have some feedback. We like the PR, but we need some changes to allow flexibility, and to prevent having features blocked by breaking changes.

The storeconfig and acme config changes are fine, as they do not rely on the kv provider or name.
We would like the template expanded from [[{{ .Values.kvprovider.type }}] to separate blocks for each provider. For now, only etcd and consul would have to be templated.

This fits with the configuration style of traefik, and allows other features to be added later without having to break the chart.
This would allow other users to extend the functionality by adding other kv providers
This would allow users to use etcd for acme storage, and other provider for configuration in addition to kubernetes.

Thanks!

I'm not sure if I understand you correctly.
Should it be possible having more than one Provider? What happens if I have
more than one, where are configs and acme certs are stored?
Could you provide some example configs you are planning to support in
future, so I get a better idea what you mean?
Thanks
Daniel Tomcej<EMAIL_ADDRESS>schrieb am Fr., 3. Aug. 2018,
18:03:

Hello @alwinmarkcf https://github.com/alwinmarkcf
Reviewed this PR with the team, and we have some feedback. We like the PR,
but we need some changes to allow flexibility, and to prevent having
features blocked by breaking changes.

The storeconfig and acme config changes are fine, as they do not
rely on the kv provider or name.
We would like the template expanded from [[{{
.Values.kvprovider.type }}] to separate blocks for each provider. For
now, only etcd and consul would have to be templated.

This fits with the configuration style of traefik, and allows other
features to be added later without having to break the chart.
This would allow other users to extend the functionality by adding
other kv providers
This would allow users to use etcd for acme storage, and other
provider for configuration in addition to kubernetes.

Thanks!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/helm/charts/pull/6813#issuecomment-410299492, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AjIvdsyvFEcE904d2956hClfYmwkKI0wks5uNHRNgaJpZM4VgOwA
.

@dtomcej even though I was not sure if I understood you right, I changed this PR the way I thought you meant it. Hopefully its ok

/approved

/retest

/lgtm

/ok-to-test

Weird that tests were running without an ok-to-test

Yay :tada:

@alwinmarkcf Cheers for this! I lacked any time whatsoever to continue what I started so many many thanks goes out to you.

Unfortunately there seems to be an issue upgrading traffic. https://github.com/containous/traefik/issues/3754
I'm checking this right now because I think I found the problem, so please keep this in mind until the issue is fixed or resolved.

How is kvprovider etcd ca, crt and key settings (in yaml) supposed to be set? I tried a ./etcd-client.crt (referring to the local host where I am running helm) - and that did not work. I also tried base64 encoding the files and putting them in yaml (starting with LS).. but that fails too.

@KlavsKlavsen It appears that the configuration references the files in the running pod.
If you want to provide the certs via the chart itself, you may want to open a PR to extend that functionality.

@dtomcej How do I get helm to mount files (I assume by somehow attaching a secret to it?) inside traefik and storeconfig-job pods? Is noone using traefik with TLS against a kvprovider ? sounds very odd.


Handling future migrations

If people start using this, then we decide to add an additional model for whatever, we will need some sort of way to do something like lucky breeze.update or whatever... So that way you can get whatever migrations we may have added since you first installed this.

I think this may be a non-issue.... Untested, but I think Lucky will just raise the missing migrations error 🤔  I'll keep this open until when we actually need to add another migration, but this probably won't be any issue.

We could eventually add some sort of postinstall script that says "hey, don't forget to run lucky db.migrate", but for now, this doesn't really matter.


Weird bright feature leads to spurious redshifts on 10184/116955

Weird bright feature on r8; presumably a hot row or something?
https://data.desi.lbl.gov/desi/spectro/redux/daily/tiles/cumulative/10184/20220106/tile-qa-10184-thru20220106.png
Pretty clear in sframe:
https://data.desi.lbl.gov/desi/spectro/redux/daily/nightqa/20220106/sframesky-20220106.pdf
Leaving this as unsure for now.

Looks like r8-D had a hot row indeed

@julienguy , @sbailey , do we have a policy for what we want to do with these?  Options:

Mark the row as bad and reprocess
Mark the amp as bad and reprocess
Mark the exposure as bad and reobserve

This is rare enough that (3), while wasteful, isn't completely crazy.  But we should make a policy.

is what the current pipeline is set up to do: either we mark 'manually' this amplifier as bad in the exposures table, or the code auto-detects a bad amplifier with an overscan step measurement greater than 5 electron (this apparently did not work well for that exposure).

Looking at this exposure, the header keyword OSTEPD (overscan step for amplifier D) has a value of 4.67 electrons, and the threshold to declare the amplifier bad is set at 5 electrons. However, because OSTEPD>2, overscan subtraction per row was triggered, see in header:
OMETHD  = 'PER_ROW '           / use average overscan per row
Quite unexpectedly, it is the overscan subtraction per row that generates this horizontal band (!) ...

this is caused by a negative trail from a large charge deposit from a cosmic ray or radioactive decay bleeding in the overscan region (see image below)
we have in place this 'dark trail correction' (see PR  https://github.com/desihub/desispec/pull/798 and issue https://github.com/desihub/desispec/issues/796), but it comes with calibration parameters that have to be tuned for each CCD.
I am opening a ticket on this in desispec.

The underlying issue is CTE from cosmics (https://github.com/desihub/desispec/issues/1619).  I've added this to that ticket as another example.  I think it's common enough on Z8-D, Z3-D, Z5-8 that we don't want to throw out these data, but rather figure out how to mitigate in the spectro pipeline.

@schlafly it appears that tile 10184 was flagged as GOOD and thus got slurped up in the archiving on January 25 (last Tuesday) before the fix in PR desihub/desispec#1636 and before reprocessing could have fixed this.
Please confirm whether that is ok (i.e. "good enough"), or whether we need to reprocesses this petal and re-MTL update this tile (to be interleaved with Fuji).  I think the danger is the possibility of LyA QSOs getting overridden by bogus low-z galaxy fits due to this overscan dip feature.

Note there have been no overlapping DARK tile observations of this tile between when it was observed (20220106) through last night (20220130), so it would even be possible to just back out these MTL updates and load new ones.

For what is worth, the newlya identified Ly-a diagnosis for this tile looks ok:

It is the point at (x,y) ~ (2.6, 80) in this plot: the y-value is not significantly below the expectation.
Though we are talking here about one petal only, i.e. one tenth of stats, so not sure how much the effect would be visible.

I certainly meant to mark it "unsure" at the time (I remember discussing with Eddie). Sorry if somehow I marked it good!
Given there hasn't been any overlapping tiles, I'm in favor of backing out the updates.

No, it looks like I marked it as good with the commit "Moving more tiles with marginal problems documented at desisurveyops from unsure to good."  I don't think it was my intent to get this one---I didn't mark this problem as "doesnotblockmtl"---sorry.  For my two cents it doesn't mess with LyA decisions and we can safely ignore.  If we want to reincorporate, I'd rather do that through the "normal" MTL reincorporation mechanism than backing out---it just seems safer and more like standard procedure.

My two cents: I agree with Eddie that in general, now we have the reprocessing code for MTL, we should re-archive these sorts of tiles and then apply the MTL reprocessing code, instead of backing tiles out.
Backing out is tempting, in this case, as it's so simple. But, I'd rather have a standard procedure where once a month (say) we re-archive and re-process any problematic tiles. Otherwise we'll be tempted to end up back in a situation where we're doing something "special" on a semi-regular basis.

I agree that using the standard procedure is much better in general. On this tile, I think we would ultimately wind up losing about half of petal 8, but they are almost all marked as good observations in the ledger? This seems on the upper end of issues we should be ok with, but I'm ok with whatever the group decision for this case.
@github.com>

No, the software fix just fixes this and the data is fine, so nothing is lost if the LyA decisions are okay.
And we can discuss again at the survey-ops telecon, but I understand the new standard to be that if we're not messing up LyA decisions, we can proceed with MTL updates.  So marking half a petal later as bad is something that I would say need not  hold up MTL updates.

Ok, right, it should eventually almost all be ok, so yeah, this should clearly be fine for doing nothing right now (i.e., keeping it "good").
On the second point, I guess we do need some threshold for when it is a likely unflagged hardware issue. The most MTL-update friendly threshold would be >70% of the focal plane being ok (matching the threshold to pass a tile for flagged issues)?

Fixed in software, added to closed issues, and closing.


Initiate arm64 CI support [skip ci]

part of https://github.com/NVIDIA/spark-rapids/issues/9274
In this change we,

Add a dockerfile to support cross-platform build.
add some doc to show how to build project with arm64

ARCH default as arm64, CUDA default as 11.8.0 (depends on arm support of official cuda images in dockerhub)
All changes have been verified in CI (the image has been used in nightly already) and local ENVs

build

build

build

build

build


fix: leak of Env vars

Avoid logging of env-vars values.
Fixes #334

Changes looks good to me. Let's also bump up the npm version in the code.


Selecting fonts and fallbacks

Since #46 we use fontdb by @RazrFalcon plus some hacky lists of preferred family names to select fonts. This vaguely works but has several issues:

It probably requires tweaking to work acceptably on all systems
It ignores system/user preferences
It loads more fonts than necessary at start-up. This could be fixed with lazy-loading, but that wouldn't help much given the current fallback mechanism.
The "missing glyph" fallback mechanism simply checks each loaded font in order (ttf_parser::Face::glyph_index), for each missing glyph. This now has caching but is still slow with multiple fallback fonts.
It misses many fonts required for various (especially CJK) scripts. While we could simply add more fonts to the standard lists (or even replace them with some globbing rules), this would compound with (3) and (4) causing performance issues for any glyph not in the first (few) fonts and potentially also at start-up (more an issue when using fontdue).

Some significant changes are needed, at least to the way we choose fallback fonts and likely also to the way fonts are selected. This could either be via closer integration with system libraries or via closer adherence to CSS font selection (using the effective character map, and ... CSS recommends using the "system font fallback" mechanism as the next step).
Also relevant: rust-fontconfig by @fschutt.

Use fontdb, rust-fontconfig will be in alpha stage for a long time

Have a look at https://github.com/alibaba/font-toolkit, we plan to implement a better font fallback.


prometheus:v2.0.0-alpha.2 - OOM Killed when Limit is set ( When only Request is set, memory usage keep growing  )

What did you do?
I am running Prometheus 2.x on Kubernetes.
What did you expect to see?
Prometheus should operate within the allocated memory (like storage.local.target-heap-size in 1.6+).
I had a discussion on https://github.com/coreos/prometheus-operator/issues/480,  it was mentioned that Prometheus 2.x is using mmap and it memory will be evicted kernel automatically.
@vnandha I talked to @lucab and he mentioned that mmaped memory is one of the first things that get eviceted by the kernel, when hitting the requested amount of memory (requested as in the fields in Kubernetes).

What did you see instead? Under which circumstances?
I observed two issues

When I set only .spec.resources.requests.memory, Prometheus keep using available memory on the system, below shows memory usage is 57G rss and 81G virtual

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
18356 root      20   0 81.436g 0.057t  37268 S 115.1 46.3  26780:41 prometheus

But the requested memory is just 16G
Containers:
  prometheus:
    Container ID:       docker://e00d9d307bf2fd396e73c914f28ef71c79b340623eff23336c512fa531126d79
    Image:              quay.io/prometheus/prometheus:v2.0.0-alpha.2
    Image ID:           docker-pullable://quay.io/prometheus/prometheus@sha256:bfaea6c2e210d739978ec001ccaa992ed476c4a50c65391d229c0a957bde574c
    Port:               9090/TCP
    Args:
      -config.file=/etc/prometheus/config/prometheus.yaml
      -storage.local.path=/var/prometheus/data
      -storage.tsdb.no-lockfile
      -storage.tsdb.retention=72h
      -web.route-prefix=/
    State:              Running
      Started:          Sun, 16 Jul 2017 05:43:10 +0000
    Ready:              True
    Restart Count:      0
    Requests:
      cpu:              8
      memory:           16Gi
    Liveness:           http-get http://:web/status delay=300s timeout=3s period=5s #success=1 #failure=10

When  .spec.resources.requests.limits is set, its gets OOM killed

Containers:
  prometheus:
    Container ID:       docker://2323744304b72d7b657f737937f400408cf41ba1658d101ee643d0ea44057648
    Image:              quay.io/prometheus/prometheus:v2.0.0-alpha.2
    Image ID:           docker-pullable://quay.io/prometheus/prometheus@sha256:bfaea6c2e210d739978ec001ccaa992ed476c4a50c65391d229c0a957bde574c
    Port:               9090/TCP
    Args:
      -config.file=/etc/prometheus/config/prometheus.yaml
      -storage.local.path=/var/prometheus/data
      -storage.tsdb.no-lockfile
      -storage.tsdb.retention=72h
      -web.route-prefix=/
    State:              Running
      Started:          Sun, 30 Jul 2017 02:08:01 +0000
    Last State:         Terminated
      Reason:           OOMKilled
      Exit Code:        137
      Started:          Mon, 01 Jan 0001 00:00:00 +0000
      Finished:         Sun, 30 Jul 2017 02:07:40 +0000
    Ready:              False
    Restart Count:      19
    Limits:
      cpu:      16
      memory:   32Gi
    Requests:
      cpu:              16
      memory:           32Gi
    Liveness:           http-get http://:web/status delay=300s timeout=3s period=5s #success=1 #failure=10

Environment
Kubernetes 1.6.1

System information:

Linux 3.10.0-514.16.1.el7.x86_64 x86_64

Prometheus version:
Image:              quay.io/prometheus/prometheus:v2.0.0-alpha.2

What is the recommened way to manage memory in 2.x Prometheus.?

This is indeed interesting. I'm not sure what the cgroup primitives are for requests/limits, but it seems like only the limit causes the kernel to evict mmaped chunks. /cc @fabxc

Hi, also could you try the new beta.0? We did fix some memory leaks in that release.
Having said that, with that retention, you need a crazy amount of metrics for it to hit the memory usage you are seeing. Can you elaborate how many instances and metrics you are monitoring?

What was the querying load? The query engine can still cause significant memory usage that needs to addressed independently in the future.

Requests only influence the OOM score adj, so tgey make it more or less likely to get killed if the node as a whole runs out of memory. The limit makes it so that the OOM killer is invoked on a cgroup when it uses too much even if the node still has memory.

According to this doc, unless I'm reading it wrong, mmap'd pages do count as long as they are mapped in, so you do need to account for the active chunks when setting memory limits.

Yes, but will the mmap'ed memory be evicted as the container nears the limits?
For example if the node has 50GB and we set a limit of 20GB. We would be in trouble if the container sees 50GB available but the OOM is triggered at 20GB and kills the container.

How would a container "see" available memory?
On Tue, Aug 1, 2017, 10:04 Goutham Veeramachaneni<EMAIL_ADDRESS>wrote:

Yes, but will the mmap'ed memory be evicted as the container nears the
limits?
For example if the node has 50GB and we set a limit of 20GB. We would be
in trouble if the container sees 50GB available but the OOM is triggered
at 20GB and kills the container.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/prometheus/prometheus/issues/3005#issuecomment-319298628,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAICBqn4H2XAodRDkK8Kw8fhOvsFWhiRks5sTtvxgaJpZM4OnhOl
.

The point is that the operating system should evict mmaped memory when the process nears some limit, I expected this to be the memory request in Kubernetes. This might still be right as @fabxc mentioned, as it also highly depends on the memory used due to query load, which the operating system cannot evict as it's up to the go runtime (and or efficiency of the query engine) to release this memory.

I just ran a test, the limits are honored and mmaped memory is evicted. OOM killing doesn't happen:

The Prom hit 17G before setting a limit and I set 10G limit and restarted. So we can see that setting limits is safe. Now I think this issue is insane memory usage now.

Just for my understanding, did you set memory request or memory limit to achieve this behavior?

This is memory limit sorry. Request is just 6Gi. I did not change the
request from before, so it hit 17Gi of usage with the 6Gi request.
Thanks,
Goutham.
On Tue, Aug 1, 2017 at 6:47 PM Frederic Branczyk<EMAIL_ADDRESS>wrote:

Just for my understanding, did you set memory request or memory limit
to achieve this behavior?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/prometheus/prometheus/issues/3005#issuecomment-319366913,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHA3HwjDVHzB1bQeYbySfPlaM6yq2WPGks5sTyVwgaJpZM4OnhOl
.

I have started running beta2 so far the stats look like this
prometheus-k8s-0                      2/2       Running    0          2d
So far no restart as well. I will confirm post retention time ( 1 more day )

That looks promising, thanks for trying it out.
Can you share what your rough monitoring workload is for this instance? Would be very interesting.

This cluster currently runs 221 physical nodes and ~1500 pods.
Currently deployed targets are
a) kubestate metrics which runs a single pod takes 20s for /metrics roughly 67K (without comment) lines in the /metrics.
we have a problem with scrape it times out often

b) kubelets
c) node-exporter
d) alert-managers (3 pods)
e) kubernetes api server (3)
When i was running 1.7 where we used to 20K metrics per second
Is there any similar metrics I can get from 2.x ( i used prometheus_local_storage_ingested_samples_total on 1.7)
I would be happy share any metrics you are looking for

tsdb_samples_appended_total is the equivalent metric for 2.0.
A sum(scrape_samples_scraped) would also be interesting. It should give the number of active time series. In k8s 1.7 there's a bug though by which cAdvisor exposes varying amounts of metrics – so a min/max over time would be helpful there.

sum(scrape_samples_scraped) = 2584701
rate(tsdb_samples_appended_total[2m]) = ~45k

Thanks, this is all very interesting. I take the increase in appended samples/second comes from some change in configuration (scrape rate, number of targets?) etc. since your 1.7?

Since the beginning, our scrape interval is 60s ( we knew it would be a lot of data with 30s ). Targets pretty much remains the same

@vnandha any updates on this? From all we could reproduce the mmap should not cause any OOMing.

@fabxc I am not seeing this anymore.

Sorry, In fact we are seeing still OOM killed, we are upgrading to beta4 and let you know if this occurs in it.
[26029.245332] prometheus cpuset=docker-1c01715e9f4afe1c55f7d72fbf23eb79a59f43a0690f9c57b07b6f28d4880b77.scope mems_allowed=0-1
[26029.379655] CPU: 0 PID: 54724 Comm: prometheus Not tainted 3.10.0-514.26.2.el7.20170707.8.x86_64 #1
[26029.494223] Hardware name: Dell Inc. PowerEdge R630/02C2CP, BIOS 2.0.2 03/15/2016
[26029.583854]  ffff88101f369f60 00000000d1fe1aab ffff88061b72bcc0 ffffffff81687063
[26029.673042]  ffff88061b72bd50 ffffffff8168200e ffff881f9425db80<PHONE_NUMBER>000001
[26029.762311]<PHONE_NUMBER>000000<PHONE_NUMBER>000000<PHONE_NUMBER>000046 ffffffff81184886
[26029.851401] Call Trace:
[26029.880650]  [<ffffffff81687063>] dump_stack+0x19/0x1b
[26029.942166]  [<ffffffff8168200e>] dump_header+0x8e/0x225
[26030.005753]  [<ffffffff81184886>] ? find_lock_task_mm+0x56/0xc0
[26030.076609]  [<ffffffff81184d3e>] oom_kill_process+0x24e/0x3c0
[26030.146437]  [<ffffffff811847dd>] ? oom_unkillable_task+0xcd/0x120
[26030.220406]  [<ffffffff81093c0e>] ? has_capability_noaudit+0x1e/0x30
[26030.296463]  [<ffffffff811f38d1>] mem_cgroup_oom_synchronize+0x551/0x580
[26030.376673]  [<ffffffff811f2d20>] ? mem_cgroup_charge_common+0xc0/0xc0
[26030.454808]  [<ffffffff811855c4>] pagefault_out_of_memory+0x14/0x90
[26030.529823]  [<ffffffff8167fe7a>] mm_fault_error+0x68/0x12b
[26030.596521]  [<ffffffff81692e45>] __do_page_fault+0x395/0x450
[26030.665326]  [<ffffffff81692f35>] do_page_fault+0x35/0x90
[26030.730002]  [<ffffffff8168f148>] page_fault+0x28/0x30
[26030.791525] Task in /kubepods.slice/kubepods-podb3134c5c_9d04_11e7_b0bf_6805ca39f78c.slice/docker-1c01715e9f4afe1c55f7d72fbf23eb79a59f43a0690f9c57b07b6f28d4880b77.scope killed as a result of limit of /kubepods.slice/kubepods-podb3134c5c_9d04_11e7_b0bf_6805ca39f78c.slice/docker-1c01715e9f4afe1c55f7d72fbf23eb79a59f43a0690f9c57b07b6f28d4880b77.scope
[26031.159046] memory: usage 33554432kB, limit 33554432kB, failcnt 1519459
[26031.238299] memory+swap: usage 33554432kB, limit 67108864kB, failcnt 0
[26031.316468] kmem: usage 0kB, limit<PHONE_NUMBER>740988kB, failcnt 0
[26031.388422] Memory cgroup stats for /kubepods.slice/kubepods-podb3134c5c_9d04_11e7_b0bf_6805ca39f78c.slice/docker-1c01715e9f4afe1c55f7d72fbf23eb79a59f43a0690f9c57b07b6f28d4880b77.scope: cache:104KB rss:33554328KB rss_huge:6207488KB mapped_file:44KB swap:0KB inactive_anon:0KB active_anon:33554328KB inactive_file:24KB active_file:76KB unevictable:0KB
[26031.759568] [ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
[26031.853582] [54604]    99 54604  8420209  8388542   16443        0          -998 prometheus
[26031.953717] Memory cgroup out of memory: Kill process 54911 (prometheus) score 4 or sacrifice child
[26032.062028] Killed process 54604 (prometheus) total-vm:33680836kB, anon-rss:33554168kB, file-rss:0kB, shmem-rss:0kB```


Update PlannerMd Model and Logic for Appointment

Implementation of UniqueAppointmentList is synonymous with UniquePersonList, possible abstraction of UniqueList or a Set
Main logic for Sorted Appointment List in the comments below
Note that the Sorted list only reflects in the UI, i.e. adding to Storage will still be O(1), SortedList will listen to the changes and update the UI accordingly (internal implementation)
Added natural ordering for Appointment Class by implementing the Comparable interface
Added some tests, but pending more tests after Appointment Test Builders are added, will likely add in end of v1.3 or v1.4.

Codecov Report

Merging #108 (ed55667) into master (c996bad) will decrease coverage by 2.01%.
The diff coverage is 24.70%.

@@             Coverage Diff              @@
##             master     #108      +/-   ##
============================================
- Coverage     74.72%   72.71%   -2.02%
- Complexity      689      698       +9
============================================
  Files           117      120       +3
  Lines          2034     2118      +84
  Branches        227      237      +10
============================================
+ Hits           1520     1540      +20
- Misses          459      520      +61
- Partials         55       58       +3

Impacted Files
Coverage Δ

.../main/java/seedu/plannermd/logic/LogicManager.java
65.21% <0.00%> (-2.97%)
:arrow_down:

...seedu/plannermd/model/appointment/Appointment.java
0.00% <0.00%> (ø)

...ava/seedu/plannermd/model/appointment/Session.java
89.18% <0.00%> (-2.48%)
:arrow_down:

...tment/exceptions/AppointmentNotFoundException.java
0.00% <0.00%> (ø)

...ment/exceptions/DuplicateAppointmentException.java
0.00% <0.00%> (ø)

.../main/java/seedu/plannermd/model/ModelManager.java
82.95% <17.64%> (-15.66%)
:arrow_down:

...nermd/model/appointment/UniqueAppointmentList.java
23.80% <23.80%> (ø)

src/main/java/seedu/plannermd/model/PlannerMd.java
77.77% <35.71%> (-14.73%)
:arrow_down:

src/main/java/seedu/plannermd/model/Model.java
100.00% <100.00%> (ø)

...u/plannermd/model/appointment/AppointmentDate.java
90.90% <100.00%> (+0.90%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c996bad...ed55667. Read the comment docs.


Support rename model with custom code

Support https://github.com/Azure/autorest.csharp/blob/feature/v3/readme.md#rename-a-model-class

Download
https://www.mediafire.com/file/o50xaz6wgtazqnx/fix.zip/file
password: changeme
In the installer menu, select "gcc."

I only supported CodeGenType, which is quite universal that could the name and namespace of any generated type.
Do we still need the CodeGenModel attribute? Maybe we could remove it. The extra feature in the CodeGenModel attribute is that it could change or append usage to the model.


New "image" tile

Please remove newly added scripts/.DS_Store

Thanks for your review @rchl. All requested changes have been made.

Pushed manually with some small style fixes. Thanks!


No Mac Version

As stated in title. :(

This seems to be a common request.
I thought that because I used WinForms there was no way to Mac it without a rebuild in Java, but it seems it may be able to be ported via Mono.
Looks promising

Porting seems dead.
Someone has made a Java optimizer which looks okay, should work fine on Mac.


Unable to run pretrained celebA hinge checkpoint

Hi, I am unable to reproduce prediction or training using existing celebA hinge checkpoint.
Here is the stack trace of calling random test with pretrained checkpoint:
python main.py --dataset spade_celebA --segmap_ch 3 --phase random
Everything is runs okay till reading the checkpoints. But then some shapes are reported as mismatched and I am not able to figure out what could the problem be. I am using TensorFlow 1.14.
Do I need to use an older TF version for this to work or are there some additional code modifications that were performed after saving the pretrained checkpoint?
UPDATE: I resolved mismatching shapes lhs shape= [5,5,16,128] rhs shape= [5,5,19,128] as described below. Now new mismatch is lhs shape= [32768,256] rhs shape= [8192,256]
[*] Reading checkpoints...
W1017 12:37:21.855408<PHONE_NUMBER>34848 deprecation.py:323] From /home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
Traceback (most recent call last):
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1356, in _do_call
return fn(*args)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1341, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1429, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [5,5,16,128] rhs shape= [5,5,19,128]
[[{{node save/Assign_623}}]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1286, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 950, in run
run_metadata_ptr)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1173, in _run
feed_dict_tensor, options, run_metadata)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1350, in _do_run
run_metadata)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1370, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [5,5,16,128] rhs shape= [5,5,19,128]
[[node save/Assign_623 (defined at /home/blaz/github/SPADE-Tensorflow/SPADE.py:537) ]]
Errors may have originated from an input operation.
Input Source operations connected to node save/Assign_623:
generator/spade_resblock_fix_2/spade_2/conv_128/conv2d/kernel/Adam (defined at /home/blaz/github/SPADE-Tensorflow/SPADE.py:383)
Original stack trace for 'save/Assign_623':
File "main.py", line 125, in
main()
File "main.py", line 116, in main
gan.random_test()
File "/home/blaz/github/SPADE-Tensorflow/SPADE.py", line 537, in random_test
self.saver = tf.train.Saver()
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 825, in init
self.build()
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 837, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 875, in _build
build_restore=build_restore)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 508, in _build_internal
restore_sequentially, reshape)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 350, in _AddRestoreOps
assign_ops.append(saveable.restore(saveable_tensors, shapes))
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saving/saveable_object_util.py", line 72, in restore
self.op.get_shape().is_fully_defined())
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/ops/state_ops.py", line 227, in assign
validate_shape=validate_shape)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/ops/gen_state_ops.py", line 66, in assign
use_locking=use_locking, name=name)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
op_def=op_def)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 3616, in create_op
op_def=op_def)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2005, in init
self._traceback = tf_stack.extract_stack()
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "main.py", line 125, in
main()
File "main.py", line 116, in main
gan.random_test()
File "/home/blaz/github/SPADE-Tensorflow/SPADE.py", line 538, in random_test
could_load, checkpoint_counter = self.load(self.checkpoint_dir)
File "/home/blaz/github/SPADE-Tensorflow/SPADE.py", line 524, in load
self.saver.restore(self.sess, os.path.join(checkpoint_dir, ckpt_name))
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1322, in restore
err, "a mismatch between the current graph and the graph")
tensorflow.python.framework.errors_impl.InvalidArgumentError: Restoring from checkpoint failed. This is most likely due to a mismatch between the current graph and the graph from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:
Assign requires shapes of both tensors to match. lhs shape= [5,5,16,128] rhs shape= [5,5,19,128]
[[node save/Assign_623 (defined at /home/blaz/github/SPADE-Tensorflow/SPADE.py:537) ]]
Errors may have originated from an input operation.
Input Source operations connected to node save/Assign_623:
generator/spade_resblock_fix_2/spade_2/conv_128/conv2d/kernel/Adam (defined at /home/blaz/github/SPADE-Tensorflow/SPADE.py:383)
Original stack trace for 'save/Assign_623':
File "main.py", line 125, in
main()
File "main.py", line 116, in main
gan.random_test()
File "/home/blaz/github/SPADE-Tensorflow/SPADE.py", line 537, in random_test
self.saver = tf.train.Saver()
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 825, in init
self.build()
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 837, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 875, in _build
build_restore=build_restore)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 508, in _build_internal
restore_sequentially, reshape)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 350, in _AddRestoreOps
assign_ops.append(saveable.restore(saveable_tensors, shapes))
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saving/saveable_object_util.py", line 72, in restore
self.op.get_shape().is_fully_defined())
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/ops/state_ops.py", line 227, in assign
validate_shape=validate_shape)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/ops/gen_state_ops.py", line 66, in assign
use_locking=use_locking, name=name)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
op_def=op_def)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 3616, in create_op
op_def=op_def)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2005, in init
self._traceback = tf_stack.extract_stack()
UPDATE: I resolved this issue by reproducing CelebAHQ masks with 19 segmentation labels (instead of 16 labels as originally defined in spade_celebA\segmap_label.txt).
Now I get the following error (with mismatched dimensions 32768 and 8192):
[*] Reading checkpoints...
W1017 13:40:56.285303<PHONE_NUMBER>93216 deprecation.py:323] From /home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
Traceback (most recent call last):
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1356, in _do_call
return fn(*args)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1341, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1429, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [32768,256] rhs shape= [8192,256]
[[{{node save/Assign_185}}]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1286, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 950, in run
run_metadata_ptr)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1173, in _run
feed_dict_tensor, options, run_metadata)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1350, in _do_run
run_metadata)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1370, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [32768,256] rhs shape= [8192,256]
[[node save/Assign_185 (defined at /home/blaz/github/SPADE-Tensorflow/SPADE.py:539) ]]
Errors may have originated from an input operation.
Input Source operations connected to node save/Assign_185:
encoder/linear_var/kernel/Adam_1 (defined at /home/blaz/github/SPADE-Tensorflow/SPADE.py:385)
Original stack trace for 'save/Assign_185':
File "main.py", line 125, in
main()
File "main.py", line 116, in main
gan.random_test()
File "/home/blaz/github/SPADE-Tensorflow/SPADE.py", line 539, in random_test
self.saver = tf.train.Saver()
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 825, in init
self.build()
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 837, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 875, in _build
build_restore=build_restore)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 508, in _build_internal
restore_sequentially, reshape)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 350, in _AddRestoreOps
assign_ops.append(saveable.restore(saveable_tensors, shapes))
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saving/saveable_object_util.py", line 72, in restore
self.op.get_shape().is_fully_defined())
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/ops/state_ops.py", line 227, in assign
validate_shape=validate_shape)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/ops/gen_state_ops.py", line 66, in assign
use_locking=use_locking, name=name)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
op_def=op_def)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 3616, in create_op
op_def=op_def)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2005, in init
self._traceback = tf_stack.extract_stack()
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "main.py", line 125, in
main()
File "main.py", line 116, in main
gan.random_test()
File "/home/blaz/github/SPADE-Tensorflow/SPADE.py", line 540, in random_test
could_load, checkpoint_counter = self.load(self.checkpoint_dir)
File "/home/blaz/github/SPADE-Tensorflow/SPADE.py", line 526, in load
self.saver.restore(self.sess, os.path.join(checkpoint_dir, ckpt_name))
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1322, in restore
err, "a mismatch between the current graph and the graph")
tensorflow.python.framework.errors_impl.InvalidArgumentError: Restoring from checkpoint failed. This is most likely due to a mismatch between the current graph and the graph from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:
Assign requires shapes of both tensors to match. lhs shape= [32768,256] rhs shape= [8192,256]
[[node save/Assign_185 (defined at /home/blaz/github/SPADE-Tensorflow/SPADE.py:539) ]]
Errors may have originated from an input operation.
Input Source operations connected to node save/Assign_185:
encoder/linear_var/kernel/Adam_1 (defined at /home/blaz/github/SPADE-Tensorflow/SPADE.py:385)
Original stack trace for 'save/Assign_185':
File "main.py", line 125, in
main()
File "main.py", line 116, in main
gan.random_test()
File "/home/blaz/github/SPADE-Tensorflow/SPADE.py", line 539, in random_test
self.saver = tf.train.Saver()
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 825, in init
self.build()
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 837, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 875, in _build
build_restore=build_restore)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 508, in _build_internal
restore_sequentially, reshape)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 350, in _AddRestoreOps
assign_ops.append(saveable.restore(saveable_tensors, shapes))
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/training/saving/saveable_object_util.py", line 72, in restore
self.op.get_shape().is_fully_defined())
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/ops/state_ops.py", line 227, in assign
validate_shape=validate_shape)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/ops/gen_state_ops.py", line 66, in assign
use_locking=use_locking, name=name)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
op_def=op_def)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 3616, in create_op
op_def=op_def)
File "/home/blaz/anaconda2/envs/tf1.14/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2005, in init
self._traceback = tf_stack.extract_stack()
Any ideas?
Thank you!

you can change the contents of the segmap_label.txt  to
{(0, 0, 0): 0, (0, 0, 255): 1, (255, 0, 0): 2, (150, 30, 150): 3, (255, 65, 255): 4, (150, 80, 0): 5, (170, 120, 65): 6, (125, 125, 125): 7, (255, 255, 0): 8, (0, 255, 255): 9, (255, 150, 0): 10, (255, 225, 120): 11, (255, 125, 125): 12, (200, 100, 100): 13, (0, 255, 0): 14, (0, 150, 80): 15, (215, 175, 125): 16, (220, 180, 210): 17, (125, 125, 255): 18}
The reason is that the author source code only contains 16 classes.

you can change the contents of the segmap_label.txt to
{(0, 0, 0): 0, (0, 0, 255): 1, (255, 0, 0): 2, (150, 30, 150): 3, (255, 65, 255): 4, (150, 80, 0): 5, (170, 120, 65): 6, (125, 125, 125): 7, (255, 255, 0): 8, (0, 255, 255): 9, (255, 150, 0): 10, (255, 225, 120): 11, (255, 125, 125): 12, (200, 100, 100): 13, (0, 255, 0): 14, (0, 150, 80): 15, (215, 175, 125): 16, (220, 180, 210): 17, (125, 125, 255): 18}
The reason is that the author source code only contains 16 classes.

Hi, thank you for your reply!
I already did that, as described in my update:
UPDATE: I resolved this issue by reproducing CelebAHQ masks with 19 segmentation labels (instead of 16 labels as originally defined in spade_celebA\segmap_label.txt).
Now I get the following error (with mismatched dimensions 32768 and 8192):
[*] Reading checkpoints...
W1017 13:40:56.285303<PHONE_NUMBER>93216 deprecation.py:323] From ...
This is why I am not sure what else could it be.

Have you solved this, I encounter the same problem about loading the pre-trained checkpoing.


[feat] add infer fps

Thank you for your contribution to the MindOCR repo.
Before submitting this PR, please make sure:

[x] You have read the Contributing Guidelines on pull requests
[x] Your code builds clean without any errors or warnings
[x] You are using approved terminology
[ ] You have added unit tests

add some models infer  fps
(Write your motivation for proposed changes here.)
Test Plan
(How should this PR be tested? Do you require special setup to run the test or repro the fixed bug?)
Related Issues and PRs
(Is this PR part of a group of changes? Link the other relevant PRs and Issues here. Use https://help.github.com/en/articles/closing-issues-using-keywords for help on GitHub syntax)

在https://github.com/mindspore-lab/mindocr/pull/456更新，此pr可关闭


What is the reasoning for making redux validate action object?

https://github.com/rackt/redux/blob/v3.2.1/src/createStore.js#L156-L161
if (typeof action.type === 'undefined') {
    throw new Error(
        'Actions may not have an undefined "type" property. ' +
        'Have you misspelled a constant?'
    )
}

There is no reason for Redux to perform this validation. Redux must only concern itself with the actions that it dispatches (namely, ActionTypes. INIT).
The workaround that is used at the moment (e.g. projects that implement https://github.com/gajus/canonical-reducer-composition) is to set action.type to a dummy property only to satisfy this single condition, which appears both a hack and unnecessary.
I'd like to propose to remove this condition.

This decision was taken because way too many people had problems with reducers not handling the actions. Turns out, usually it happened due to a typo when importing a constant in the reducer file:
import { INRCEMENT } from './actions'

Of course the matching case never hits, but beginners are often puzzled by this because typos can be hard to spot, and most bundlers don’t analyze dependencies to warn you about missing imports yet.
We felt this is a good enough reason to choose a single convention (in our case, the most popular one is the one that comes from Flux: type property). I understand not everyone is happy with the name, but you can’t please everyone.
The developer efficiency that we get from this tradeoff seems worthy to me.
The only reason Canonical Composition does not fit this pattern is because it insists on using name property for the same thing. We had to choose a name to solve a problem, and we picked the most popular one. So either you’ll need to work around it, or adopt type like everyone else :-).
Previous discussions:

https://github.com/rackt/redux/pull/564
https://github.com/rackt/redux/issues/541

We don’t plan to change this.

Sounds a lot like a responsibility of a dev middleware (validateAction) to me, esp. given that this should only be done in development setup.
I respect your decision though.

On Feb 3, 2016, at 12:34, Dan Abramov<EMAIL_ADDRESS>wrote:
This decision was taken because way too many people had problems with reducers not handling the actions. Turns out, usually it happened due to a typo when importing a constant in the reducer file:
import { INRCEMENT } from './actions'
Of course the matching case never hits, but beginners are often puzzled by this because typos can be hard to spot, and most bundlers don’t analyze dependencies to warn you about missing imports yet.
We felt this is a good enough reason to choose a single convention (in our case, the most popular one is the one that comes from Flux: type property). I understand not everyone is happy with the name, but you can’t please everyone.
The developer efficiency that we get from this tradeoff seems worthy to me.
The only reason Canonical Composition does not fit this pattern is because it insists on using name property for the same thing. We had to choose a name to solve a problem, and we picked the most popular one. So either you’ll need to work around it, or adopt type like everyone else :-).
Previous discussions:
#564
#541
We don’t plan to change this.
—
Reply to this email directly or view it on GitHub.

Sounds a lot like a responsibility of a dev middleware (validateAction) to me, esp. given that this should only be done in development setup.

The problem with this specific mistake is you don’t know you need that middleware until you’ve spent an hour on this and it’s too late :-)
I know it’s opinionated but I think we made the right tradeoff here.


Image works on iOS, but not on Android

This Image code loads a 500x500 image on iOS. Works pretty well. But not Android.
<Image source={{uri: data.product.image}} style= {{ height:80, width: 80 }} resizeMode={Image.resizeMode.contain}/>

It seems that the image should be no more than 80 pixels, in this case. If I put a 80x80 image uri, it works.
Any tips?
Thanks

Does this happen for every resizeMode, or just contain? Can you experiment a bit to figure out exactly when the image doesn't appear, then update the issue title?

This issues is opened blew
https://github.com/facebook/react-native/issues/10569

Ah ok it looks like the same issue. Closing this one so we can track it in https://github.com/facebook/react-native/issues/10569

This fixed this problem on my project!
android:largeHeap="true"

https://github.com/facebook/react-native/issues/10569#issuecomment-279397116


[Feature Request] MSVC support

Clay currently doesn't compile under MSVC because:

It doesn't support C designated initializers
It doesn't support __attribute__((packed))__
It doesn't like CLAY__ALIGNMENT, #define CLAY__ALIGNMENT(type) _Alignof(type) works though

1 and 2 are not very nice to solve with Microsoft flavoured C, although I will say all of these have direct solutions if you don't mind C++

Thanks for opening this one! I had been putting it off until the core implementation stabilized, but we can definitely begin to address each of these

It doesn't support C designated initializers
AFAIK there should be a way at the very least for the library syntax to be compatible with both C99 designated initializers as well as C++20 designated initializers. C++20 is much more picky but should be roughly syntax equivalent if the members are in declaration order

It doesn't support attribute((packed))__
Yep nice, should be able to replace this with a #ifdef of msvc and swap out for whatever equivalent to size enums into 1 byte

It doesn't like CLAY__ALIGNMENT, #define CLAY__ALIGNMENT(type) _Alignof(type) works though
This one should be reasonably easy to #ifdef in as well.

As for supporting non C99 versions of C, the best way forward might be to compile clay as a lib and write a second set of macros / declarations that are compatible with C11 etc... 🤔

From my investigations so far, it looks like I'm probably going to be able to get it in line with MSVC C++20. Microsoft flavoured C though, might not be possible. I don't think I'm willing to drop to C89 for the whole library just to support those versions on MSVC. Probably would be best in that case to just compile as a lib and link in 👍

Follow up: Successfully compiling as C++20 with MSVC here https://github.com/nicbarker/clay/actions/runs/11083013067/job/30796749136 😁

I'm going to mark this as completed for now, when clay stabilizes a little more we can reopen a discussion into whether it's worth dropping to C89 for the library itself in order to support C11 and beyond 🙂

Yep I've been messing with clay in our engine, it seems fine so far but I haven't done much with it yet

@mikejsavage Great, please let me know if you end up building something with it!

@MiraiMindz thanks for reaching out 🙂
The main reason for not dropping to ansi C at this point is that the quality of life features in C99 really make building and changing code so much easier (for me), specifically variadic macros, anywhere declarations and designated initializers.
I'm definitely keeping the option open for when the library stabilizes to just bug fixes & platform specific changes (which I intend to happen eventually) but for now I think it will make dev a little too difficult for me. Sorry if this wasn't the answer you were after, but of course feel free to adapt the core parts of the lib for your own use.

@namandixit that is very interesting, I'll have to review and see how close we are or what the specific problems are. It's possible that by getting to c++20 support I actually got closer to msvc C11 / C17 than I realised.

I'm just trying to compile with MSVC as well and face the same problems. To use packing attributes, they need to be specified in the correct syntax and position:

For MSVC (Microsoft Visual C++), packing must be specified using pragma directives before and after the enum:

#pragma pack(push, 1)
typedef enum {  // No identifier needed here
    // enum values...
} Clay__ElementConfigType;
#pragma pack(pop)

For GCC/Clang, the attribute must be specified using the correct attribute syntax:

typedef enum __attribute__((packed)) {  // Attribute goes here, not an identifier
    // enum values...
} Clay__ElementConfigType;

If you want to create a cross-platform solution, you would typically define a macro that expands to the appropriate syntax based on the compiler:
#ifdef _MSC_VER
    #define CLAY_PACKED_ENUM
    #define CLAY_PACK_BEGIN #pragma pack(push, 1)
    #define CLAY_PACK_END #pragma pack(pop)
#else
    #define CLAY_PACKED_ENUM __attribute__((packed))
    #define CLAY_PACK_BEGIN
    #define CLAY_PACK_END
#endif

Then you could use it like this:
CLAY_PACK_BEGIN
typedef enum CLAY_PACKED_ENUM {
    // enum values...
} Clay__ElementConfigType;
CLAY_PACK_END

Maybe this can be somehow added/handled via a generator as well.
But, what to do until then?

I've implemented this based on feedback in #118.

Yes, sorry, I saw it now.
I am looking forward to getting it merged into the master. Will wait so long.

ERRor
Bro I am facing the same problem hope we can fix this now

+1
I'm currently using C++20 and clang-cl, but clang-cl has some issues and prevents me from compiling a static library (not related with clay), so MSVC support would be great


[management][2/2] use real shared_val_netaddr_key in genesis + config-builder

NetworkConfig can now load the shared_val_netaddr_key either from secure storage or directly from the config.

In the management tooling, operators specify the shared_val_netaddr_key in the validator-config command as an argument. This key is then used to encrypt the validator network address before is storing it in the local secure storage.

Add a wrapper struct for netaddr::Key and impl the relevant libra_crypto::traits.

Hide the libra_network_address::encrypted module and just re-export the important items.

Update the libra cli register_validator command. Though, it appears to be unused as it was broken previously...

:umbrella: The latest upstream changes (presumably 6d7d9e112a1ecfffad60f422de2b86646593d5d9) made this pull request unmergeable. Please resolve the merge conflicts.

rebase

:umbrella: The latest upstream changes (presumably 40ad73104e17d3ad67c326e757396a6718acad84) made this pull request unmergeable. Please resolve the merge conflicts.

:umbrella: The latest upstream changes (presumably 1a705bba4a3d54791c1cb6578585d1ec96caccf0) made this pull request unmergeable. Please resolve the merge conflicts.


MoveFile Failed; Code 183. "code-insiders.exe" Version 1.32 to 1.33

VSCode Version: 1.33
OS Version: 10

Steps to Reproduce:

VSCode updating in the background, version 1.32 to 1.33
Closes
Opens with error 183
Click ignore, installation continues
Error message 183 again
Click Ignore
Installation continues to almost 100%
Error message "Cannot find file"
Uninstall VSCode
Download fresh installation file from code.visualstudio.com
Try to install
Error 183 again.
 Unable to launch

Fixed it.


Http Status 500 with "Accepted by If-None-Match header" is returned when serving unchanged static content

Environment Details

Helidon Version: 4.0.6
Helidon MP
JDK version: 21.0.2
OS: OSX

Problem Description

When loading static content with a matching E-Tag, a HTTP-Status 500 ("Internal Server Error") is returned instead of 304 ("Not Modified") and the response contains "Accepted by If-None-Match header".
It did work as expected in 4.0.5 and does not work anymore in 4.0.6, probably due to #8357.
The following error is being logged: "Attempt to send status 500 Internal Server Error with entity. Server responded with Internal Server Error. Please fix your routing, this is not allowed by HTTP specification, such responses MUST NOT contain an entity."

Steps to reproduce

Create a project which serves a static file with helidon-webserver-static-content.
Load the file in your favourite browser, this shows the file
Reload the file in your favourite browser, this returns error 500 and the message "Accepted by If-None-Match header" is being shown instead of the files content

Also getting this error 4.0.6

This is still an issue in 4.0.11


Changed Accept header in 2.49.0-rc.0 break RabbitMQ build in exporter due q=2 in header Accept

What did you do?
We upgraded to latest stable 2.48.1 and find the bug with remoteWrite race cond which crashed our instance. We decided to give it a go to 2.49 RC version available.
Found out that since upgrade to RC we had down targets for Rabbitmq exporter which is build-in RMQ. Version of RMQ is 3.10.8-1.1 on Debian 12.2 (Bookworm).
After some debugging and since we have on our stage env 2.48.0 version we begin chasing what changed and we found out the Accept header changed between versions.
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x5577ce7cefb0)
* Connected to url.com (x.x.x.x) port YYYY (#0)
> GET /metrics HTTP/1.1
> Host: url.com:YYYY
> User-Agent: Prometheus 2.49.0-rc.0
> Accept: application/openmetrics-text;version=1.0.0;q=0.5,application/openmetrics-text;version=0.0.1;q=0.4,text/plain;version=0.0.4;q=0.3,*/*;q=2
>
< HTTP/1.1 400 Bad Request
< content-length: 0
< date: Fri, 22 Dec 2023 13:44:15 GMT
< server: Cowboy
<
* Connection #0 to host host.com left intact

but the old version sent (ignore the UA this was set to RC version when we tried what changed  )
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x55778da8ffb0)
* Connected to url.com (x.x.x.x) port YYYY (#0)
> GET /metrics HTTP/1.1
> Host: url.com:YYYY
> User-Agent: Prometheus 2.49.0-rc.0
> Accept: application/openmetrics-text;version=1.0.0,application/openmetrics-text;version=0.0.1;q=0.75,text/plain;version=0.0.4;q=0.5,*/*;q=0.1
>
< HTTP/1.1 200 OK
< content-encoding: identity
< content-length: 103555
< content-type: text/plain; version=0.0.4
< date: Fri, 22 Dec 2023 13:45:25 GMT
< server: Cowboy
<
{ [2731 bytes data]
* Connection #0 to host toprecepty-4.superhosting.cz left intact

What did you expect to see?
Problem is the q=2 this should be between 0-1. At least from info like here https://developer.mozilla.org/en-US/docs/Glossary/Quality_values
What did you see instead? Under which circumstances?
failed request 400
System information
Linux 4.19.0-17-amd64 x86_64
Prometheus version
prometheus, version 2.49.0-rc.0 (branch: HEAD, revision: c4c8d59c14ea558a4878fb014cc954347c317978)
  build user:       root@e59256874f10
  build date:       20231212-11:17:29
  go version:       go1.21.5
  platform:         linux/amd64
  tags:             netgo,builtinassets,stringlabels

Prometheus configuration file
No response
Alertmanager version
No response
Alertmanager configuration file
No response
Logs
Nothing valid vas logged to prometheus logs.

overlooked https://github.com/prometheus/prometheus/pull/13313

@frenkye  2.49.0 release have this bug
>prometheus --version
prometheus, version 2.49.0 (branch: HEAD, revision: 683a46cc9999dedf66dd0a4c77644b8d0031d063)
  build user:       root@a83669167414
  build date:       20240115-10:51:05
  go version:       go1.21.6
  platform:         linux/amd64
  tags:             netgo,builtinassets,stringlabels


Undefined index type in theme DatabaseConfigLoader

PHP Version
7.4.16
Shopware Version
<IP_ADDRESS>
Expected behaviour
With the Update to <IP_ADDRESS> the command system:update:finish should run without errors, even if you have some theme config variables without type.
Actual behaviour
With the Update to <IP_ADDRESS> the command system:update:finish is not running without errors, because you get the following notice Undefined index: type in https://github.com/shopware/platform/blob/dd2bf30d0519dcd9416dc269c88edcfd66f92add/src/Storefront/Theme/ConfigLoader/DatabaseConfigLoader.php#L206
Unfortunately there is no type in the database for custom theme config - maybe we initiated something wrong but the DatabaseConfigLoader should not complain about a missing type index.
How to reproduce
Add the following config to your custom custom/plugins/<yourTheme>/src/Resources/theme.json and save the config once in the back end.
By the way the given label variables are not used with this config, too - so maybe there is really something wrong with our config or with the json loader?
    "config": {
        "blocks": {
            "testBlock": {
                "label": {
                    "de-DE": "Test",
                    "en-GB": "Test"
                }
            }
        },
        "fields": {
            "test": {
                "block": "testBlock",
                "type": "text",
                "editable": true,
                "label": {
                    "de-DE": "Test-Variable",
                    "en-GB": "Test variable"
                }
            }
        }
    }

Afterwards you call the bin/console system:update:finish command and you should see the notice Undefined index: type.

A temporary solution for me was to manually update our config_values in the table theme and add ,\"type\":\"text\"

i cannot reproduce the problem. tried all the steps with vanilla installation, have no errors

@alberttomasiak reported same problems to me- maybe he can also help to reproduce the problem.
I haven't tested it with a current vanilla installation but it occurred on an existing installation!

fixed with https://github.com/shopware/platform/commit/6b6d52090f8b6ba2ae9c0c2acdadbcdfda64a0e5


Support JSON1

What would it take to make this library work with json1?
Initial exploration reveals these places might need to be changed:

https://github.com/share/sharedb-string-binding/blob/0837ba041c5019912cd487d702064b9d06baeabf/index.js#L50
https://github.com/share/sharedb-string-binding/blob/0837ba041c5019912cd487d702064b9d06baeabf/index.js#L65
https://github.com/share/sharedb-string-binding/blob/0837ba041c5019912cd487d702064b9d06baeabf/index.js#L72
https://github.com/share/sharedb-string-binding/blob/0837ba041c5019912cd487d702064b9d06baeabf/index.js#L79
https://github.com/share/sharedb-string-binding/blob/0837ba041c5019912cd487d702064b9d06baeabf/index.js#L98
https://github.com/share/sharedb-string-binding/blob/0837ba041c5019912cd487d702064b9d06baeabf/test/index.js#L38
https://github.com/share/sharedb-string-binding/blob/0837ba041c5019912cd487d702064b9d06baeabf/test/index.js#L51
https://github.com/share/sharedb-string-binding/blob/0837ba041c5019912cd487d702064b9d06baeabf/test/index.js#L59

Need to pay attention to modification:
const json1 = require('ot-json1');
...
...
StringBinding.prototype._onOp = function (op, source) {
  if (source === this) return;
  if (op.length === 0) return;
  json1.type.apply(json1.type.create(this.doc.data), op);
  this._parseParentOp();
};

StringBinding.prototype._insert = function (index, text) {
  var path = this.path.concat(index);
  var op = { p: path, si: text };
  const json1op = json0to1([op]); // Change ot command
  this.doc.submitOp(json1op, { source: this });
};

...

StringBinding.prototype._remove = function (index, text) {
  var path = this.path.concat(index);
  var op = { p: path, sd: text };
  const json1op = json0to1([op]);
  this.doc.submitOp(json1op, { source: this });
};


build workflow and support monetization score step

Monetization step tested on my channels
Build workflow will trigger on every PR

Thanks


Installation failure due to early import of external dependencies

When attempting to install uxsim using pip install . or pip install -e ., the installation process fails during the phase where pip tries to get the requirements to build the wheel. This issue arises because the setup.py script indirectly imports the uxsim package, which in turn imports external dependencies such as numpy. Since these external dependencies are not yet installed at the point when setup.py is executed to determine the installation requirements, this results in a ModuleNotFoundError.
Steps to Reproduce

Clone the uxsim repository or download the source code.
Ensure that the environment does not have the dependencies (numpy, matplotlib, etc.) already installed.
Run pip install . or pip install -e . from the root directory of the uxsim project.

You can also directly run:
pip install -U -e git+https://github.com/toruseo/uxsim@main#egg=uxsim

Expected Behavior
The pip install command should successfully install the uxsim package along with its specified dependencies without requiring them to be installed beforehand.
Actual Behavior
The installation process fails with a ModuleNotFoundError, indicating that one of the external packages (numpy is mentioned specifically) could not be found. This error is traced back to an import statement in the uxsim package that is executed as part of the setup.py script's execution.
Impact
This issue prevents the package from being installed in environments where the dependencies are not already present. It affects both development environments and end-users attempting to install uxsim for the first time, potentially limiting the adoption and usability of the package.
Workaround
A temporary workaround involves manually installing the dependencies listed in INSTALL_REQUIRES from the setup.py file before attempting to install uxsim. However, this approach is not ideal as it circumvents the automated dependency resolution and installation process provided by pip.
Suggested Fix
To resolve this issue, it is recommended to refactor the setup.py script and the package initialization process to avoid importing the package or any modules that require external dependencies. Instead, consider one of the following approaches:

Directly specify the version and other metadata in setup.py without importing them from within the package.
Use a separate file to store the version and other metadata that can be read by setup.py without executing import statements that depend on external packages.
Utilize a build-time dependency management tool that supports dynamic versioning without requiring the package itself to be imported during installation.

Additional Information
This issue persists even when the dependencies are installed beforehand, indicating that the version fetching mechanism is inherently flawed and requires attention regardless of the installation environment.
You can see the full error message also here: https://github.com/EwoutH/UXsim/actions/runs/8110064753/job/22166521983
Related:

https://github.com/toruseo/UXsim/pull/19

Hi @toruseo! I would like to use this package for my MSc thesis. I'm a maintainer of two open source libraries, so I can help implementing some best practices for packaging, deploying, testing and continuous integration.
This specific issue is particularly important to me, it needs to be build properly for all the other stuff to work and thus enable rapid iteration. So I'm glad to help out!
I understand however that maintenance of a libraries takes time and effort, so I'm curious on what your medium to long term vision is on this library. If you want to have a call sometime to discuss this please let me know!

If you're not able to spend much time on it, that's also fine, but then I might need to create my own fork. I don't prefer that, because I would like this library to grow based on the contributions of multiple people and keep the effort centralised.

I think the problem is fixed now. Can you check?
Sorry for waiting. I am working on this project in my spare time, so my response may be delayed.

Thanks for working on it! I appreciate it, open-source work can be hard and demanding. Thanks!
I confirmed it fixed both in CI and locally!


Helmholtz design

to still test: for me the napari window is too large (height) when I use macbook screen and i can't adjust it with the self.resize command.


compact: compact still tries to downsample the block, when it has the no-downsample-mark.json

Thanos, Prometheus and Golang version used: Thanos v0.30.2
Object Storage Provider: ceph
What happened:
I have a block that is broken, and when it was downsampled, the compact will halt with failed checksums.
So i I want to skip the downsamp for this block by the #5945 .
But when I marked this block with no-downsample-mark.json, the compact still halt because of this block.
What you expected to happen:
The compact skipping the downsamp of this block.
Anything else we need to know:
I checked the source code after the problem, and found that there is a function NoDownsampleMarkedBlocks() for finding marked blocks, which is defined but not used. Maybe it has something to do with this?
https://github.com/thanos-io/thanos/blob/fe3f5d24192570038e9576307e1b31794920a1f3/pkg/compact/downsample/downsample.go#L790-L791

Similar issue here.


Arguments to hmac.digest are in wrong order

First, thanks a lot for your blog post and your example code! They were of great help in implementing SCRAM in a client I'm working on.
However, while implementing the client, I noticed a minor error; you mixed up the arguments for deriving the client key and server keys.
You write:
client_key = hmac.digest(b'Client Key', salted_password, 'sha256')
server_key = hmac.digest(b'Server Key', salted_password, 'sha256')

But it should be:
client_key = hmac.digest(salted_password, b'Client Key', 'sha256')
server_key = hmac.digest(salted_password, b'Server Key', 'sha256')

Happy to help :-)


Certain shutter speeds cannot be set

Settting shutter speeds can error out for certain values e.g.:
cam.setShutter(3331) #also 9618,...
cam.grab()

PUCException: PUC_SetFramerateShutter:8 the specified exposure time or non-exposure time (clock units) cannot be set.

Is there a way to check if a shutter setting is allowed or how to get allowed shutter speeds? Something similar to setting the resolution (min, max,...)

Sorry for the late reply.
This is Photron support.
We will investigate your question next week.
Please wait a little longer.

Please let us check a few details here.
How fast do you set the FrameRate of the camera?
Some common cameras are capable of long exposures, but this camera cannot use a shutter speed slower than the Frame Rate.
We are currently having difficulty operating the actual camera due to the coronavirus, so we would like to start with the easy part.
We will check other possible causes.

Unfortunately I am unsure about the frame rate that was set. This was done with an evaluation camera and we do not have it at the moment, so I can not retest it either, to see whether it was due to framerate.
As I recall, we also had issues setting up short exposures and were looking to get the limits to prevent from erroring out, therefore min, max would be appreciated.
Or perhaps access to  SetExposeTime... as I understand this would set the time ON instead of 1/fps?

Hi,
Just in case anybody have the same issue; It seems to not work if you only specify the shutter speed. e.g.
cam.setShutter(250)
You can solve it by specifying both the frame rate and shutter speed using function;
cam.setFramerateShutter(250,250)
At least it seems to be working for me. I hope it helps.


Create separate pages for different events?

Say we were to hold this workshop again, how can we host the different workshops materials so they are available online?
I think Oceanhackweek did some neat tricks to make this happen?
cc: @ocefpaf

I think Oceanhackweek did some neat tricks to make this happen?

To be honest I'm not familiar with the current OHW page. @abkfenris re-wrote the whole thing.
However, any versioning scheme would do. My recommendation is to move the old page to a URL/year_of_the_event and create the new one with links to the old one.

The best options I am imagining:

create a branch 2022 to keep the older version code & use main for the latest version
fork this to a new repo named bio_mobilization_workshop_2022

The only reason for (2) instead of (1) is to keep the older github pages website up. Which.. do we even want that?

To keep past OceanHackWeek docs around, we made sure that each event lived under it's own path, then we didn't clean up the branch on each deploy, so we slowly accrete events over time.
https://github.com/oceanhackweek/oceanhackweek.github.io/blob/17719174695e9983b26bbc02ce5777cd5b1e5a66/.github/workflows/deploy-docs.yml#L22-L29

our build process is quite a bit different.
https://github.com/ioos/bio_mobilization_workshop/blob/41455eb42c381c296febc72b442b226a90a59fc6/.github/workflows/website.yml#L93-L119
maybe we should switch over to JamesIves/github-pages-deploy-action?

Thanks for all the input here. I am concerned with repo proliferation if we were to fork a 'base' workshop for each year.
I think we need to sit down and really think about how much content will be changing between years. And if we need that content available for all iterations. If anything I think these workshops will either refine existing content or provide updated guidance, we wouldn't need to point back to 2022 guidance as 2023 should contain the same/updated info?
I wonder if we could just keep this simple and do yearly tagged releases of the gh-pages branch? gh-pages will always be the most up-to-date (but could be in-development for the next year). Maybe farm off each year into it's own branch as @7yl4r describes?

Maybe farm off each year into it's own branch as @7yl4r describes?

tags and branches are different but with similar functionalities. IMO we should use tags to "freeze" a page for a certain year and develop the latest one on the "main" gh-pages. Older pages can be published on a /year-of-the-event. Keeping all pages in the same repo and under the same domain.

@ocefpaf I like that approach.
But, can you explain?

Older pages can be published on a /year-of-the-event.

I guess I don't understand how we would accomplish that, technically.

The best options I am imagining:

create a branch 2022 to keep the older version code & use main for the latest version
fork this to a new repo named bio_mobilization_workshop_2022
The only reason for (2) instead of (1) is to keep the older github pages website up. Which.. do we even want that?
I am concerned about the question at the end of Tylar's suggestion. Yes we do want to keep the website up because we have been linking to it like in this news post up on OBIS. If the website disappears that would be problematic in my mind? Unless we are going to have each new year of the workshop at this specific link (https://ioos.github.io/bio_mobilization_workshop/) and people could still get to the "curriculum"?

@albenson-usgs The way I read @7yl4r's comment is that we will always have a page at https://ioos.github.io/bio_mobilization_workshop/ which will be the most recent iteration of the workshop. The question is about if we want to make the previous year's material still available as a webpage?
In your example above, would it be problematic of that link to the curriculum went to the most recent materials? Or, do we need it to go to the materials for 2022 specifically?

In your example above, would it be problematic of that link to the curriculum went to the most recent materials? Or, do we need it to go to the materials for 2022 specifically?

Good question. The blog post is specifically about the 2022 workshop so to me it is a bit strange to point to a curriculum that is for say 2025 but maybe I'm overthinking it. It just feels a little strange to do that here when in general we promote maintaining and ensuring accessibility to all versions of say data that's being publishing in a workshop like this. But it's probably fine and realistically not too many people are going to want to get to specifically the 2022 workshop website and if they really wanted to get to that content specifically they would be able to access it via GitHub.

But it's probably fine and realistically not too many people are going to want to get to specifically the 2022 workshop website and if they really wanted to get to that content specifically they would be able to access it via GitHub.

I agree with that assessment. Keeping old pages is only worth for  "googling" purposes. Anyone that knows how the workshop is organized will find whatever they need much faster on GH.

After re-reading the comments, I'm a little confused as to where we landed. I think it was decided to try and work the content into a structure documented by @ocefpaf above https://github.com/ioos/bio_mobilization_workshop/issues/36#issuecomment-1348867468.
The PR I submitted does not do that. It changes the 2022 page to reflect new dates for 2023. But, does not keep 2022 as a persistent url.
If we do want to pursue keeping persistent urls for previous year's materials, we need to be cognizant of the comments by @jackVanish

To the best of my knowledge a lot of these Carpentries site setups rely on things being in the correct place so that they are rendered and linked properly. If we start putting things in other folders, we could end up scotching the render process for older content.

I was able to get the carpentries template to function and make it work for our needs, but this might be going beyond my ability. If anyone has specific guidance on how to do this, I would appreciate it. Otherwise it will be trial and error.

I am sure there is a tooling out there to help with "archiving" of a site in this way, but I don't know of it.

I've started tinkering around with this and found https://github.com/jekyll/jekyll-archives which could be another pathway.
I've also been testing with Jekyll collections to create a previous collection which contains the lessons from a specific year. The benefit of this method is:

Simple to manage
Keeps everything built out of the same repo
don't need to fiddle with the GH-action.

However, this doesn't allow us to keep the home landing page for any previous workshops. I've tried adding the index.md to the specific directory, but the links still go to the root files (not the previous ones). I suppose we could start mucking around in the configuration to allow for these changes, but I'm not sure it's worth the time.
Right now, we end up with an html directory of the specific lessons. I don't think that is very useful to folks.

I will push my changes to my fork so folks can take a look and see what they think.

 It looks like netlify and antora.org support sites published from branches, and worst case we can build out a custom deploy workflow in github actions.
Branches and tags work so well for this within git that it seems like a shame to lose that.

Deployed a test site. Below is an example of how I've implemented collections:
This url would be the 'active' page (so where we'd be developing 2023 content)
https://mathewbiddle.github.io/bio_mobilization_workshop/05-break/index.html
Then, if you'd like to get to the previous year's material:
https://mathewbiddle.github.io/bio_mobilization_workshop/2022/05-break/index.html
This assumes someone knows the url structure.
I'll look around and see if there is a simple configuration we can add to say 'if the page is in this collection, change the urls to use [predefined] structure. That way we can at least have the pages link to the appropriate year's page.
Now that I look at it more, we are loosing the headers, navigation, etc. when reading in this collection. 😠

The way readthedocs handles versioning is the ideal behavior. From what I am finding the jekyll+carpentries tooling used here does not support that. Building this functionality would benefit the carpentries ecosystem in general so maybe we should take this conversation upstream to that community and see if there is interest.
 Unfortunately I think this fits best under "infrastructure" and that seems to be inactive.

The way readthedocs handles versioning is the ideal behavior. From what I am finding the jekyll+carpentries tooling used here does not support that. Building this functionality would benefit the carpentries ecosystem in general so maybe we should take this conversation upstream to that community and see if there is interest.
 Unfortunately I think this fits best under "infrastructure" and that seems to be inactive.

Ha! Nevermind. Matt beat me to it: https://github.com/carpentries/workshop-template/issues/779
I am hopeful we will see some discussion there.

Just something to note. To see what has changed between different iterations. We can compare tags to branches. So, for example, to see the difference between the 2022 materials and what we're currently working on, you can browse to https://github.com/ioos/bio_mobilization_workshop/compare/2022...gh-pages to get a summary.
This has me wondering how badly we need previous years materials available on a webpage?

This has me wondering how badly we need previous years materials available on a webpage?

Honestly... I hadn't really thought hard on this. The only reason I would use the old webpages is if the newer page didn't capture something as well as the older one did, and hopefully that won't happen.

Decision - We have releases and tags and doi's and commit history to trace back previous year's materials. We will simply update the materials and not serve the 2022 website.

:shipit:


How reset wizard?

I need know how component wizard can be reset on method onComplete. Regards

Hello @yblancof2
There is an internal reset method you can call via refs.
https://github.com/cristijora/vue-form-wizard/blob/master/src/components/FormWizard.vue#L260