Using NetCDF4 instead of Zarr format for Cloud-Optimized Data

I was looking at the ingest document and saw that rechunking to Zarr format was mentioned.
The Zarr format was invented to address a perceived problem that the NetCDF4 format was not cloud-performant.
We have since learned that it's not the NetCDF4 format but the NetCDF4 Library that is not cloud-performant.
We can now read collections of NetCDF4 files as a single virtual dataset using the Zarr library in Python with access speeds identical to reading Zarr format data.
Since NetCDF4 files are friendlier to R and other non-Python users,  it's friendlier to distribute cloud-optimized data as collections of NetCDF4 files.   Python users can still access the collection of NetCDF4 files as a single dataset (lazily of course) in Xarray from an Intake catalog using fsspec's referenceFileSystem (just as if it were in Zarr format).
It's still important to consider rechunking the NetCDF4 files for optimal performance for a variety of use cases.
Probably worth some discussion.  :)

The Data Conversion step mentioned there is meant to be an example or possible step in the ingest process. I've noted that converting everything to zarr is likely a no-go as a requirement, but some data owners may want to provide zarr data. If instead one wants to chunk netcdf files then that is what would occur in that step.
We also need to keep in mind that we'll need to support a variety of file types, not just grids, and so how can we find common ground among those? I think it will come down to consistent metadata standards to describe those objects.
In terms of making data analysis-ready, what is the common ground that we need to look for? Is a bucket of netcdf/zarr files (ala NODD) sufficient? If so what conventions do we need to adhere to in order to make these buckets interoperable?

In reference to NODD, I think we also need to explore workflows where data has already been uploaded to the cloud but we want to make it discoverable and interoperable in DMAC without making copies of the data.

Thanks for the input @rsignell-usgs!  In a way, this is reassuring to read, however since the last 4+ years Zarr has been all the rage in terms of cloud data optimization, it's probably not cut and dry as far as what format to use for different use cases.
Are there advantages to using Zarr rather than netCDF, if you're willing for example to overlook R compatibility as an advantage (for argument's sake)?  How difficult is it to re-chunk and generate the referenceFileSystem metadata for a large netCDF dataset vs convert to Zarr and re-chunk in the process?  Maybe it isn't, I'm not sure.  What about other APIs compatibility with referenceFileSystem metadata - can OGC EDR API read it for example as it does with Zarr?
We can do some experimentation as part of this project.  Maybe we should convert a few good candidate datasets to both Zarr and netCDF/RFS and put them both on an open-access cloud object store, update in real-time (as in forecast model output), and see which attracts more use and/or can be used in more downstream products that we either build ourselves or others do.
Also, should we take into account analyses that have already been done about the pros/cons for different cloud-optimized formats?  For example, the ESIP Cloud Computing cluster guide - but maybe this is outdated already?

@mwengren, the kerchunk/referenceFileSystem approach uses the Python Zarr library to read the collection of NetCDF files, so yes, any Python tool that reads the Zarr format can read collections of NetCDF4 files (or GRIB files) via referenceFileSystem

I am going to take a stab to quickly prototype out kerchunking GFS and OFS data from NODD. A few steps I will take:

Kerchunk a single model run output of GFS (or GFS Wave)
Implement something smart that can do automatically dedupilcate variables, making sure that the variables kept are from the newest model runs. This is easy with xarray but need to figure out the flow with kerchunk.
Test serving up the data with zarrdap or restful-grids (our code-sprint hack router extending xpublish)

Beyond this, the next steps would be creating intake catalogs for downstream consumers to read from. BUt oging to start at a high level to get through the functionality we rely on from TDS in our production systems.

@mpiannucci , I have a message in my inbox about upcoming development efforts on zarrdap.   I'm going to ask them if they are okay discussing the effort out in the open at https://github.com/NCEI-NOAAGov/zarrdap/issues

@mpiannucci here's the notebook we started on together yesterday, which kerchunks a single TBOFS file:
https://jupyter.qhub.esipfed.org/hub/user-redirect/lab/tree/shared/users/rsignell/notebooks/COAWST/rpsgroup_kerchunk.ipynb


Self-referring links in manual are broken

Due to the move of manual structure, the links that refer to parts of the manual itself are broken.

@averbraeck How do we make consistent relative URL's? The link /manual/02-model-structure/events/ works for opentrafficsim.org, but not for readthedocs.io. There, this results in https://opentrafficsim.readthedocs.io/manual/02-model-structure/events/ but should be https://opentrafficsim.readthedocs.io/en/latest/02-model-structure/events/.

What move of the manual took place? Because right now, it really has to reside as a 'docs' folder in the project root of the parent project to work (all scripts on the Delft server are also rely on that).

It was a while ago when you placed it in the current automated workflow using the .yml file. At least, I assume that's when the links got broken. For example we have [Deviative nature of operational plans](/behavioral-models/tactical-planner#operational-plan), which has the old folder structure in it. It should in the end refer to https://opentrafficsim.org/manual/06-behavior/tactical-planner/#operational-plan.

The events page that you used as an example works fine, both on readthedocs and on opentrafficsim.org. Of course, I had to refresh the page (ctrl/f5 in the browser) to reload the structure. Or do you mean the links within a certain page to another page?
The only ting that did not work for me were the links on the tutorial page.

And the tutorial page now also works.

On the djutils page (https://opentrafficsim.readthedocs.io/en/latest/02-model-structure/djutils/) on readthedocs, when I click on the event list link, I'm referred to:
https://opentrafficsim.readthedocs.io/manual/02-model-structure/events/
This is a page, but it has no layout and only shows the menu in the most basic html formatting. It should refer to:
https://opentrafficsim.readthedocs.io/en/latest/02-model-structure/events/
The perception is broken too. I see the same ugly menu. It should also refer to .../perception/introduction/ as the chapters no longer have an introduction that is not a sub section.

So it is about (relative) links on pages. I will check.

Links now work, in opentrafficsim.org, in readthedocs, and also in Github itself. Note that:

references should ALWAYS be relative (so using ../
for links to also work in Github, add the '.md' to the filename. The compiler will replace this with .html for readthedocs and opentrafficsim.org.
a file that is on the same level, does not need a subdirectory, so the link to the 'event' page was just: \[[List of all event types in OTS](events.md)\].
you can use anchors as you would do normal; so for 08-tutorials/index.html: [How to set up a simulation](simulation-setup.md#how-to-set-up-a-simulation)<br>. Again, the filename has the .md to make it work in Github as well.

All links have been fixed. The trick is really to always point to a .md, even when referring to a section as a whole.


Missing parameter password

I used the nextjs example of next-iron-session, and set the SECRET_COOKIE_PASSWORD in the .env but whenever I try to do a login I get the Missing parameter password error.
I checked in the /lib/session.js and it is using process.env.SECRET_COOKIE_PASSWORD as well.
I don't have an idea of where else again I have to pass a password parameter.
this is the /lib/session.js file.
// this file is a wrapper with defaults to be used in both API routes and getServerSideProps functions

export default function withSession(handler) {
  return withIronSession(handler, {
    password: process.env.SECRET_COOKIE_PASSWORD,
    cookieName: "next-iron-session/session/next.js",
    cookieOptions: {
      // the next line allows to use the session in non-https environements like
      // Next.js dev mode (http://localhost:3000)
      secure: process.env.NODE_ENV === "production",
    },
  });
}

Do a console.log(process.env.SECRET_COOKIE_PASSWORD) just before passing it down to next-iron-session and see if it contains a value. If no then you should learn/read on how to use env variables in your Next.js code (https://nextjs.org/docs/basic-features/environment-variables)


Middleware per request

Hey,
I love the concept of using middlewares. But as far as I see, these are only used globally for the whole monk/mongo instance. I'd love to be able to create an instance of monk with middlewares applied per request basis, so data can be filtered by some user fields. Something like that:
const db = require('monk')('localhost/mydb')
server.use((req, res, next) => {
   const clone = db.clone();
   // Add a monk middleware that filters data by userid
   clone.addMiddleware(context => next => (args, method) => {
     return next(args, method).then((res) => {
        return res.filter(x => x.userId === req.user.id);
   });
   req.db = clone;
   next();
})
server.use(graphql)

Would this be hard to implement? Any thoughts about this?

Hum that seems hard yeah.
How about passing the userId as an option of the query? So you could do something like this:
context => next => (args, method) => {
  // check if the userId field is set in the options and store it
  const userId = (args.options || {}).userId

  // if it is, remote so that it doesn't pollute the query
  if (typeof userId !== 'undefined') {
    delete args.options. userId
  }

  return next(args, method).then((res) => {
    if (typeof userId !== 'undefined') {
      return res.filter(x => x.userId === userId);
    }
    return res
  }
}

Hey, thanks for the fast response! But how do I pass the args on a per-request base? I thought I could only add options via the .addMiddleware function.

db.find({}, {userId: req.user.id})

Woops, alright, that was obvious. Thanks a ton, works perfectly! Keep up the great work!


[🐞] V2: Bug when signal of useComputed$ is used in useTask$ or useVisibleTask$

Which component is affected?
Qwik Runtime
Describe the bug
When a useComputed$ signal is read in useVisibleTask$, the task is executed twice (or multiple times if multiple useComputed$ are used). Also, useComputed$ is unnecessarily re-executed afterwards.
Reproduction
https://stackblitz.com/edit/github-o2f6fj9n-meq2plo7?file=src%2Froutes%2Findex.tsx
Steps to reproduce

Open StackBlitz link
Open and clear browser console
Reload Qwik App

Now you should see this in the console:
useVisibleTask$: Do stuff with text...
useVisibleTask$: Do stuff with text...
useComputed$: Computing initial text...

The expected output would be:
useVisibleTask$: Do stuff with text...

This is because useComputed$ should only run once on the sever and useVisibleTask$ should only run once in the browser. A similar bug exists when using useTask$.
System Info
-

Additional Information
No response

https://qwikdev-build-v2.qwik-8nx.pages.dev/playground/#f=7VZBDoIwELz7it7gYDEmhkQinnyCX4CEC5oAXggv8OQL%2FKJPcHa30AUNiTHevBDabrft7OzuKNLEm3gIFNY9AZYYSfulP81nN9YVb7n4TroIXsRp4DXKHW8jVn1rD7RVYuSXtEZRFjXSkZ1FURTgxbTXQR%2FgRsYQb2fCAbVUmMf9dvUiCdNHtBL2mrb07XqYPbDklblNpsjZc4XX0NXJPnFo7kQTUfZM%2BcInpAohPpMd7cOwFXy6ARC01KyC4qit6%2FC2hIHFTMYyaVWeLFCyFwmUrRGp19CFUqbncNbmiTlAjNRNnkv%2Fm8Cst%2FISCmaT8eIYdBaY9DyRmIze%2B1Ky3iqx%2FEkpGTHxX0h%2BUUie


non-unique task ids

Summary
At https://github.com/codedellemc/libstorage/blob/8b71a2f1e15d1a09be434ddc6a48d94afbe44d2c/api/server/services/services_task.go#L167 the taskid is set to the size of the tasks map. This means that tasks don't have unique IDs. This means operations like inspecting, waiting, and removing tasks can lead to unexpected behaviour.
Bug Reports
Non-unique task IDs treated as unique.

Hi @gpaul,
Yeah, I goofed here. They're unique for the duration of their existence, but you're correct in that attempts to track them could result in collisions. Mea culpa.
I was hesitant to use UUIDs due to the cost of creating them and what they'd look like in a URL. I will likely end up using the first eight characters of a SHA1 hash (much like Git commit IDs are abbreviated). The chances of a collision are very small and would still look aesthetically pleasing in a URL.
To alleviate the cost of creation I'll probably create a buffered bucket of these using a goroutine that is kept full or allows for a certain level of drain during burst periods in order to offset the cost of creating a task ID with each API call.

Yeah, I goofed here. They're unique for the duration of their existence, but you're correct in that attempts to track them could result in collisions. Mea culpa.

It happens :+1: "With enough eyeballs, all bugs are shallow."

I will likely end up using the first eight characters of a SHA1 hash (much like Git commit IDs are abbreviated). The chances of a collision are very small and would still look aesthetically pleasing in a URL.

Should be perfectly fine. May I ask why you don't just have an incrementing counter? Very similar to how you're using len() at the moment.

Issue moved to codedellemc/rexray #967 via ZenHub


Parsing /etc/resolv.conf flips DNS server order

in my case i have this /etc/resolv.conf
domain internal.mycompany.com
search internal.mycompany.com
nameserver <IP_ADDRESS>
nameserver <IP_ADDRESS>

while <IP_ADDRESS> can resolf SOA for www.gateway.internal.mycompany.com .. our internal DNS server cannot (<IP_ADDRESS>) since it is using a usual domain override ( <IP_ADDRESS> ) .
In case that is of any interest, we use the reference implementation of unbound in https://opnsense.org/ with a appliance based private domain defintion.
So beside all the issues with https://github.com/xenolf/lego/issues/544 in general, this issue is about:
When resolv.conf gets parsed, the order of nameservers is inverted, thus <IP_ADDRESS> is checked first for _acme-challenge.www.gateway.internal.mycompany.com SOA and this is an empty response. That will sho a SERVFAIL and the whole challenge fails

Extending this section, the order is not simply "inversed". When i use
domain internal.mycompany.com
search internal.mycompany.com
nameserver <IP_ADDRESS>
nameserver <IP_ADDRESS>
nameserver <IP_ADDRESS>

it will still not work
domain internal.mycompany.com
search internal.mycompany.com
nameserver <IP_ADDRESS>
nameserver <IP_ADDRESS>
nameserver <IP_ADDRESS>

will work though. Could be still inverse since you might just do dedublication


Missing Batflat translations

Hello, Batflat community! 🦇
We miss several translations and we need your help!
ℹ After clicking Click to expand you will see source translations. All you need to do is translate English phrases into the given languages.
:tr: Turkish:

Click to expand!
modules/dashboard/lang/admin:
rss_fail_title = "Connection timeout"
rss_fail_desc = "Unable to download latest news from Batflat.org."

modules/devbar/lang/admin:
module_name = "Debug bar"
module_desc = "Shows helpful information for developers"

modules/searchbox/lang:
placeholder = "Search for..."
results_fo = "Search results for '%s'"
too_short_phrase  = "The phrase you entered is too short! Please enter at least %d characters."
no_results = "No results found for '%s'."

modules/searchbox/lang/admin:
module_desc = "Allows you to search for pages and blog entries that contain the phrase you entered."

modules/sitemap/lang/admin:
module_name = "Sitemap generator"
module_desc = "Sitemap.xml generator for Batflat."

modules/statstics/lang/admin:
Missing all translations below:
module_name     = "Statistics"
module_desc     = "Provides simple statistics information about website."
unique_today    = "Unique today"
today_visits    = "Today visits"
online          = "Online"
unique          = "Unique"
all_unique      = "All unique"
visits          = "Visits"
all_visits      = "All visits"
show            = "Show all"
all_referrals   = "All referrals"
yesterday       = "Yesterday"
7days           = "Latest 7 days"
30days          = "Latest 30 days"
total           = "Total"
target          = "Target"
pages           = "Pages"
os              = "Operating Systems"
browsers        = "Browsers"
countries       = "Countries"
referrals       = "Referral URLs"
direct_visit    = "direct visit"
url             = "Url"
all_pages       = "All website visits"
referrals_from  = "All pages reffered from"
from_address    = "From url"
stats_page      = "Stats for page"

:sweden:  Swedish:

Click to expand!
modules/devbar/lang/admin:
module_name = "Debug bar"
module_desc = "Shows helpful information for developers"

modules/searchbox/lang:
placeholder = "Search for..."
results_fo = "Search results for '%s'"
too_short_phrase  = "The phrase you entered is too short! Please enter at least %d characters."
no_results = "No results found for '%s'."

modules/searchbox/lang/admin:
module_desc = "Allows you to search for pages and blog entries that contain the phrase you entered."

modules/sitemap/lang/admin:
module_name = "Sitemap generator"
module_desc = "Sitemap.xml generator for Batflat."

modules/statstics/lang/admin:
module_name     = "Statistics"
module_desc     = "Provides simple statistics information about website."
unique_today    = "Unique today"
today_visits    = "Today visits"
online          = "Online"
unique          = "Unique"
all_unique      = "All unique"
visits          = "Visits"
all_visits      = "All visits"
show            = "Show all"
all_referrals   = "All referrals"
yesterday       = "Yesterday"
7days           = "Latest 7 days"
30days          = "Latest 30 days"
total           = "Total"
target          = "Target"
pages           = "Pages"
os              = "Operating Systems"
browsers        = "Browsers"
countries       = "Countries"
referrals       = "Referral URLs"
direct_visit    = "direct visit"
url             = "Url"
all_pages       = "All website visits"
referrals_from  = "All pages reffered from"
from_address    = "From url"
stats_page      = "Stats for page"

:it: Italian:

Click to expand!
modules/searchbox/lang:
Done. Thanks @tecnoweb
modules/searchbox/lang/admin:
module_desc = "Allows you to search for pages and blog entries that contain the phrase you entered."

modules/sitemap/lang/admin:
module_name = "Sitemap generator"
module_desc = "Sitemap.xml generator for Batflat."

modules/statstics/lang/admin:
module_name     = "Statistics"
module_desc     = "Provides simple statistics information about website."
unique_today    = "Unique today"
today_visits    = "Today visits"
online          = "Online"
unique          = "Unique"
all_unique      = "All unique"
visits          = "Visits"
all_visits      = "All visits"
show            = "Show all"
all_referrals   = "All referrals"
yesterday       = "Yesterday"
7days           = "Latest 7 days"
30days          = "Latest 30 days"
total           = "Total"
target          = "Target"
pages           = "Pages"
os              = "Operating Systems"
browsers        = "Browsers"
countries       = "Countries"
referrals       = "Referral URLs"
direct_visit    = "direct visit"
url             = "Url"
all_pages       = "All website visits"
referrals_from  = "All pages reffered from"
from_address    = "From url"
stats_page      = "Stats for page"

ℹ In the reply, please write:

language name,
path/to/file
code = translation

Italian:
modules/searchbox/lang:
placeholder = "Cerca per.."
results_fo = "Risultati trovati per '%s' "
too_short_phrase  = "La parole inserite sono troppo corte! Inserisci una frase che contenga almeno %d caratteri."
no_results = "Non sono stati trovati risultati'%s'."

Hi, long time no see :)
I have just finished all the Swedish translations. But I am unfamiliar with the process how to upload them here.
I will send you an email later. When I am finished with the demo pages ;). The DOCS are ready to go. Just so you know.
Best regards!
/Michael, Sweden

Hi there, you´ll find the German translation here: https://github.com/smitti/batflat/blob/master/Batflat_1-3-6_de_german.zip
Regards Uwe

Hi,
Have not yet learned how to add zip files. :). But here are all the Swedish files.
https://github.com/Silverunit/Batflat-lang-SE


Update doobie-core, doobie-h2, ... to 0.12.1

Updates

org.tpolecat:doobie-core
org.tpolecat:doobie-h2
org.tpolecat:doobie-refined

from 0.10.0 to 0.12.1.
GitHub Release Notes - Changelog - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Files still referring to the old version number
The following files still refer to the old version number (0.10.0).
You might want to review and update them manually.
project/plugins.sbt

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.tpolecat" } ]

labels: library-update, semver-minor, old-version-remains

Superseded by #244.


Privacy Manifest in it's own bundle causing AppStore Connect issue?

Initially I had several warnings in the App Store Connect email about various APIs that needed warnings, I updated my dependencies to versions that included their privacy manifests. And then the next submission I was left with just one:

ITMS-91053: Missing API declaration - Your app’s code in the “Elfin” file references one or more APIs that require reasons, including the following API categories: NSPrivacyAccessedAPICategoryFileTimestamp

I know I don't use any of the APIs in my own app code. I checked to confirm it's just this library that is needing this category eg using this here.
I uploaded a build to App Review without Firebase (and therefore without this library) and didn't get the warning email for ITMS-91053.
I noticed that all the other bundles for privacy manifests in my app bundle are named exactly after the framework/targets, where as this library has it's own bundle for the privacy manifest.
According to this Apple Engineer here (https://forums.developer.apple.com/forums/thread/742221?answerId=776008022#776008022) each target needs to have its own privacy manifest, so would it be possible to update this library so that each target has its own privacy manifest directly?
(Apologies for not being a Googler, but the issue seems to be directly with this library and not Firebase)

@ncooke3 Really sorry to mention you directly, but as it's May 1st tomorrow which is Apple's deadline, would you be able to take a look at this?

If you haven't yet seen the latest guidance from Apple, see https://developer.apple.com/news/?id=pvszzano.
The "newly added third-party SDK" clause may have impacted our testing. We'll investigate.

Hi everyone, this has been addressed in the just released 7.13.3. Thanks for reporting!


Add mod.conf

for compatibility.

This was added in the latest version in the minetest-mods  repo.  All development has moved there.


Configure Tensorflow on GPU using Java API

Hello,
How to set following config "gpu_options.allow_growth = True" for Tensorflow, using Java API?
I tried using this code but it doesn't work:
        ConfigProto config = ConfigProto.newBuilder()
                  .setGpuOptions(GPUOptions.newBuilder()
                                         .setAllowGrowth(true)
                                         .setPerProcessGpuMemoryFraction(0.04)
                                         .build()
                                        ).build();
           model.session().runner()
                    .setOptions(config.toByteArray())
                    .feed("image_tensor", input).fetch("detection_scores")
                    .fetch("detection_classes").fetch("detection_boxes").fetch("num_detections").run();

I know the method setOption is an experimental method.
Is there another away to achieve it?
Thanks

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
Have I written custom code
OS Platform and Distribution
TensorFlow installed from
TensorFlow version
Bazel version
CUDA/cuDNN version
GPU model and memory
Exact command to reproduce

Could you elaborate on what you mean by "it does not work". Can you provide a minimal, complete, verifiable example to reproduce the problem?
Thanks

Used this code code,
Coco Models  for Inference: faster_rcnn_nas
I ran in AWS using AWS EC2, type p2.xlarge, with Tesla K80,
I chose a Linux instance for Deep Learning,
Tensorflow GPU version: 1.8.0 and 1.9 rc
Java 8

The process reserve the whole memory but the actual utilization is under 40% - 50%. I used nvidia-smi queries for monitoring the GPU.

Oh, I see.
Session.runner().setOptions() is supposed to be provided with a RunOptions protocol buffer (see javadoc), not a ConfigProto message.
The ConfigProto message needs to be provided when creating the Session object and currently SavedModelBundle.load() doesn't make this possible.  PR #18716 was supposed to make this possible, but it seems like the PR author didn't get around to updating the PR to fit into the master branch. I can try to fix that up.
Ideally, this error would have been caught by the compiler if the API accepted Java objects for the protocol buffer messages instead of a byte[] of serialized data. However, in an effort to ensure that
this lowest layer of the Java API has no dependencies, we avoid a dependency on the Java protocol buffer libraries, which comes at the cost of this confusion (since the serialized ConfigProto message provided can be parsed into a RunOptions message without error).
I'll make a PR to allow a serialized ConfigProto to be provided when loading a SavedModelBundle which would then address your use case.

Thank you!

Hello,
How can I test it?Can I use this version 1.9.0-rc2?
Thank youAdrian    On Saturday, July 14, 2018, 12:28:05 AM GMT+3, Asim Shankar<EMAIL_ADDRESS>wrote:
Oh, I see.
Session.runner().setOptions() is supposed to be provided with a RunOptions protocol buffer (see javadoc), not a ConfigProto message.
The ConfigProto message needs to be provided when creating the Session object and currently SavedModelBundle.load() doesn't make this possible. PR #18716 was supposed to make this possible, but it seems like the PR author didn't get around to updating the PR to fit into the master branch. I can try to fix that up.
Ideally, this error would have been caught by the compiler if the API accepted Java objects for the protocol buffer messages instead of a byte[] of serialized data. However, in an effort to ensure that
this lowest layer of the Java API has no dependencies, we avoid a dependency on the Java protocol buffer libraries, which comes at the cost of this confusion (since the serialized ConfigProto message provided can be parsed into a RunOptions message without error).
I'll make a PR to allow a serialized ConfigProto to be provided when loading a SavedModelBundle which would then address your use case.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

@AdrianXIA : You'll have to build from source. This change isn't included in 1.9 or 1.10 (whose branch was cut just before the change was merged).
Instructions for building from source: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/java#building-from-source

Hello,
I actually built from source the version with the Loader in SavedModelBundle.
However it seems that using SavedModelBundle.withConfigProto(config) doesn't impact anything compared to the "default" configuration.
Code is below :
`        ConfigProto configProto = ConfigProto.newBuilder()
.setAllowSoftPlacement(true) // allow less GPUs than configured
.setGpuOptions(GPUOptions.newBuilder().setPerProcessGpuMemoryFraction(0.01))
.build();
SavedModelBundle  bundle = SavedModelBundle.loader(modelDir).withTags("serve").withConfigProto(configProto.toByteArray()).load();`
Am I missing something ?

@DutAlex : Just to confirm: When you say there is "no impact" you mean that you see all of the GPU memory being allocated by the JVM process (as reported by nvidia-smi)? Could you post some more detail on how you're running the program and observing the change?

@asimshankar Yes, all the GPU memory is allocated to the JMV process (using nvidia-smi ). To run the program, I am building the .jar file using mvn package and running it with using java -cp jar-file-with-dependencies.jar.
In the main file, I am loading a model and open a file where I make prediction on 10000 inputs (1-by-1). I am using System.nanoTime() to profile how long did the prediction of those 10000 examples take.
I see no difference in execution time whether I am setting the GPUOptions (GPUOptions.newBuilder().setPerProcessGpuMemoryFraction(0.01).build()) or using the historical version of the SavedModelBundle (SavedModelBundle.load(modelDir, "serve")).
In addition to that, I am also profiling my GPU activity every second while the model is making predictions (using gpustat) and see that its activity during the predictions is around ~65% whether I am setting the GPUOptions or not.

@DutAlex : There seem to be two things in your comment

Even after setting per_process_gpu_memory_fraction, all GPU memory is being allocated.
Setting that option doesn't change the time it takes to run predictions.

I don't quite see why setting the memory fraction option should change the latency of predictions. The 65% utilization of GPU cores is not something this option would address. Most likely there is some other limitation (in the model) that precludes better GPU utilization.
So, of 1. and 2., I think only 1. is concerning here :)
Could you create a minimal, verifiable, complete example to help reproduce the problem?
For example, I tried the following  - https://gist.github.com/asimshankar/2d65be7ef53f1888cf7dc93758a565c9 - which suggests to me that the change is working (in that it is respecting options provided to the ConfigProto message).
In particular, note that without the setPerProcessGpuMemoryFraction() line, nvidia-smi shows that the JVM process has allocated all ~12GB of GPU memory:
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0    241863      C   /usr/lib/jvm/java-8-openjdk-amd64/bin/java 11615MiB |

but with that line, it allocates ~370MB:
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0    241613      C   /usr/lib/jvm/java-8-openjdk-amd64/bin/java   371MiB |

A way to reproduce the problem will be helpful, otherwise it may be hard to figure out what's going on in your particular setup. Thanks!

@asimshankar I managed to reproduced the results of your gist. I found out that I was using a wrong version of the tensorflow-jni library... . It actually works well now.
Thank you for your help.
Looking forward to see the version in a stable release !


ENH Improve deprecation logging

DO NOT SQUASH - There are two distinct commits for two distinct changes, but both are required in this PR.
Parent issue

https://github.com/silverstripe/silverstripe-framework/issues/10608

I'll deal with the CI failures if you're happy with the changes here so far

Guess who had the defaults back to front, because they didn't change them when the variables changed from suppress to show? 😅

Need to make sure the any of the following work as expected for .env: 1, true, on, 0, false, off - they need to work not matter if the casing or whether or not they're surrounded by quotes.

Framework doesn't support false as a value, because Environment::getEnv() returns false if there is no set value.
I've added in a new method Environment::hasEnv() to get around this limitation.

Added that new API in its own commit - do not squash.

When calling deprecated code in phpunit, this PR will have no effect, which I'm fine with.
+1 to this, ideally automated tests are the very first place you'll run into these deprecation warnings and they should be dealt with then. I'd go as far as to say it's a feature that deprecation warnings show up there!

Self merging as already approved and just confirmed offline with @emteknetnz that it's good to merge.


improvement: driver deployer shouldn't cleanup previous deployment if Kubernetes version changes

Before this PR:

Deploy Longhorn master-head
Upgrade Kubernetes
Crash longhorn-driver-deployer pod
Observer that the longhorn-driver-deployer clean up and redeploy csi-attacher/csi-provisioner/csi-resizer/csi-snapshotter/longhorn-csi-plugin/Longhorn csidrivers object. This is:

Unnecessary as these components are still up to date
Even problematic as during this period Kubernetes cannot perform volume attach/detach/mount/unmount thus prevent kubernetes from draining the node

After this PR:
longhorn-driver-deployer does not clean up and redeploy csi-attacher/csi-provisioner/csi-resizer/csi-snapshotter/longhorn-csi-plugin/Longhorn csidrivers object after Kubernetes upgrade
Additional context:
The old behavior of longhorn-driver-deployer was necessary when upgrading Kubernetes from v1.11 to v1.12 (ref https://github.com/longhorn/longhorn/issues/426). However, it is no longer necessary now as we no longer support these Kubernetes versions
longhorn/longhorn#5474

@mergify backport v1.5.x v1.4.x


Add links on README.md to current Roadmap

I noticed that there are no Milestones or Projects here in Github being utilized.
Our project was looking for the current Roadmap and only saw past Roadmaps under the Wiki.
It would be useful for other projects (like OpenRefine) looking to utilize react-virtualized to have a sense of where future development is going, at least with basic Milestones or some Roadmap or Project plan captured somewhere, if not in Github itself.

Issue #658 is about all I have for roadmap. Planning roadmaps is a lot of work and I'm not really able to keep up with questions and issues as it is lately.
If you'd like to help, I'd love for you to join the Slack channel, message me to add you to the maintainers group, and then help coordinate things. 😄
Realistically though, I don't have the time to create a roadmap.

@bvaughn thanks, that issue is enough info for now.


Is this bug in safari fixed in Dexie?

https://twitter.com/feross/status/1404568122158313474

Yes, I use @jakearchibald's workaround in latest beta.
npm install dexie@next


A another step towards library

close #37
TODOs:

[x] refactor resolver by taking submodules into consideration
[x] reuse compiled core to speed up compilation
[x] make recompile decision with a file dependency graph
[ ] serialize/deserialize AyaBinOpSet

Suggestion: migrate the success tests to make it a package

My reviews are mostly questions, so if you believe I didn't understand anything, just lmk

bors r+


Ambari spelling error

https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-hadoop-use-blob-storage#identify-storage-path-from-abmari

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 4855a93d-b0e1-0656-b28c-d422b5bc7bb7
Version Independent ID: c3733aca-47ce-bf8a-556e-198345ecb755
Content: Query data from HDFS-compatible Azure storage - Azure HDInsight
Content Source: articles/hdinsight/hdinsight-hadoop-use-blob-storage.md
Service: hdinsight
GitHub Login: @hrasheed-msft
Microsoft Alias: hrasheed

@gavinaguiar Thank you for feedback and bringing this to our notice. At this time we are reviewing the feedback and will provide an update as appropriate

@gavinaguiar I have submitted a PR#103489 which should fix the typo. Once the reviewer reviews and merge it, doc will reflect the changes.
Thanks much for pointing this out. :)


CloudTrail load: Error on requestParameters.overrides.containerOverrides.environment due to varied type

We are seeing some errors for CloudTrail on es-loader with object mapping for [requestParameters.overrides.containerOverrides.environment] tried to parse field [environment] as object, but found a concrete value. This error seems to come from CloudTrail entries where there is "environment": "HIDDEN_DUE_TO_SECURITY_REASONS" instead of an object/hash.
This is happening with a deployment of this sample at v2.10.2a, on AWS-managed OpenSearch 2.11.
I see this key for a couple actions, snippets from examples below.
From Batch / Step Functions SubmitJob (no error):
            "requestParameters": {
                "jobName": "Utility",
                "containerOverrides": {
                    "command": [
                        "bin/echo"
                    ],
                    "environment": [
                        {
                            "name": "MANAGED_BY_AWS",
                            "value": "STARTED_BY_STEP_FUNCTIONS"
                        },
                        {
                            "name": "JOB_DATA",
                            "value": ""
                        }
                    ]
                },

From ECS RunTask (error):
            "requestParameters": {
                "platformVersion": "LATEST",
                "overrides": {
                    "containerOverrides": [
                        {
                            "name": "default",
                            "command": [
                                "bin/echo"
                            ],
                            "environment": "HIDDEN_DUE_TO_SECURITY_REASONS"
                        }
                    ],
                    "cpu": "2048",
                    "memory": "4096"
                },

+1
Experiencing the same issue.


Uncaught TypeError: Cannot read property 'toString' of undefined at MediaRecorder.mediaRecorder.onerror (RecordRTC.js:1990)

Hi, Muaz Khan
I have an issue in chrome.
Uncaught TypeError: Cannot read property 'toString' of undefined at MediaRecorder.mediaRecorder.onerror (RecordRTC.js:1990) .
previously it worked fine. Can you help me solving this issue. This is the code where i'm getting the error. The mediaStream is stopped recording.
    mediaRecorder.onerror = function(error) {
        if (!config.disableLogs) {
            // via: https://w3c.github.io/mediacapture-record/MediaRecorder.html#exception-summary
            if (error.name.toString().toLowerCase().indexOf('invalidstate') !== -1) {
                console.error('The MediaRecorder is not in a state in which the proposed operation is allowed to be executed.', error);
            } else if (error.name.toString().toLowerCase().indexOf('notsupported') !== -1) {
                console.error('MIME type (', config.mimeType, ') is not supported.', error);
            } else if (error.name.toString().toLowerCase().indexOf('security') !== -1) {
                console.error('MediaRecorder security error', error);
            }

            // older code below
            else if (error.name === 'OutOfMemory') {
                console.error('The UA has exhaused the available memory. User agents SHOULD provide as much additional information as possible in the message attribute.', error);
            } else if (error.name === 'IllegalStreamModification') {
                console.error('A modification to the stream has occurred that makes it impossible to continue recording. An example would be the addition of a Track while recording is occurring. User agents SHOULD provide as much additional information as possible in the message attribute.', error);
            } else if (error.name === 'OtherRecordingError') {
                console.error('Used for an fatal error other than those listed above. User agents SHOULD provide as much additional information as possible in the message attribute.', error);
            } else if (error.name === 'GenericError') {
                console.error('The UA cannot provide the codec or recording option that has been requested.', error);
            } else {
                console.error('MediaRecorder Error', error);
            }
        }
        (function(looper) {
            if (!self.manuallyStopped && mediaRecorder && mediaRecorder.state === 'inactive') {
                delete config.timeslice;

                // 10 minutes, enough?
                mediaRecorder.start(10 * 60 * 1000);
                return;
            }

            setTimeout(looper, 1000);
        })();

        if (mediaRecorder.state !== 'inactive' && mediaRecorder.state !== 'stopped') {
            mediaRecorder.stop();
        }
    };

Hi,
I appreciate your quick response,Thanks!!
Yes, I tried the above code snippet but the recording is stopped anyway. I will share the screenshot of console. please take a look into it. Thank you in Advance.

Please share complete demo. I want to check if MediaStream is still active, including all its tracks.

This is my code below.
Here i'm using 2 functions one is to start recording and one is stop. And i'm using jquery version 3.3.1 and browser is chrome Version 58.0.3029.110 (64-bit). Os: ubuntu 14.04
var mediaStream;
window.start = function(){
var mediaConstraints = { video: true, audio: true };
function errorCallback(error) {
console.log("error while using mediaDevices",error);
}
function successCallback(stream) {
    mediaStream = stream;
    var options = {
      mimeType: 'video/webm\;codecs=h264',
      bitsPerSecond: 524288
    };
    cameraPreview.srcObject = stream;
    cameraPreview.play();
    recordRTC = RecordRTC(stream, options);
    recordRTC.startRecording();
}
navigator.mediaDevices.getUserMedia(mediaConstraints).then(successCallback).catch(errorCallback);
    timeOut = setTimeout(function() {
        stop();
    }, 2 *60 * 1000)

}
window.stop = function(){
mediaStream.stop();
recordRTC.stopRecording(function (audioVideoWebMURL) {
var recordedBlob = recordRTC.getBlob();
recordRTC.getDataURL(function(dataURL) {
//here i'm sending dataURl to the backend to store.
});
}


JSON allow values to overwrite eachother

{
  "a" : 12
  "a" : 24
}

should become
{
  "a" : 24
}

Fixed in branch bug/6


Add HTTP header validation to http_validator

Tried to keep the same semantics with existing success_status_codes
and failure_status_codes (first check for fail part).
Did not add value checking now because there is a need for more discussion.
For example header can have multiple values (array).
Thought about reusing the already defined Header
(in probes/http/proto/config.proto) but because of future compatibility
and no value check at this time I created a new one.

@benner, As discussed in https://github.com/google/cloudprober/issues/529#issuecomment-760661294, let me know when you update the PR.

@benner, As discussed in https://github.com/google/cloudprober/issues/529#issuecomment-760661294, let me know when you update the PR.

Added header value regexp functionality

Thanks for the changes, @benner. I've gone ahead and committed your change: https://github.com/google/cloudprober/commit/9674ae27076360098bc178330cc05191cb17ee89 (#561), following the process described at:
https://github.com/google/cloudprober/blob/master/CONTRIBUTING.md#source-of-truth-sot-and-commit-process
Closing this PR now. Can you please send me another PR for documentation and examples changes? Those changes don't follow the above commit process -- I can commit them directly.

OK. Sure


shared src/main/resources

This is a small change which enables us to delete a lot of duplicated files.

This is great. Could you do the same for src/main/resources and src/test/resources in chess-listener?

@burtbeckwith

This is great. Could you do the same for src/main/resources and src/test/resources in chess-listener?

https://github.com/micronaut-projects/micronaut-guides/pull/404/commits/ab912ed4c74eb363a3d12e3be5667b6784cbe2ff


File size increases almost 300%

I have been running into the issue where if a user uploads an jpeg and we store it via Image::make(img-data) and then $image->save(....); The file size increases almost 300%. For example, I uploaded a 323mb file and it uploaded as a 900mb file.
Does saving it out at the default 90% quality increase file size even if the image was optimized before being uploaded?

Pictures of code?
Greetings

Hi,
With Glide :  https://github.com/thephpleague/glide
when I run :
$server->outputImage($path, []);

My image size too increases around 200% or 300%.
Glide related code is here :
https://github.com/thephpleague/glide/blob/master/src/Server.php#L461
An example image :
https://68.media.tumblr.com/80f9f86a4db07102a44851b46aa01788/tumblr_opajjcPvjs1vwae5go1_1280.jpg

I'm running into the same issue where my images filesize is being increased by 300% when using $image->save(....); - does anyone know whats going on?

I have the same issue:
public static function resizeWithFallback($img, $width, $height, $new_name = false, $quality = 90){

    $new_name = (!empty($new_name)) ? $new_name : $img;

    $imgHandler = Image::make($img);
    $imgHandler->orientate();

    $old = array(
        'filesize'  => $imgHandler->filesize(),
        'width'     => $imgHandler->width(),
        'height'    => $imgHandler->height()
    );

    $imgHandler->widen($width, function ($constraint) {
        $constraint->aspectRatio();
        $constraint->upsize();
    })->heighten($height, function ($constraint) {
        $constraint->aspectRatio();
        $constraint->upsize();
    });

    $imgHandler->save($new_name, $quality);

    $status = array(
        'status' => true,
        'filename' => $new_name
    );

    if(file_exists($new_name) && $imgHandler->filesize() > $old['filesize'] && ($old['width'] == $imgHandler->width() && $old['height'] == $imgHandler->height())){
        if($new_name != $img) {
            unlink($new_name);
        }

        $status = array(
            'status' => false,
            'filename' => $img,
            'debug' => array(
                'old' => $old,
                'new' => array(
                    'filesize'  => $imgHandler->filesize(),
                    'width'     => $imgHandler->width(),
                    'height'    => $imgHandler->height()
                )
            )
        );
    }

    return $status;

}

I'm trying this code, what do you think about?

This is still an issue, it is not handling the file as I would expect... Here is some example code and steps to repeat (really.. save any image with 100 quality)

Grab any image
Run script below

<?php

$fileopen = __DIR__.'/testimage.jpg';
$filesave = __DIR__.'/testimage_saved.jpg';

// report file size
clearstatcache();
$filesize = filesize($fileopen);
echo "Current file size: {$filesize}\n";

// load image and save it immediately
require 'vendor/autoload.php';
use Intervention\Image\ImageManager;
$manager = new ImageManager(array('driver' => 'gd'));
$image = $manager->make($fileopen);
$image->save($filesave, 100);

// report file size
clearstatcache();
$newsize = filesize($filesave);
echo "Current file size: {$newsize}\n";

// report increase
$increase = round(($newsize / $filesize) * 100, 5);
echo "Increase = {$increase}%\n";

Output:
Current file size: 1138320
Current file size: 2960342
Increase = 260.06237%

Expected: File size to be the same, or a very very small difference
Result: File size increases dramatically

Not sure on cause, maybe file system? (I'm on a mac), maybe GD library? Who knows.
What this means is that if you do: $image->save($filesave, 80); it will be "80% quality of whatever quality intervention is currently at and not 80% quality of the original?
Output with 80%
Current file size: 1138320
Current file size: 1261386
Increase = 110.8112%

I have the same problem. I solve it by using quality 60 - 80.

I have the same problem. I solve it by using quality 60 - 80.

Jan 9, 2023. The issue still exists. Driver: Imagick. It is enough to save at 100% quality to reproduce. Any suggestions?

I have been running into the issue where if a user uploads an jpeg and we store it via Image::make(img-data) and then $image->save(....); The file size increases almost 300%. For example, I uploaded a 323mb file and it uploaded as a 900mb file.
Does saving it out at the default 90% quality increase file size even if the image was optimized before being uploaded?

Facing same issue but not suitable solution founded

The same issue (


[Bug] request localhost:3000  failed http_code=500

My steps are as follows
step1:
git clone https://github.com/solana-labs/explorer.git
cd explorer
pnpm i
npm dev

step2:
curl http://localhost:3000
I get result
...
GET / 500 in 8745ms
...
Attached are specific logs
solana.err.log

Please upgrade your pnpm version to 9.10.0 or higher
Then reset your changes & restore the lockfile then re-install.
git stash
pnpm i
git stash pop
pnpm dev


Package source needed to be added

The included PowerShell does not work for first time users due to the error "Unable to find package". None of the nuget install commands work due to the error. To resolve, had to run the following in PowerShell first. This adds the nuget.org source to the user's scope. See https://docs.microsoft.com/en-us/nuget/reference/cli-reference/cli-ref-sources
./nuget sources Add -Name "nuget.org" -Source https://api.nuget.org/v3/index.json

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 9d7d5f36-11fc-9ac5-778b-c03823bbb072
Version Independent ID: 814e151a-2018-285a-697f-f929a73957d4
Content: Download tools from NuGet (Developer Guide for Dynamics 365 Customer Engagement)
Content Source: ce/customerengagement/on-premises/developer/download-tools-nuget.md
Service: crm-online
GitHub Login: @JimDaly
Microsoft Alias: jdaly

@stonekw  The sample PowerShell script in the topic needed updating, so grab a new copy and try it again. Let me know if you are still having trouble.

@stonekw  We are not seeing such a problem reported from anyone else. Make sure to use Windows PowerShell to run the script.


Fix document since PR#313

@carsonwang @kira-lin I forgot to update the doc since the merge of this PR: https://github.com/oap-project/raydp/pull/313

Thanks


Store libraries in lib folder

Currently it is stored in libs which is not by convention

Duplicate of #32


TypeScript colorizer confused by comments after colon in class member function signature with Generic return type

🔎 Search Terms
color
comment
white
Promise
async
function
type
declaration
🕗 Version & Regression Information

I was unable to test this on prior versions because no idea when this started

⏯ Playground Link
https://vscode.dev
💻 Code
import os from 'os';

type Test<T> = T;

export class test {
  fine(): Test<void> {
    const test = 0;
  }
  ok(): /**/ Test<void> {
    const test = 0;
  }
  bad(): // any comment with a new line breaks colorizer for a function
  Test<void> {
    const test = 0;
    os.hostname();
    true ? false : undefined;
    throw new Error('');
    return;
  }
  fixed() // Not broken this way. Compare colorization with the bad function.
  : Test<void> {
    const test = 0;
    os.hostname();
    true ? false : undefined;
    throw new Error('');
    return;
  }
  alsoBad(): /* can be a /* comment too, but formatters usually remove the new line unless it is really long */
  Test<void> { // all but { colors are different
    const test = 0; // const is white
    os.hostname(); // all alpha is white
    true ? false : !!undefined; // true and false are different
    throw new Error(''); // throw is different, but not new or Error
    return; // return is different
  }
}

function ok(): // not broken outside of class
Test<void> {
  const test = 0;
}

🙁 Actual behavior
In the class functions that have comments after the : in the signature but before the actual Generic type, I noticed many differences from normal:

Type coloring is different
const is white
global variables are white
true and false are different in ternary
throw is different, but not new or Error
return is different

🙂 Expected behavior
Consistent colorizing, no matter the comments
Additional information about the issue
Does not reproduce on GitHub or playground's colorizer.
Reproduces quickly and easily on vscode.dev.
I wasn't sure where to post this issue. It seem like a very TypeSript specific issue, but maybe this should be a VS Code bug...

I’m not sure if this is one, but there are a lot of known issues and known limitations at https://github.com/microsoft/TypeScript-TmLanguage/issues.


[Orient DB 2.2.0-Beta] OCaseInsensitiveCollate is not working anymore

Hi,
we were running on Orient DB 2.1.2. for quite some time now and decided to upgrade to 2.2.0-beta. Everything is up and running and we experienced some performance improvements (which is nice), but OrientDB seems to ignore the OCaseInsensitiveCollate setting on vertex properties for some reason. There's no compile error or anything, just our search fields (which forward to orient db queries) are case sensitive all of a sudden, which is bad for the user experience in our application.
Has there been any change in the collation system between 2.1.2 and 2.2.0-beta?

Hi @Alan47
Thanks for reporting, having users feedback on beta releases is critical to have a stable final.
There was no explicit change in this, so I guess it's a regression. It will be addressed ASAP.
It would be great if you could provide a simple test case to reproduce it.
Thanks
Luigi

Hi,
thanks for the very fast reply, good to know that there's someone on "the other side" :)
Here's a concrete, stand-alone JUnit test case that shows the error:
import static org.junit.Assert.*;

import org.junit.Test;

import com.orientechnologies.orient.core.collate.OCaseInsensitiveCollate;
import com.orientechnologies.orient.core.metadata.schema.OType;
import com.orientechnologies.orient.core.sql.OCommandSQL;
import com.tinkerpop.blueprints.Vertex;
import com.tinkerpop.blueprints.impls.orient.OrientGraphFactory;
import com.tinkerpop.blueprints.impls.orient.OrientGraphNoTx;
import com.tinkerpop.blueprints.impls.orient.OrientVertexType;
import com.tinkerpop.blueprints.impls.orient.OrientVertexType.OrientVertexProperty;

public class CollationTest {

	@Test
	@SuppressWarnings("all")
	public void run() {
		// OrientGraphFactory factory = new OrientGraphFactory("plocal:" + testDirectory.getAbsolutePath());
		OrientGraphFactory factory = new OrientGraphFactory("memory:CollationTest");
		OrientGraphNoTx noTxGraph = factory.getNoTx();
		// create graph schema

		// This schema will be very simple:

		// class Person {
		// - firstName: OString (Collation: CI)
		// - lastName: OString (Collation: none)
		// }

		OrientVertexType tPerson = noTxGraph.createVertexType("Person");
		OrientVertexProperty pFirstName = tPerson.createProperty("firstName", OType.STRING);
		pFirstName.setCollate(new OCaseInsensitiveCollate());
		tPerson.createProperty("lastName", OType.STRING);

		// now, insert a person
		Vertex vPerson1 = noTxGraph.addVertex("class:Person");
		vPerson1.setProperty("firstName", "John");
		vPerson1.setProperty("lastName", "Doe");

		// not sure if this is requried at all, but it does no harm
		noTxGraph.commit();

		// try to find it with a query on the first name that uses
		// the wrong case (which should be ignored according to the schema)
		String query = "SELECT FROM Person WHERE firstName LIKE 'jo%'";
		Iterable<Vertex> vertices = (Iterable<Vertex>) noTxGraph.command(new OCommandSQL(query)).execute();
		// count the size of the iterable
		int size = 0;
		for (Vertex vertex : vertices) {
			size++;
		}
		// well, we expect our person to be in the result
		assertEquals("Failed to find John Doe!", 1, size);
	}

}

The case above is working perfectly fine with OrientDB 2.1.2, but fails with 2.2.0-beta. The behaviour is consistenly reproducible.

Great! Thank you very much
Luigi

You're welcome. Hopefully this little issue will be sorted out soon.

Hi @Alan47
This issue was fixed months ago, v 2.2 GA does not have this problem.
I ran your test to double check and it passed correctly.
I'm closing this, please reopen if you still see similar problems.
Thanks
Luigi


⚠ Bofrost.fr has degraded performance

In ff7c567, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 1972 ms

Resolved: Bofrost.fr performance has improved in e194d4c after 31 minutes.


Fix test for fractional-hour offsets, fixes #2389

When this test was switched from .zone() to .utcOffset(), the sign of the value changed, which broke the test for time zones that use fractional-hour offsets, such as UTC+05:30.  We now need to round down instead of up.
Fixes #2389.

Also fixes a large number of duplicate issues under 307.6 or 2337.6.

Good catch! Thanks.

Merged in b3c812eeeb4315e17d7ba5474ae60dfcac21e880


bug tool not found with shift_longitudes on climate.usegalaxy.eu

python3: can't open file '/opt/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/climate/shift_longitudes/5708a3a46f2e/shift_longitudes/shift_lon.py': [Errno 2] No such file or directory
I am not sure what is wrong but I used  $tool_directory/shift_lon.py but it appears to be different on climate.usegalaxy.eu
@bgruening Any idea?

Same problem with cds_essential_variability. Maybe $tool_directory is not correct?
python3: can't open file '/opt/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/climate/cds_essential_variability/8b30be2cef81/cds_essential_variability/ecv_retrieve.py': [Errno 2] No such file or directory

try again in 5min, I forgot to sync ...

both tools ...

It seems to work now so we close this issue.


[VS Code] Cannot read property 'sessionId' of undefined

Error:
Cannot read property 'sessionId' of undefined
Steps to Reproduce:

Version Data

extensionName
VSLS

extensionVersion
0.3.897

protocolVersion
2.2

applicationName
VSCode

applicationVersion
1.27.2

platformName
MacOS

platformVersion
18.0.0

Thanks for reporting this. Can you please run the command "Live Share: Export Logs" and attach the zip here or on issue #1172 or send it to<EMAIL_ADDRESS>Also, any other information you can provide will help us investigate this. What were you doing at the time that this happened?
I am going to close this issue as a duplicate of #1172.


Add link to shared TPV database to readme files

closes: https://github.com/galaxyproject/total-perspective-vortex/issues/37

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 94.852%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
688

Relevant Lines:
718

💛  - Coveralls

Thanks


Add cmp and dev for I²C based SGP sensors

Like #1 but for SGP.
I'm using the generic IC symbol here.

This is actually wrong, we need the 5-pin symbol :stuck_out_tongue:

There are two branches in this repo, one named sgp30 and the other named sgp. Are both meant to be merged?

Also the PR should be rebased, since it is quite behind :slightly_smiling_face:

The sgp30 branch can be ignored.
@rnestler branch is now updated with the fixed component, as well as with two devices for SGP30 and SGPC3. Maybe you can take a look now?

I'm not sure if the generic DFN package works optimally for the SGPs. @ubruhin what do you think?

Yes we should use a SGP specific package anyway because of the very special sensing "hole" at the top...

Ah, good point. We discussed that some time ago, but I forgot to add a TODO note to this PR :slightly_smiling_face:

Devices updated with custom package. The package was generated with https://github.com/LibrePCB/librepcb-parts-generator/pull/70.


[docs] Support demos with side effect imports

[x] I have followed (at least) the PR section of the contributing guide.

In the X repo there's a demo that imports a locale from dayjs, but this import is not used along the code. This causes the demo to crash because: 1. the demos are loaded with react-runner; which 2. requires all imports to be specified at front, but since the regex that searches the imports to be loaded doesn't match imports not used along the code the module can't be found.

Side note: Today I learned that this kind of import is called "side effect import" by MDN.
Related to https://github.com/mui/mui-x/pull/6369#pullrequestreview-1181335928

I don't understand. It seems that by only https://github.com/mui/mui-x/pull/6864 the monorepo in master solved the problem:

@m4theushw I covers a bit of the why in

Now, it uses the v1 of the live edit, the v2 makes the demo work when no edits are done. So, the bug will be less visible. TL:DR updating the @mui/monorepo will fix a large chunk of the issue.

https://github.com/mui/mui-x/pull/6369#pullrequestreview-1181335928
Try to make an edit in the field, it will still be broken.

For reference, this is what we use in toolpad: https://github.com/mui/mui-toolpad/blob/3ef124ab46e4ba0682665b6cc93b87d6fc0434d9/packages/toolpad-app/src/utils/strings.ts#L99
We have a very basic test suite that you can copy if you want https://github.com/mui/mui-toolpad/blob/3ef124ab46e4ba0682665b6cc93b87d6fc0434d9/packages/toolpad-app/src/utils/strings.spec.ts#L3

Is the regexp ready or do we need to wait ?

One thing to note: the regex from Toolpad is matching spaces when the import name is "module-2\s\s"

We were specifically looking to match on any import specifier (e.g. anything between ""). The idea being that it's up to subsequent validation to verify whether it can resolve to a module or not.


Fixes #60 -- broken subscriber deletion

This fixes #60. In summary:

The callback in subscriber.coffee is the opposite to what it should be. async.serial expects null as the first argument of the 'task' functions in order to proceed with the remaining tasks. The code returned @id on success, hence only the first subscriber of each 'page' was processed.

When deleting in chunks from a data source the code should not increment the deletion 'pointer' by page * perPage offset as the code did, but a) either delete records from the end and 'decrement' the pointer, or b) delete from 0 up to 100 (or as much as is left, whichever applies). I implemented (b) in this case, as it was shorter and it seems to work well.

(1) requires some additional changes in api.coffee too. This kind of messes up error reporting, but I guess you could improve this if you wanted -- say, by providing another layer of encapsulation and a richer signature to the callbacks.

I also bumped the node-redis dependency to ~1.0.0. I guess we could, ideally, use newer versions of some of the dependencies that remain backwards compatible with this codebase.

It seems to work well, but I haven't tested the code much. Also, note that I haven't looked at/modified the tests.

Thank you for this PR, it looks promising. I can't merge it while tests can't pass. Do you plan on fixing the tests as well?

I will at some point. This PR is flawed -- hadn't noticed you used the same iterator for accessing and deleting. I will revise and push a fixed PR at some point, including tests.


Parsing errors due to compiler options not loaded from zapp file

Z Open Editor version: 3.3.1
Editor Platform: Visual Studio Code
Editor Platform Version: 1.84.2
Operating System: Almalinux 9.2
Java Version: OpenJDK 11.0.20
Logs attached: yes

Problem Description
In some instances, the compiler options (PLI or COBOL) from the zapp file are not loaded, which causes parsing errors. In the log, the "compilerOptions" passed to the language client is blank:
2023-11-11T17:33:11.700+01:00 DEBUG: AbstractZLanguageClient.initializeLanguageClient: Started initializing a new PL1 language client with these initialization options: {"publishDiagnostics":true,"includeFileExtensions":[".inc",".include",".plinc",".pcx",".pci"],"compilerOptions":""}.
2023-11-11T17:33:11.733+01:00 DEBUG: ZAPP: A valid ZAPP file was found in "/home/oseam/test01".
2023-11-11T17:33:11.822+01:00 INFO : IBM Z Open Editor extension has (re)started.

It might be because the zapp file is loaded after the initialization of the language client?
Detailed steps for reproducing the problem:

Add a compiler options in the zapp file, for example:

propertyGroups:
  - name: pl1-settings
    language: pl1
    compilerOptions: "NOT(^), OR(!)"

Open the following test program in a new tab:

 TEST: PROC() OPTIONS(MAIN);

    DCL VAR1 BIT INIT('0'B);
    DCL VAR2 BIT INIT('0'B);

    IF VAR1 ! VAR2 THEN DO;
    END;

 END TEST;

Restart vscode (which will reopen the workspace with the test program)

Observed behavior
Parsing errors, due to the character "!" not recognized.
Expected behavior
No errors.

A workaround would be to close your editor tabs and open them again.

Fixed in v3.3.3. Can you try @oseam87?


[bug] failed to run custom build command for webview2-com-sys v0.11.0

Describe the bug
program can run.
but the configuration file reports an error, and the red looks uncomfortable

Reproduction
No response
Expected behavior
No response
Platform and versions
yarn run v1.22.10
$ tauri info

Operating System - Windows, version 10.0.18362 X64
Webview2 - 98.0.1108.50
Visual Studio Build Tools:
   - Visual Studio ���ɹ��� 2019

Node.js environment
  Node.js - 17.4.0
  @tauri-apps/cli - 1.0.0-rc.2
  @tauri-apps/api - 1.0.0-rc.0

Global packages
  npm - 8.3.1
  pnpm - Not installed
  yarn - 1.22.10

Rust environment
  rustc - 1.58.1
  cargo - 1.58.0

Rust environment
  rustup - 1.24.3
  rustc - 1.58.1
  cargo - 1.58.0
  toolchain - stable-x86_64-pc-windows-msvc

App directory structure
/.git
/build
/node_modules
/public
/src
/src-tauri

App
  tauri.rs - 1.0.0-rc.0
  build-type - bundle
  CSP - default-src 'self'
  distDir - ../build
  devPath - http://localhost:3000/
  framework - React
Done in 7.30s.

Stack trace
No response
Additional context
No response

This is an issue with your LSP (rls), this not an issue with tauri and we recommend that you switch to rust-analyzer.


NS7 - Nativescript Angular Navigation issue

Environment
package.json contents
 "dependencies": {
    "@akylas/nativescript-sqlite": "^3.3.11",
    "@angular/animations": "~11.1.0",
    "@angular/common": "^11.1.0",
    "@angular/compiler": "^11.1.0",
    "@angular/core": "^11.1.0",
    "@angular/forms": "^11.1.0",
    "@angular/platform-browser": "~11.1.0",
    "@angular/platform-browser-dynamic": "~11.1.0",
    "@angular/router": "~11.1.0",
    "@mapo80/nativescript-ngx-shadow": "^7.0.1",
    "@mashdog/nativescript-accordion": "^7.0.6",
    "@nativescript-community/appurl": "^1.4.2",
    "@nativescript-community/ui-material-bottomsheet": "^5.2.3",
    "@nativescript-community/ui-share-file": "^1.2.1",
    "@nativescript/angular": "11.0.1",
    "@nativescript/core": "^7.2.1",
    "@nativescript/firebase": "^11.1.3",
    "@nativescript/geolocation": "^7.0.0",
    "@nativescript/local-notifications": "^5.1.0",
    "@nativescript/social-share": "^2.0.1",
    "@nativescript/theme": "^3.0.1",
    "@nativescript/types": "^7.1.0",
    "@nativescript/ui-charts": "0.0.5",
    "@nstudio/nativescript-checkbox": "^2.0.4",
    "@nstudio/nativescript-loading-indicator": "^4.0.0",
    "@proplugins/nativescript-drop-down": "file:src/local-packages/proplugins-nativescript-drop-down-9.0.2.tgz",
    "@proplugins/nativescript-platform-css": "^2.0.0",
<EMAIL_ADDRESS>"^18.6.4",
    "copy-webpack-plugin": "4.6.0",
    "core-js": "3.6.5",
    "d3-ease": "^2.0.0",
    "jwt-decode": "^3.1.2",
    "nativescript": "^7.2.0",
    "nativescript-barcodescanner": "^4.0.1",
    "nativescript-google-maps-sdk": "^3.0.1",
    "nativescript-imagepicker": "~7.1.0",
    "nativescript-menu": "^1.1.6",
    "nativescript-microsoft-appcenter": "^2.0.0",
    "nativescript-permissions": "^1.3.11",
    "nativescript-qr-generator": "^2.0.0",
    "nativescript-ripple": "^4.0.1",
    "nativescript-shimmer": "file:src/local-packages/nativescript-shimmer-7.0.0.tgz",
    "nativescript-ui-autocomplete": "^7.0.2",
    "nativescript-ui-listview": "^9.1.0-rc.1",
    "nativescript-ui-sidedrawer": "^9.0.3",
    "nativescript-windowed-modal": "7.0.0",
    "raw-loader": "^1.0.0",
    "reflect-metadata": "~0.1.12",
    "rxjs": "^6.4.0",
    "tslib": "1.10.0",
    "zone.js": "~0.10.2"
  },
  "devDependencies": {
    "@angular-devkit/build-angular": "~0.1101.1",
    "@angular/cli": "^11.1.1",
    "@angular/compiler-cli": "^11.1.0",
    "@nativescript/android": "7.0.1",
    "@nativescript/ios": "^7.2.0",
    "@nativescript/schematics": "^11.0.0",
    "@nativescript/webpack": "~4.0.0",
    "@ngtools/webpack": "10.1.3",
    "@types/d3-ease": "^2.0.0",
    "@types/lodash": "^4.14.165",
    "codelyzer": "6.0.0",
    "node-sass": "^4.7.1",
    "sass-loader": "^9.0.1",
    "ts-node": "~8.3.0",
    "tslint": "~5.19.0",
    "typescript": "^4.1.3"
  },

Describe the bug
Navigation on android stops working when navigation after the app resumes. I implemented opening the app from an external link which holds some data in the query parameters. Dependant on those parameters I navigate the app to a specific part of the application. The specific case is when the app is running, I suspend it (put it in background) to get to the link, tap the link (which opens the app through an intent-filter), then I handle the data coming from the link. After that I navigate the app (I have to call ngZone.run() to navigate otherwise it doesn't work.
e.g. using the handleOpenURL from '@nativescript-community/appurl',
I call
this._ngZone.run(
                    () => {
                        this._router.navigate(['/auth',  'reset-pw'],
                       {
                           replaceUrl: true, state: { resetCode: code, viewType: 'reset-password'}
                       });
                    }
                );
});

Ill paste the routing modules just so you can see the structure.
auth-routing.module.ts
const routes: Routes = [
    {path: '', redirectTo: 'login'},
    {path: 'reset-pw', redirectTo: 'reset-password'},
    { path: 'login',  component: LoginComponent},
    { path: 'register', component: RegisterComponent },
    { path: 'reset-password', component: ResetPasswordComponent }

];

@NgModule({
    imports: [NativeScriptRouterModule.forChild(routes)],
    exports: [NativeScriptRouterModule]
})
export class AuthRoutingModule { }

app-routing.module routes:
const routes: Routes = [
    {path: '', redirectTo: 'agro-stanice', pathMatch: 'full'},
    {path: 'evidencije', canLoad: [AuthGuard], loadChildren: () => import('./evidencije/evidencije.module').then((m) => m.EvidencijeModule)},...
    {path: 'auth', loadChildren: () => import('./auth/auth.module').then((m) => m.AuthModule)},
    {path: 'navigation', canActivate: [AuthGuard], component: NavigationComponent}
];

after navigating to the reset-pw, no navigation seems to work, back button does nothing, routerExtensions.navigate(...)
does nothing, calling goBack on routerExtensions also does nothing.
This behaviour does not happen if the app is closed when I click the link, and this behaviour does not happen if I navigate to ['/auth'] only
e.g.
this._ngZone.run(
                    () => {
                        this._router.navigate(['/auth'],
                       {
                           replaceUrl: true, state: { resetCode: code, viewType: 'reset-password'}
                       });
                    }
                );
});

I tried with redirects both on the main routing module and module specific routing module the issue persists.
EDIT: I tried subscribing to router events, they do trigger everytime I tap the buttton calling the routerExtensions.navigate but the app stays on the same page. Also I am using only one page-router-outlet in the app.component.html
        this._router.router.events.subscribe(
            (val) => {
                console.log(val)
            });

Hi,
I was going to post a comment saying I had the same problem - after spending too many hours - but I just checked one thing before and it seems like I solved my issue.
In the @nativescript-communite/appurl plugin, it says in the readme:

The android:launchMode="singleTask" tells the Android operating system to launch the app with a new instance of the activity, or use an existing one. Without this your app will launch multiple instances of itself which is no good.

I guess the router gets a bit lost if it isn't in singleTask.
So this is the line that saved me spending even more time:
<activity android:name="com.tns.NativeScriptActivity" ... android:launchMode="singleTask"...>

I hope this helps you !
PS: the navigation was kind of working when I was logging the component but it wouldn't display anything so it looked like it wasn't working.


checkout button does not show for magento 2.3

will you be able to debug it? thanks.

Hi @erobotics, unfortunately we are moving towards deprecating support for the Magento plugin as it hasn't captured enough audience for us to maintain it at this time. However, we may come back to it in the future. The php SDK however will be kept alive. Feel free to clone this repository to make modifications for your use, which it seems by now you may have, or post any PR's you would like to fix for the php SDK to share with the community. Sorry we couldn't be more helpful at this time.


Index Carbon Gatsby Theme

Repo: https://github.com/carbon-design-system/gatsby-theme-carbon
External docs: https://gatsby-theme-carbon.vercel.app
No Storybook? (No demo links?)
List of components to index: https://github.com/carbon-design-system/gatsby-theme-carbon/blob/main/packages/gatsby-theme-carbon/index.js
(Compare list with docs site side nav.)

@ljcarot
https://github.com/carbon-design-system/gatsby-theme-carbon
The Carbon Gatsby Theme has 329 stars.
https://github.com/carbon-design-system/gatsby-theme-carbon/network/dependents
314 repositories depend on it.
https://www.npmtrends.com/gatsby-theme-carbon
Averages around 1,200 weekly downloads.
The Carbon Gatsby Theme is a library that install in a Gatsby site that includes default configurations and functionality. That functionality includes React components that are built using the Carbon React components (component extensions).
The Carbon Gatsby Theme is part of our ecosystem. As long as it's part of our ecosystem, we can prevent confusion by properly indexing the library and its assets so our users know the difference between the two.
If it's too confusing, then we have bigger issues to address in the catalogs when viewing assets that initially appear to be similar. E.g.:

@mattrosno The reasons that I have for not including it is 1) it's no longer maintained and 2) if used within IBM, it does not follow the IBM standards for websites so it exacerbates a problem with proliferation of micro sites.    In the past DPO maintained it, but Vince is on a different project now and we don't have resources to maintain it.  While it may persist as a community supported library beyond IBM, I just don't want to dial up the volume on Gatsby within Platform.

@mattrosno to schedule a call with squad and Carbon leads to determine indexing go/no-go.

No rush to index this for v1. Let's index this library and its components once we can show the status note on asset detail pages (v1.x design effort), and once we know what messaging we'd want to put in those status notes.
Also, this will be marked as "community maintained", although the Carbon core team will still be minimally involved given our dependency on the Gatsby theme for the sites that we maintain.


pubsub api

This is just a place holder to discuss api design and internal design of the pubsub layer. Some pointers:

CouchDB's change api. Thanks @tisba

HTTP chunked responses should handle this well. For web clients InfluxDB server can return JSONP.
HTTP overhead in this case is really not a concern, headers are sent once per connection.
@schmurfy I don't think that subscribing changes to db would really add overhead to engine. In case of no active subscribers there is no data to be sent/synchronize. In go it could be implemented as simple list of subscribers per series/database. Subscribers could listen on given series or regexp.

I was not entirely speaking about performance, part of why I think influxdb is really good is that they defined a limited (in a good way) scope and until now kept to it allowing to focus on the core and not utility features that may be useful... or not.
That's my opinion anyway :)

I think that  WAMP (Web Application Messaging Protocol) is worth looking into.  There is a Go implementation called Turnpike.
The biggest benefit is that there is a thriving ecosystem of WAMP libs, which would immediately be able to communicate with influx, without any additional work.

I'd like this to happen, and I could even give it a try with some help.
First of all I'd like to see how this should be implemented.
I'd distinguish three areas of work:
The "frontend", or transport
Here I would stick to HTTP, but with bidirectional streaming.
I would imagine a "/subscribe" route (similar to the "/query" one), where an HTTP request to the route would actually create the subscription channel, which could work in one of three ways (we should pick one):

The old "comet style" of streaming contents on an open HTTP connection
Web sockets
The new HTTP 2.0 full duplex streaming mode

In any case, a request on this route should create a "subscription" connection, with a reader and a writer on the server that received the request:

the reader should be used to accept subscription "commands" (more on that later)
the writer whould be used to publish the requested data

The "subscription object"
Each HTTP connection to the "/subscribe" route creates a "subscription" on the server, which is active as long as the HTTP connection is open.
Subscription "commands" (or "requests") received on a subscription channel should look like a query, but (IMHO) without the ability to group, or do other "stateful processing".
In practice they should be pure filters, with operators on the data point properties (the name, tags, and fields).
To be clearer: a subscription should be a predicate on an arbitrary data point.
Data points that pass the predicate should be published, and data points that don't pass it should be ignored.
The client should be able to modify this "predicate" (or filter) writing "commands" on the HTTP channel.
How the server would process "events" that must be published to subscriptions
This is the thing that made me think the most, and I'd like to know if my understanding of this issue is correct.
Infuxdb, internally, implements a "raft log", and every broker node participates in the raft.
This means (again, please correct me) that every committed data point will also "flow" through the log, through every broker.
And the clients connect to an arbitrary broker to write data points, so we can assume that clients will also subscribe to an arbitrary broker to subscribe to "events" (as described above).
If this is true, it would be "enough" for every broker to process every data point committed to the log in the following "logical" way:

for every open subscription channel, apply the filter to the data point
if it passes the filter, write it on the channel

I said "logical" way because this is a nested loop (subscriptions, and then filters), and maybe there is a way to store the filters and the subscriptions that makes this processing faster.
Also, individual subscriptions could be complex logical expressions that maybe could be optimized to fail faster (and waste less time) in the most common cases.
Anyway, this "subscription processing" could be asynchronous with respect to the log processing: it could be a goroutine started from the log processing, maybe even on a different physical thread to avoid slowing down the broker log.
What matters is that data points that pass the filter are written (streamed) on the subscription HTTP connection.
An important property of this approach should be that every data point that goes into the DB could be published, including those produces by continuous queries.
If my understanding is correct, this would happen because continuous queries push data into the log, which then distributes it to the data nodes (I hope I did not misunderstand this).
Help I would need to give it a try
First: I am, unfortunately, not committed to this.
It might be useful for me at work but before I am sure that this can "go somewhere" I cannot really spend much time on this.
And I have no go-lang experience at all :-/
That said, maybe with a bit of help I could give this a try.
I would mostly need the following:

a validation that the above reasoning on how to exploit the log is correct, and a pointer to the exact place in the code where this "log tapping" should occur
an explanation on how to recognize and decode "data point insertion entries" among all the entries that are applied to the log
some hints on where to "store" the "subscription object" inside the broker, so that both the log code and the http handler can see it: should this stay in the server, the broker or the log...
if there are multithreading issues and locks involved...
state your preference between "comet", "web sockets", or "HTTP2" about the way in which the HTTP socket should become a pair of streams

With the above directions, maybe I could give this a try and prepare a pull request :-)
And BTW, I'll be away and with no internet for a few days, so I'll not see your (eventual) replies until April 1st (and this is not a joke!).

+1

There's now a basic pubsub interface that Kapacitor uses for streaming data. It's likely that any additional work will be built on top of that design, so I'm closing this out for now.

Is there any documentation of the interface that Kapacitor is using? :)

You can get a basic idea by either sniffing (tcpdump, ngrep) what kapactor sends to influxdb but some basic documentation would be nice.
If I remember what I saw correclty you ned to open an udp port and then sends something like this to influxdb:
SUBSCRIBE <topic> <udp_port>

And after this you will receive every matching line.
PS: I think the syntax is correct but I am not sure if "subscribe" is the correct command name

Looks like this is the code responsible? https://github.com/influxdata/influxdb/tree/master/services/subscriber


How to get progress status of API calls?

Hi Replicate team,
I am using the replicate API for image generation and fine-tuning. In our front-end, we would like to show how much % of the job is complete, so users have a better idea of how much it's complete, how long it takes to complete etc (e.g 10%, 30% complete etc.).
For image generation, I use replicate.predictions.create to start the image generation, and use the prediction id in replicatepredictions.get(pred_id), to check the status. I am wondering if there's a way to calculate/estimate the completion progress using replicate API. I know that Prediction.logs shows some information about the progress, but it's all in string.
I can do a hacky way to extract the progress from the logs, but let me know if there's a better way to calculate the progress (e.g how much of the iterations are complete out of XX total iterations etc).

Hi @921kiyo. Thanks so much for opening this issue.
Parsing logs is indeed the correct way to track the progress of a prediction as it's running. An affordance for this is provided in the Go client library and other libraries, but not Python — at least right now.
I just opened #155, which adds a progress property to Prediction that should do what you need. Please take a look and let me know what you think!

This is now available in 0.13.0.

@mattt Thank you for much for adding this feature so quickly! I will try 0.13.0 to show the progress logs :)

@mattt Just testing this new progress property. While it's working for vanilla SDXL model, progress property does not seem to be available for fine-tuned SDXL model (I've just fined tune a new model today by following https://replicate.com/blog/fine-tune-sdxl). I suspect if this is possibly a bug/edge case (or is this expected?).

@921kiyo Sorry for not responding sooner. I just replied to https://github.com/replicate/replicate-python/issues/171.


Is there any plan to support Back-Channel Logout and Front-Channel Logout

Expected Behavior
It would be nice to support Back-Channel Logout or Front Channel Logout, where we can notify the RP logout the EU, which is very helpful for some scenarios like SLO.
Current Behavior
Not yet support.
Context
As part of the OP logging out the End-User, the OP uses the logout mechanism(s) registered by the RPs to notify any RPs logged in as that End-User that they are to likewise log out the End-User. RPs can use any of OpenID Connect Session Management 1.0 [OpenID.Session], OpenID Connect Front-Channel Logout 1.0 [OpenID.FrontChannel], and/or OpenID Connect Back-Channel Logout 1.0 [OpenID.BackChannel] to receive logout notifications from the OP, depending upon which of these mechanisms the OP and RPs mutually support.

Hello,
If I'm not mistaken, it is already implemented as discussed here.
But you need to use the 1.1 version of the Authorization Server, you can find the documentation here.
I've created the basic version of the Server as explained in the documentation and used it as an Identity Provider with Keycloak, the logout initiated by Keycloak seems to work, and the user is correctly redirected to the defined postLogoutUri.
Hope this helps.

@ValonRexhepi There are difference between RP-Initiated Logout and this question.
According to the definition of post_logout_redirect_uri in the RP-Initiated Logout specification, it is used to redirect to a page specified by the RP after the user exits the OP.
URI to which the RP is requesting that the End-User's User Agent be redirected after a logout has been performed.
Of course we can access the spring-security exit address on the jump based on this logic, but when the same user is authorized in multiple RPs, the matter becomes subtle.
Based on the RP-Initiated Logout feature, when a user initiates an exit operation through an RP, he can only log out of the current RP, and other RPs are not aware that the user has logged out without notification from the OP.
The OpenID specification defines three policies to allow RPs to be aware of such session change operations, such as OpenID Connect Session Management, OpenID Connect Front-Channel Logout and OpenID Connect Back-Channel Logout. Which strategy to adopt depends on the support of OP and RP.
According to the OpenID Connect Back-Channel Logout specification, if OP support Back-Channel Logout the OpenID Connect Discovery endpoint should should contains backchannel_logout_supported which is boolean value specifying whether the OP supports back-channel logout. It can also contain backchannel_logout_session_supported which is boolean value specifying whether the OP can pass a sid (session ID) Claim in the Logout Token to identify the RP session with the OP.
If RP support Back-Channel Logout, then a backchannel_logout_uri should be registered with the OP to receive notification of session changes.
These specifications can be practical individually or used together to solve problems in different scenarios, and currently AS only supports RP-Initiated Logout.

@Scarange, thanks for reaching out! However, as it currently stands your issue is phrased as a question that would be better suited to Stack Overflow. We prefer to use GitHub issues only for bugs and enhancements. Feel free to update this issue with a link to the re-posted question (so that other people can find it).
Having said that, please see gh-58 which will address OpenID Connect Session Management 1.0. The above comment could be used as the basis for adding an issue to support either OpenID Connect Front-Channel Logout and OpenID Connect Back-Channel Logout, so feel free to submit a specific issue for one/both of those.

@sjohnr thanks for response! i have submit two issue for OpenID Connect Front-Channel Logout and OpenID Connect Back-Channel Logout.

I see gh-1200 and gh-1201, thanks.


Redirect to Home Page

Is your feature request related to a problem? Please describe.
It’s always nice when clicking on the logo redirects to the home page.
Describe the solution you'd like
Add feature so that when one clicks on the VBAY logo present on top left of all pages, it should redirect to home page.

@Bhavya1403 I'd like to work on this :D

@Bhavya1403  The feature on the website so that when the VIBAY logo is present on the top left corner can be fixed by adding an anchor tag that is assigned with the value of the index.html file and then placing the image inside it.
`

			`

@Bhavya1403 The feature on the website so that when the VIBAY logo is present on the top left corner can be fixed by adding an anchor tag that is assigned with the value of the index.html file and then placing the image inside it.

You have to make changes in the source code and make a pr don't put your solutions in comments


Error exporting namespace

It seems I can't export a namespace which exports any variables/functions:
export namespace Foo
{
    export const bar = "bar";
}

Produces:
local exports = exports or {}
local Foo = Foo or {}
do
    local bar = "bar";
    exports.Foo.bar = bar --attempt to index field 'Foo' (a nil value)
    Foo.bar = bar
end
exports.Foo = Foo
return exports

Which throws an error as soon as it's imported due to exports.Foo being accessed before being assigned.

This is also related to #207 we changed the export mechanism to export values at the end of a block. Seems like this doesn't work very well with namespaces.

The the issue in the second comment isn't that important imo. Since you TS won't allow you to access bar since it was not exported. This will only cause issues if you import Foo and access bar via (Foo as any).bar

Good point - definitely not a serious issue. I can imagine extreme edge cases where someone might iterate members of the namespace and get more than they expect, though.

What is the purpose of exporting twice here:
    exports.Foo.bar = bar
    Foo.bar = bar

The assignment to exports seems redundant, doesn't it? Also, it's added even if the namespace is not exported:
export interface ForceModule {};

namespace Foo
{
    export const bar = "bar";
}

local exports = exports or {}

local Foo = Foo or {}
do
    local bar = "bar";
    exports.Foo.bar = bar --should not be exported
    Foo.bar = bar
end
return exports

It seems like it's not needed at all. Or am I missing a case where it would be necessary?

The problem is that you need to duplicate the namespace if not all members are exported. This used to work. But i think we just  removed/changed some of that functionality accidentially.
If irrc it used to be like this:
namespace Foo
{
    export const bar = "bar";
    const baz = 3;
}

local Foo = Foo or {}
exports.Foo = exports.Foo or {}
do
    local bar = "bar";
    exports.Foo.bar = bar
    Foo.bar = bar
    local baz = 3
    Foo.baz = baz
end
return exports

The problem is that you need to duplicate the namespace if not all members are exported.

My apologies, I'm not totally sure what you mean here.
In an attempt to wrap my head around this, I took a look at the JS that tsc generates:
var Foo;
(function (Foo) {
    Foo.bar = "bar";
})(Foo = exports.Foo || (exports.Foo = {}));

In lua that would basically be this:
local exports = exports or {}
local Foo = exports.Foo or {}
exports.Foo = Foo
do
    Foo.bar = "bar"
end
return exports

Since Foo should always be the same object as exports.Foo, there doesn't seem to be a need to do the assignment 'exports.Foo.bar = "bar"'.

No you are actually right, JS doesn't duplicate the namespace.
"use strict";
exports.__esModule = true;
var test;
(function (test) {
    var x = 3;
    test.y = 21;
})(test = exports.test || (exports.test = {}));
console.log(exports.test === test);

prints true for me.
(test = exports.test || (exports.test = {})) basically translates to
if (exports.test) {
  test = exports.test;
} else {
  exports.test = {}
  test = exports.test; // since (exports.test = {}) returns the result of the assignment which is exports.test
}

So yeah we should make the changes you suggested.
The reason why I thought it needed to be duplicated is because I thought you could access namespaces members inside the same file regardless if they are exported or not.
namespace x {
 const y = 12;
}
const z = x.y;

turns out you can't do that so there is no reason to keep an extra table.

Ah, ok! My knowledge of JS/TS is much less than that of lua, so I thought I there was something I wasn't aware of going on here.
I'll take a look at doing a fix for this since it's a bit of a blocker for me right now.


Update ammonite-spark to 0.13.1

Updates sh.almond:ammonite-spark from 0.13.0 to 0.13.1.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "sh.almond", artifactId = "ammonite-spark" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "@monthly" },
  dependency = { groupId = "sh.almond", artifactId = "ammonite-spark" }
}]

labels: library-update, early-semver-minor, semver-spec-patch, commit-count:1

Handled from https://github.com/almond-sh/almond/pull/961.


Scrolling locks grid

With a single ag_grid in the layout (no callbacks) and about 300 records, scrolling the grid up and down with the mouse wheel for maybe 15 seconds causes the browser to become intermittent and finally die.  (repeatable in Chrome and Firefox).

Alpha version 2.0.0a2
Dataframe is [318 rows x 15 columns]

Odd observation: It happens much faster if there is an external style_sheet (external_stylesheets=[dbc.themes.DARKLY]). With no stylesheet it still occurs, but take longer.
tbl = 'Site'

app = Dash(__name__, external_stylesheets=[dbc.themes.DARKLY])

table = dag.AgGrid(
            id='ag_grid',

            columnDefs=columnDefs,
            className='ag-theme-alpine-dark',
            rowData=df.to_dict('records'),
            columnSize='autoSizeAll',     # sizeToFit, autoSizeAll
            # cellStyle=cellStyle,      # would be used for conditional formatting (see demo-stock-portfolio.py)
            dashGridOptions={
                'undoRedoCellEditing': True,
                'rowSelection': 'single'
            },
            defaultColDef=dict(
                resizable=True,
                sortable=True,
                filter=True,
                floatingFilter=True
                ),
            )

header = html.Div(tbl + ' Data',
                  className='h2 p-2 text-white bg-primary text-center')

app.layout = dbc.Container(
    [
        header,
        dbc.Row(dbc.Col(table), className='py-4'),
    ],
)

a typical columnDef:
 {
                'headerName': 'Smithsonian',
                'field': 'Smithsonian',
                'filter': 'agTextColumnFilter',
                'editable': False,
}

Will do.    This morning I cannot get it to fail.  No changes, no restart.
If it does it again I will send a functioning version with a data set.
Thanks.  Great piece of software.

Ok, I'll close this for now, but please feel free to open a new issue if you can isolate the problem.
Glad you like dash-ag-grid :slightly_smiling_face:


sherlock: 0-unstable-2024-05-15 -> 0-unstable-2024-05-22

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for sherlock is: Hunt down social media accounts by username across social networks
meta.homepage for sherlock is: https://sherlock-project.github.io/
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0-unstable-2024-05-22 in filename of file in /nix/store/lqdh49bi8f04jy056y5g4yj70h0gh1g9-sherlock-0-unstable-2024-05-22

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

2 package rebuild(s)

First fifty rebuilds by attrpath

sherlock

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/lqdh49bi8f04jy056y5g4yj70h0gh1g9-sherlock-0-unstable-2024-05-22 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A sherlock https://github.com/r-ryantm/nixpkgs/archive/0fcdadf73a2acdc4ebc3b49a0470878a4871e93d.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/0fcdadf73a2acdc4ebc3b49a0470878a4871e93d#sherlock

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/lqdh49bi8f04jy056y5g4yj70h0gh1g9-sherlock-0-unstable-2024-05-22
ls -la /nix/store/lqdh49bi8f04jy056y5g4yj70h0gh1g9-sherlock-0-unstable-2024-05-22/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review --extra-nixpkgs-config '{ allowInsecurePredicate = x: true; }' run on x86_64-linux 1

1 package built:

sherlock

Maintainer pings
cc @applePrincess for testing.

Result of nixpkgs-review pr 314156 run on x86_64-linux 1

1 package built:

sherlock