Bucketmax write strategy

Bucketmax writes the metric with the most datapoints just as max does but uses
a different approach to determine which metric has the most datapoints.
At each write operation 'max' loops once over the list of metrics and picks the metric
with most datapoints.
'bucketmax' stores all metrics in a list where each index represents the number of
datapoints in the specific metrics. For example buckets[5] contains a list of all
metrics which have 6 datapoints. At each write operation 'bucketmax' selects the
first metric from the largest bucket.
The time complexity of 'max' is linear, 'bucketmax' is near constant time but runs
on every ingestion where 'max' only runs on every write. There is also some overhead
in resizing the buckets list. I've done some limited testing and 'bucketmax'' CPU
usage was almost half of that of 'max'. The constraint of bucketmax is that the number
of datapoints per metrics must be relatively low compared to the total number of
unique metrics (which I think is the most common use-case in Graphite usage). I don't
know where the tipping point is but if one has for example 10 metrics with each 100000
datapoints I would assume 'bucketmax' would perform worse than 'max'.

Great idea, I like that, @piotr1212 !
For me it looks like much proper max stratege, I mean, for typical use-case.

I don't see where the metric is removed from the associated bucket when it is flushed?
I'm also concerned about the impact on memory usage of maintaining the buckets, since you're going to have a second copy of all the metric names in the bucket data structure.

I don't see where the metric is removed from the associated bucket when it is flushed?

It is being popped here: https://github.com/graphite-project/carbon/pull/879/files#diff-b95553eddf4a9a25750a9a1d17f2be0cR161

I'm also concerned about the impact on memory usage of maintaining the buckets, since you're going to have a second copy of all the metric names in the bucket data structure.

Good point, i'll see if I can store pointers instead of the full string somehow.

It seems that Python does not copy the metric name but just stores a reference to the string object.
See below for a small simulation, the refcount to the metric name increases when I add it to the buckets. The size of the buckets is smaller than the actual size of the string.
In [62]: a_metric_name = 'a very long string representing a metric name in graphite, foobar, some more text, even more and more and more and more'

In [63]: sys.getsizeof(a_metric_name)
Out[63]: 168

In [64]: sys.getrefcount(a_metric_name)
Out[64]: 2

In [65]: a_metric_cache = dict()

In [66]: sys.getsizeof(a_metric_cache)
Out[66]: 248

In [67]: a_metric_cache[a_metric_name] = [1, 2, 3, 4, 5]

In [68]: sys.getsizeof(a_metric_cache)
Out[68]: 248

In [69]: sys.getrefcount(a_metric_name)
Out[69]: 3

In [70]: buckets = list()

In [71]: sys.getsizeof(buckets)
Out[71]: 72

In [72]: buckets.append(a_metric_name)

In [73]: sys.getsizeof(buckets)
Out[73]: 104

In [74]: sys.getrefcount(a_metric_name)
Out[74]: 4

That's good to know.  The only other comment I'd have would be around how we're now mingling handling between the strategy and the cache, and that the cache has strategy-specific code in it.
One thought was that the buckets could be defined within the strategy and a method on the strategy called each time we add a new point so that it could do any housekeeping.  That would remove the strategy-specific code from the cache.

That's good to know. The only other comment I'd have would be around how we're now mingling handling between the strategy and the cache, and that the cache has strategy-specific code in it.
One thought was that the buckets could be defined within the strategy and a method on the strategy called each time we add a new point so that it could do any housekeeping. That would remove the strategy-specific code from the cache.

Thank you for the suggestion. Updated, code looks much cleaner now.

That's good to know. The only other comment I'd have would be around how we're now mingling handling between the strategy and the cache, and that the cache has strategy-specific code in it.
One thought was that the buckets could be defined within the strategy and a method on the strategy called each time we add a new point so that it could do any housekeeping. That would remove the strategy-specific code from the cache.

Thank you for the suggestion. Updated it, code looks much cleaner now.

squashed the commit

Cool! will try to run in on https://github.com/graphite-project/graphite-test-docker this weekend

Ok, I made graph below on my MBP 2017 (docker had 4 cores/6GB RAM) using https://github.com/graphite-project/graphite-test-docker
haggar generated -agents=10 -metrics=10000 = 100K metrics
MAX_CACHE_SIZE = Inf
MAX_UPDATES_PER_SECOND = 5000
MAX_CREATES_PER_MINUTE = 50

CACHE_WRITE_STRATEGY = max

CACHE_WRITE_STRATEGY = bucketmax

But CPU is nearly identical, memory usage is higher. That is quite different from what I measured. I'll do some more testing

50k metrics every 10s, metrics were already existing.
carbon.conf
MAX_CACHE_SIZE = inf
MAX_UPDATES_PER_SECOND = 500
MAX_CREATES_PER_MINUTE = 1000000

load generator is  https://github.com/Civil/graphite_perf_test_go
~/go/bin/graphite_perf_test_go -host <IP_ADDRESS>:2003 -connec
tions 50 -interval 10
2020/02/09 16:45:33 Starting...
2020/02/09 16:45:33 ===New iteration===
2020/02/09 16:45:33 Load    :  50 x 1000 = 50000  metrics
2020/02/09 16:45:34 Spent   :  0.043315 seconds
2020/02/09 16:45:34 Speed   :  1154334.5 metrics/second
2020/02/09 16:45:34 Sleeping:  9.956684 seconds

first 30m is bucketmax, then 15m max, then bucketmax again. You see a significant lower CPU usage with bucketmax.

But CPU is nearly identical, memory usage is higher. That is quite different from what I measured. I'll do some more testing

Well, memory is the same, I just waited more time. But you're right, some owerflowing test has much more sense here.

Ok, merging this.

Is there an explanation for the significantly reduced pointsPerUpdate? I thought the behavior would be equivalent.

With the 'max' strategy CPU goes to 100%. So I assume it does less write
operations which results in larger queues so more points per update. I
probably should have added the 'write_operations' to the graph and do a
test where it does not overload the CPU.
On Thu, 21 May 2020, 20:45 Dieter Plaetinck<EMAIL_ADDRESS>wrote:

Is there an explanation for the significantly reduced pointsPerUpdate? I
thought the behavior would be equivalent.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/graphite-project/carbon/pull/879#issuecomment-632277284,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAWGWWSSJ7SFJPOV2CXJ76TRSVZE7ANCNFSM4KPXYTRQ
.

Ah that makes sense and explains why pointsPerUpdate grows linearly.  This also means the memory usage in non overload scenario is likely about the same as judging from the start of the 'max's run


Could add a parameter to function face_encodings？

Description
Could add a "model" parameter to face_encodings(face_image, known_face_locations=None, num_jitters=1)?
I want to control  input of  function_raw_face_landmarks.

What is your overall goal?

when using function face_encodings, I want to use "model" parameter to control function _raw_face_landmarks parameter "model".
Now _raw_face_landmarks default parameter is "small",I want to use "large".

Why?

I don't think that would change the results at all. Have you done any tests or think that it would help?

@ageitgey do you think it would produce same output with 5_face_landmarks and  68_face_landmarks?

Yes


bug: After the drawer is closed, marker id is not removed from the url

Bug Definition
https://user-images.githubusercontent.com/22892227/218235830-6e82c8c6-52d2-4e2d-9492-b6e9304aa411.mov
-- Please add discord contact information
** discord username: @puskuruk  **
## Bug environment
rc.afetharita.com
Describe how you are producing the bug step by step

Go to rc.afetharita.com
Click to a marker
See marker id on search parameters
Close the drawer
You'll still have id(marker id) on your url search parameters

Expected Behaviour
We should remove marker id after user closed the drawer
Desktop Information
Chrome 110

PR ready if approved

@eytanfb ben waiting-test ile waiting approval'u karistirdim o yuzden.

We are still getting the same URL with id marker after drawer closed unless we move the map.

this is excepted behaviour.

It is expected because of;
The id data is already being deleted onMapMove action.

Related PR has been created


add request JSON data to RunningJob in Python client

Coverage remained the same at 88.845% when pulling ff61d8659f90fb47bdbd0fcce7eac74f73ee1f31 on irontable:develop into 617b13530ddfbfc79d8e0774164b2be6d4e4fb99 on Netflix:release/3.0.0.


rerender fixes

EventsTable fix for 2+ renders, linechart fixes for brush re render and draw on resize. Other small styling changes.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: niusoff sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


symbolic links in path names

Because of the way we have our go projects configured we symlinks to directory outside the normal go path which contain code.  I have found that when I jump to a definition and it's the first time visit to the file will be opened with the name outside the path.  I have traced the problem back to using file-truename over expand-file-name.  You can see an example of this problem in godef--call.  When called with a buffer with a file name in the go path (but containing a symlink in the path) the resulting file name will be outside the path and chaos results.
Since file-truename calls expand-file-name to do almost all the work would it be possible to switch to using expand-file-name?  I have patched my personal copy, and I not had any issues, but I am sure there are lots of things in go-mode that I have not tried yet.

There are setups where we need to follow symlinks. These setups usually involve symlinks from outside a GOPATH to inside a GOPATH, in particular when the value of GOPATH itself is a symlink. We're not actually calling file-truename for the effects that expand-file-name has, but explicitly for the symlink following.
The go tool itself has a history of not fully supporting symlinks (for example go list ... will not work properly), so I am inclined to not make any changes in go-mode and to suggest that you maintain your own version. It should also be noted that other Emacs packages for Go, such as the integration with Guru, use the same logic as go-mode.

Thanks for the feedback.  I wondered if there where problems with symlinks
and now I know there is a deeper problem and will follow your suggestion.
In the end it looks like we need to rethink how we are using symlinks so we
can avoid the problem.
On Fri, May 20, 2016 at 2:53 AM, Dominik Honnef<EMAIL_ADDRESS>wrote:

There are setups where we need to follow symlinks. These setups usually
involve symlinks from outside a GOPATH to inside a GOPATH, in particular
when the value of GOPATH itself is a symlink. We're not actually calling
file-truename for the effects that expand-file-name has, but explicitly for
the symlink following.
The go tool itself has a history of not fully supporting symlinks (for
example go list ... will not work properly), so I am inclined to not make
any changes in go-mode and to suggest that you maintain your own version.
It should also be noted that other Emacs packages for Go, such as the
integration with Guru, use the same logic as go-mode.
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/dominikh/go-mode.el/issues/138#issuecomment-220531137

--
C. Paul Bond<EMAIL_ADDRESS>


Add official logo

Adding the new logo from JuliAnn Tulberg via @BenLloydPearson.

I realized I probably should've pinged @Szymongib about this, not @jwilander. My bad! Let me know if you have any questions about this one.

Looks good :)
I think @gabrieljackson can approve


Improving performance

I would like to ask if there is a way to shorten load times and reduce network payload.
Perhaps there is a way to customize and compile the source files individually, and to cut back on unused MatBlazor components, given the fact that the only components that I use are MatExpansionPanel, MatToast and MatChip. Seems like an overkill to load the whole package just for these 3 components.
Or maybe it's just that resources are uncompressed?
Any help would be appreciated.

@jasiunaz maybe we should wait for .NET 5 for some speed improvements, have a look at https://visualstudiomagazine.com/articles/2020/09/14/aspnet-5-rc1.aspx


jQuery & jQuery UI Updates

Explanation About What Code Achieves:

During the UI setup part of Bottery startup, I noticed there were several 404 errors looking for assets in the non-existent /css/images/ directory. I assumed these were icon tilesets from jQuery UI that for whatever reason were not included in the repository. To test this hypothesis, I downloaded a minimal themed version of the latest jQuery UI (1.12.1) and put it in my development repository. The 404 errors went away, and the panels now have little corner grabbers indicating their resizability.
Unfortunately, this update also made the sliders in the Controls panel visually unresponsive for some reason. I couldn't nail down the cause of this, so I replaced the jQuery UI sliders with native sliders with nearly the same functionality (fires events when the mouse button is released instead of on every move of the slider).
These larger, native sliders also pushed down the buttons at the bottom of the Controls panel past the edge of the panel, so I resized and repositioned the panel to fit all of its components. I also took the liberty of slightly tweaking the rest of the panels' sizes and positions to maintain consistent margins.
Since I had updated jQuery UI to the latest version, I decided to try updating jQuery proper (3.2.1) to see if there were any other issues. I did not notice any adverse effects after swapping in the new version, so I included it in this PR
Steps To Test:

Before PR:

After PR:

TODOs:

None

@calvinashmore Can I get some feedback on this, and my other PRs I've submitted?

Hey there, sorry for the delay. I wanted to take a closer look at this one since it's a significant change to replace the jquery version. I haven't had the chance to actually test it out yet, but I hope to do so by the end of day today

Also your contributions have been really valuable and I'm glad that you're helping :)

okay, sorry for the crazy delay on this one! I was able to test it and it looks great


/articles: some links go to a Stampy Google Doc instead of a published article

Example from https://redesign.aprillion.workers.dev/questions/8AEQ/What-is-behavioral-cloning:

Not sure whether this is intentional or not.

Sort of. The problem is that only live on site articles should be viewable on the page - articles that are still being written are sort of in a grey zone where they exist, but aren't on the site. So this is a compromise. They should get automatically updated when the status of an article changes


Split published artifacts

Is your feature request related to a problem? Please describe.
Create a package for the common libraries and different packages containing the libraries of each protocol/extension.
Describe the solution you'd like
Currently the published package contains the full deploy of the repository. Since the repository divides itself in a core logic and different implementations on each protocol, creates multiple packages each one containing the corresponding libraries and code.

The project structure shall be changed to accept the split, otherwise too many projects will be in the same folder. The contexts will be:

native: for projects managing native code
net: for projects related to .NET
java: for projects related to Java


Nostd?

Due to heavy use of std::io traits, no_std is currently impossible
See: https://github.com/rust-lang-nursery/portability-wg/issues/12#issuecomment-382072568


Angular FESM modules have imports that produce 'undefined value' exceptions

Hi,
I'm using localforage 1.7.1 in one of the library modules created using Angular CLI 6.0.0-rc.6 (@angular-devkit 0.5.8). My code in one of the services looks like this:
import * as localforage from 'localforage';

@Injectable()
export class LocalStorageService {
  public setItem<T>(key: string, value: T): Observable<T> {
    return Observable.fromPromise(localforage.setItem(key, value));
  }
}

Angular CLI internally uses ng-packagr/rollup to create bundles for a number of module types.
For FESM5 and FESM2015 generated bundles look something like this:
import { setItem, getItem, removeItem } from 'localforage';
...
LocalStorageService.prototype.setItem = function (key, value) {
  return Observable.fromPromise(setItem(key, value));
};
...

Unfortunately, this produces
Uncaught TypeError: Cannot read property 'ready' of undefined
    at localforage.js:916

As you can see the methods got imported in this case and 'self' references in localforage code are undefined as a result.
So, is there a way to import localforage in order for it to work in FESM bundles?
The only solution I came up with so far was to replace "import * as localforage from 'localforage'" with:
const localforage = require('localforage');

to prevent rollup in this case.

That does seem like an extreme code optimization from rollup. I think that you should open an issue at rollup to get their opinion on this. It seems that import localforage = require('localforage'); is the way to go for your case, which is kind of expected since localforage is a UMD package.


Certify homepage url updated

Homepage url updated to: https://certifyyy.netlify.app/

@jayantkandpal Thanks for pointing out the issue. It would be great if you could open an issue for the same first and mention this PR there. Let's keep things in sequence


Fog color is too dark

While working on #12248 I noticed the fog color overall seems too dark compared to other platforms like Google Earth. I would expect the colors to more closely match the light white/yellow at the base of the atmosphere.

This gets even worse/more obvious if you crank up the density

Looking back at some older fog related issues it seems like it wasn't always this way so hopefully it's an easy fix that was just missed somewhere along the way
From https://github.com/CesiumGS/cesium/issues/8454

or

I think I have a fix for this started in https://github.com/CesiumGS/cesium/compare/atmopshere-scattering-fx?expand=1
Basically ground atmosphere scattering is not using the same amount of light color as the sky, plus the alpha value was not being correctly computed.

As discussed in #12248 when the issues with the fog darkness are fixed we should increase the default visualDensityScalar so fog is more visually impactful.


Rule: Report unsafe call on nullable types

I'd like to highlight unsafe calls as warnings
fun foo(bar : String?){
    println(bar!!.length)
}

P.S. I'll create PR for this check, this issue just for discussion

This rule implemented in #174

#174 merged.


All classes -lg or -md - sm

For example : position-fixed-lg
then if screen size is large this class worked
It is a great idea for Bootstrap 5

Or position-lg-fixed? Hmm, confusing...

You can enable these if you like in our utilities API.


Add Further Information to Document

Under Initial Setup add:

That the directory to be used (C:/Code/Shenanigans) has to be made beforehand.

Somewhere  add:

That FSP can be forked into a different directory and still work

A note - I think it would make sense to create the directory as part of setup. People will be copy-pasting the samples as is.

That is true, I noticed the mistake of not making the directory almost right away but it could slip by someone. It's ultimately up to you.


fix: Don't generate TS types for verify-token

k8s.io seems to be broken somehow
k8s.io/api/storage/v1alpha1<EMAIL_ADDRESS>read<EMAIL_ADDRESS>stream error: stream ID 221; INTERNAL_ERROR; received from peer

However it only seems to be failing on the build-continue-no-error job but not on others
https://github.com/slsa-framework/slsa-github-generator/actions/runs/4694783478/jobs/8323414976?pr=1991#step:2:842

k8s.io vanity package names seems to be broken somehow
k8s.io/api/storage/v1alpha1<EMAIL_ADDRESS>read<EMAIL_ADDRESS>stream error: stream ID 221; INTERNAL_ERROR; received from peer

However it only seems to be failing on the build-continue-no-error job but not on others https://github.com/slsa-framework/slsa-github-generator/actions/runs/4694783478/jobs/8323414976?pr=1991#step:2:842

k8s.io vanity package names seems to be broken somehow
k8s.io/api/storage/v1alpha1<EMAIL_ADDRESS>read<EMAIL_ADDRESS>stream error: stream ID 221; INTERNAL_ERROR; received from peer

However it only seems to be failing on the build-continue-no-error job but not on others https://github.com/slsa-framework/slsa-github-generator/actions/runs/4694783478/jobs/8323414976?pr=1991#step:2:842

I've seen other errors today and I suspect it's due to GH itself. Re-running fixed the problem in other PRs

I've seen other errors today and I suspect it's due to GH itself. Re-running fixed the problem in other PRs

I hope so. I re-ran it twice so far and it's failed consistently each time, passing on the build job
https://github.com/slsa-framework/slsa-github-generator/actions/runs/4694783478/jobs/8324204627?pr=1991#step:2:847
But failing on the build-continue-no-error job
https://github.com/slsa-framework/slsa-github-generator/actions/runs/4694783478/jobs/8324204433?pr=1991#step:2:1151
Since they basically do the same thing it makes no sense to me.


Add support for image info dicts

In addition to an array of inputs, and an array of targets, also support
an array of info dictionaries. They are one-to-one with the array of
inputs.
The LabelKFold splitting uses a value in the info dictionary; the value
to use is indicated by the label_key.

I'm going to merge this in a bit. Comments welcome!


docs: Use new Google Analytics 4 site tag

Contributes to https://github.com/argoproj/argo-site/issues/102
As mentioned in that umbrella issue, the UA site tag is connected from the new GA4 site tag and so will continue receiving events.

/cc @alexmt

@alexmt @alexc et al.: Is the following an official Argo website:

https://blevinson.github.io/argo-workflows/
This is hosted from https://github.com/blevinson/argo-workflows by @blevinson

I'm asking because this GitHub-hosted site is generating analytics as if from the main site.


Update JSDOM

Follows https://github.com/rstacruz/mocha-jsdom/pull/28 and #18 in an attempt make the Travis tests pass.
Since we are now requiring JSDOM >10.0.0, we may need to bump the major version up.

I can confirm that this PR works. Could this be merged?

Just add the right version of JSDOM in dependencies

Unfortunately, this does not work for me, because it means all other dependencies that rely on JSDOM also need to be downgraded. For now, I've solved my own issue by simply using plain JSDOM code to give me what I needed. But I think that if you look at all 3 scenarios - removing mocha-jsdom, downgrading jsdom for working around this, or merging the PR - only the PR brings this project forward.

I said that mocha-jsdom should have the right version of jsdom in its package.json. It should nt be a unmet dep.

Ah, sorry, my mistake. But the current package.json of mocha-jsdom mentions the following: "jsdom": "7.0.2",. Isn't that what you mentioned?

It mentioned as a dev dep and as peers dep. It should be removed from this two place and moved to the dependenties section. That way, the jsdom@7 should be correcly installed.

Ah, I learned something today - thanks!

Hi there! Is there a plan to merge this fix in the near future?

:+1:

Hello @rstacruz, do not forget this merge...

Oops! Let me attend to this in a bit. For some reason I didn't see this in my notifications. Thanks for bringing it to my attention :)

Merged! I'll also do some more updates #31. I'll push a 1.x-compatible version (v1.2.0), and a new version (v2.0.0) which includes jsdom as a direct dependency.


chore: set prettier to error lint

fixes #902

Codecov Report

Merging #918 into rc0.14.0 will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           rc0.14.0     #918   +/-   ##
=========================================
  Coverage     94.09%   94.09%
=========================================
  Files            86       86
  Lines          3643     3643
  Branches        574      574
=========================================
  Hits           3428     3428
  Misses           90       90
  Partials        125      125

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 862e04e...9e1365f. Read the comment docs.


ESLint v9

https://eslint.org/blog/2024/04/eslint-v9.0.0-released/
Plugins have compatibility issues as of time of writing.

Fixed with https://github.com/janik6n/typescript-starter/commit/229be9150f8cb5042bb4a5d7a22c42af2bae6afc

Fixed with https://github.com/janik6n/typescript-starter/commit/229be9150f8cb5042bb4a5d7a22c42af2bae6afc


⚠ Samsung has degraded performance

In 8298ed9, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 1929 ms

Resolved: Samsung performance has improved in 77b2a6f.


Colors do not display properly based on repo language.

In repo.js Line 86.

Colors make the page crash due to being undefined.
TypeError: Cannot read property 'color' of undefined

Hey so I'm not sure if I have the perfect fix but I just tried something and I get the repo languages to show the right colors. Hope you do not mind if I make a PR for it.

Hey so I'm not sure if I have the perfect fix but I just tried something and I get the repo languages to show the right colors. Hope you do not mind if I make a PR for it.

So I've worked a fix for this bug and I'll make a pull now

So I've worked a fix for this bug and I'll make a pull now


feat(NODE-4336): deprecate old write concern options and add missing writeConcern to MongoClientOptions

Description
What is changing?

Added writeConcern to MongoClientOptions
Added deprecation flags to the top level versions of the options
Added a type test

I don't think we need to mark these deprecations a feat since we already had them deprecated elsewhere in the driver, and there was a message printing out when they were used.
Double check the following

[x] Ran npm run check:lint script
[x] Self-review completed using the steps outlined here
[x] PR title follows the correct format: <type>(NODE-xxxx)<!>: <description>
[x] Changes are covered by tests
[x] New TODOs have a related JIRA ticket

Alright yea, fair enough better to call it out then it perhaps get overlooked in a sub bullet


Generic disappear when pasting Java code

Hi,
I am using AsciidocFx to write books and articles about Java, and I often need to past Java code snippets comming from IntelliJ. When doing so, type parameters simply disappear, which means that:
List list = new ArrayList<>();
becomes:
List list = new ArrayList();
A minor problem is that pasting (from Intellij) inside a block of code adds  a full block, with delimiters. So adding a method in a class gives:
[source,java]
public class MyClass {
[source,java]
public String addedMethod() {
}
}
Even less important (but still annoying) is that a blank line is added at the end of the inserted code.
The workaround is to first paste to get the delimiters and then select the code and past raw, or to insert the delimiters by using the corresponding button and then paste raw. But it would be better if it worked just by pasting.
Thanks
Pierre-Yves

Thank you @pysaumont for the reporting


Create CSV import function

• Import a CSV file in the same format as the CSV export button generates
• Replacing only changed values, matching on the basis of the ID number (first column)
• Generating a list of changed and unchanged "Annotation ID Gloss: Dutch"s that can be saved

Note to self: I've ignored everything which is not a simple model field for now.

The basic functionality for this now works: I've exported some signs to a CSV, made some changes, and imported the CSV again, and the changes were executed.
However, there are still a number of corner cases to take care of. Some notes:

If you export all signs and import them again, the change detector will crash because some choice fields are still empty in the database.
If haven't tested the change detector for all types of fields yet.
Fields that reference other parts of the database (like translations) are ignored for now.

As far as I can see, this now fully works for simple fields. There are two steps still to be taken, but I'm not sure how important it is to do them now - let me know:

Some fields do not contain actual content, but relations to other parts of the database, like to which language a gloss belongs, or the translations (=relation to keyword). If you edit those in the CSV, the import won't notice.

I haven't tested this for every possible change, so I wouldn't be surprised if there were still some corner cases left that don't work or spawn errors. With exporting and importing a whole database with all kinds of different fields, there's a lot that can go wrong.

It appears that boolean fields are not exported in the CSV export; does anything need to be changed in the import module for that as well?

I fixed this two weeks ago :) https://github.com/Woseseltops/signbank/commit/fcd93798801ed765abffce332650443990a5575c Could it be the case that you are using an older export? And if not, what fields are we talking about?

In some (not all) cases, replacing a value in a multiple choice field with ' ' (whitespace) results in a machine value that is identical to the gloss. For example in the gloss 'test5':
For test5, Initial Primary Location changes from Above head (38) to (None).
For test5, Weak Hand changes from R (47) to (test5).

A more comprehensive test (where I changed all fields to whitespace) reveals that the above behaviour only occurs with the Weak Hand field.

It required some hard thinking, but I think I figured this one out: your problem probably wasn't the Weak Hand field itself, but the fact that you put in a space instead of nothing. I added support for spaces to indicate empty fields, does that fix your problem?

Re: Wessel's comment of Febr 4: please do allow import of relations to other parts of the database. In particular, relations to Keywords are important, as this is a field that we will regularly want to modify in an exported version (that will be imported back), to ensure consistency.

When adding functionality for importing Keywords, please make sure this will be case-sensitive.

The main functionality works, but I didn't look into case-sensitivity yet.
About case-sensitivity: after some thinking I've come to the realization it's not that straightforward how exactly 'import keywords in a case-sensitive way' should work. Let's agree on this before it build it. It's going to take some text to explain all of this, so brace yourself :).
First, you should understand that the whole import mechanism works with two algorithms:

The change detection algorithm compares the whole string of keywords. So if something has the keywords 'one', 'two' and 'three', and 'four' is added, it will compare the string 'one, two, three' to 'one, two three, four'. If a change is detected, this is given to the update algorithm (if the user agrees).
The update algorithm will throw away the current relations between the gloss that is being updated and the keywords, and then go over each of the keywords in the new string. For each keyword in the list, it tries to find a keyword in the database that has exactly the same text, and creates a relation to it if it finds one. If it can't find a keyword that matches, it creates a new keyword, and creates a relation to it.

My proposal would be to make these simple changes:

For the detection algorithm, temporarily make both strings completely lowercase before comparison. This means that a change like 'one, two' to 'ONE, TWO' but also 'amsterdam' to 'Amsterdam' won't be detected.
Make the search for existing keywords case-insensitive, which means that it will simply link to the first keyword it finds with the same letters.

These proposals have the following consequences:

When a keyword is removed from a gloss, it will remain in the total list of keywords. This is because you only remove the relation. If you want it to be removed completely, it can be removed in the Django admin.

When a keyword is added, it will be added exactly as it was given. So if we get 'aMSTERDAM', 'aMSTERDAM' will be added.

There is no such thing as 'changing' keywords. If you change the letters of a keyword, it will be recognized as a new keyword (while the old one still exists in the database), whereas if you only change whether something is upper or lowercase, it won't do anything.

A thing to consider: this is quite a lot of work for (what I expected to be) a non-important feature. You might also consider to leave the system case-sensitive as it is now.

Note: in the above, I assumed the current system is case-sensitive, and you want it to be canse-insensitive, but according to @ocrasborn it's actually the other way around. The above is still relevant for the discussion, though, but some things might be confusing ;).

Regarding case: the current system is indeed case-sensitive, and good as it is.
Two remaining issues:

For WAAR-D, Keywords changes from waar (waar) to 1 (1).
In WAAR-D, 'waar' is the only keyword, and it is imported as boolean 'true'. This does not happen when 'waar' is part of a comma separated string, e.g. WAAR-C 'echt, waarheid, waar'

Importing a selection of columns works tot the extent that you always need a dummy column at the end for changes in the second to last column to take effect. Example:

importing SignbankID, StrongHand changes nothing, even if values in Stronghand have changed
importing SignbankID, StrongHand, WeakHand only changes values in StrongHand, even if values in WeakHand also have changed
et cetera
This is circumvented by including a column that includes no changes (e.g. Gloss), but that shouldn't be necessary.

As for your first remaining issue; I cannot reproduce it:
For WAAR-D, Keywords changes from test (test) to waar (waar).
Any hints on what I should do?

The second problem is fixed now: the problem was that there was a linebreak character at the end of the line with the keys. So in your example, the algorithm tried to find a field with the name 'StrongHand\r', which didn't exist, so it ignored it.

It appears that 'waar' was represented as boolean 'true' in the excel-file that formed the basis for the csv-import. Sorry for the inconvenience...


Makefile: Speed up containerized builds

Keep a Go pkg cache outside the container than can be used inside the
container.
Also, pass "-i" to the "go build" command so the cache is populated

LGTM


Enhancement: Amazon Bedrock Support

What features would you like to see added?
I'd like to see support for Amazon Bedrock. I have several customers that are looking to leverage models in Bedrock to give their users a ChatGPT like experience. LibreChat would be an awesome option for this.
More details
Support for Amazon Bedrock and KB would be an awesome feature.
Which components are impacted by your request?
No response
Pictures
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Yup, adding native bedrock support is planned!

You may could use LiteLLM as a temporary workaround.
https://docs.litellm.ai/docs/providers/bedrock

anthropic keeps banning account for no reason, hopefully bedrock support is coming soon

Hello, we are planning on using LibreChat at a wide scale at Cornell University. We have a requirement to use Bedrock and have a workaround at the moment, however we want to understand the current state of this native integration. We may be able to contribute if it's just a matter of priorities being elsewhere. However, if there are technical roadblocks preventing this enhancement from being implemented at the moment, it would be good to know that so that we can focus on the workaround for now.
Thanks,
Marty Sullivan

Hi @marty-sullivan that's exciting!

We may be able to contribute if it's just a matter of priorities being elsewhere.

It is just a matter of priorities being elsewhere. I'm hoping to get to it early July. My idea is that it should be a separate endpoint, similar to Azure OpenAI vs. OpenAI
Also, correct me if I'm wrong, I think Anthropic models have a different API than the other bedrock models (and there may be more differences if Mistral or other big providers are in the same boat).
For this reason, I was thinking of making "Anthropic - Bedrock", "Mistral - Bedrock" their own endpoints, resembling Azure OpenAI vs. OpenAI as mentioned before.

@danny-avila Just as an update, I think we are going to move forward with using LiteLLM as suggested in other areas.
I do think this would be a great enhancement though, to natively support Bedrock. We will revisit later, unless you get to it first.

@danny-avila Just as an update, I think we are going to move forward with using LiteLLM as suggested in other areas.
I do think this would be a great enhancement though, to natively support Bedrock. We will revisit later, unless you get to it first.

Native AWS Setup is coming, and I've already tested extensively using Bedrock models.

Hi all, this is a great tool, and would be awesome to use it with AWS bedrock.
I have tried to setup with LiteLLM as well following the tutorials though unsuccessfully. I did try a fresh install following the GitHub thread. Anyone who had luck getting litellm work, please tell how!
I might be missing a simple step somewhere but have tried several times.

It's not just you; we also had issues using LiteLLM (which works fine on its own) as a custom endpoint in LibreChat. Having native AWS support that integrates directly with Bedrock, similar to this project would be fantastic. Bedrock is expanding rapidly, supporting more inference endpoints for some of the best models in the world. Now that AWS Activate credits can be used, more companies are migrating to it. I applaud @danny-avila  for making this a priority in the near future, as he seems to have suggested.

Thanks @NN1985 for confirming the issue with LiteLLM. And more so for mentioning the AWS Activate. As big companies battle it out, the SMEs and entrepreneurs like us have only benefits to reap. Ever since we found LibreChat, we saved money in ChatGPT subscriptions. And now, we are holding up any further recharge to our OpenAI account to replace it with bedrock.
Have you already tried the aws-samples repo for using bedrock? It could serve as a temporary alternative for us if there's some wait for LibreChat bedrock support..

No problem @aryansaurav  . I am indeed using the linked repo project to access Bedrock directly for the Anthropic API, which would be prohibitively expensive to use at scale otherwise. I haven't found anything else quite like it at the moment—it's a true one-command AWS console zero-config install, and we were up and running in about 30 minutes. We can add all our users with a login similar to LibreChat, but be warned that it's quite basic and buggy in comparison.
I want to be clear that I strongly support LibreChat and vastly prefer it. Once native Bedrock endpoint support is added, or even better, true AWS native support, we would migrate immediately. As you mentioned, I only suggest using this as a temporary measure in the meantime.
Regarding your comment about recharging your OpenAI account, I'd suggest looking into Azure startup credits to cover OpenAI models (if you're not already aware of this option). Since Bedrock doesn't support the OpenAI API and isn't a replacement for us, using Azure for OpenAI models and AWS for Bedrock would give us full coverage with all the models and APIs we need under a single application—LibreChat. This approach could significantly reduce your costs.

@NN1985 Can you expand on what is generally covered by AWS Activate credits? Like are the Claude models covered by those credits, for example?

@NN1985 Can you expand on what is generally covered by AWS Activate credits? Like are the Claude models covered by those credits, for example? I ask because Claude works through the marketplace, and AWS credits generally can't cover marketplace products.

I'm happy to clarify, though I'll keep it brief out of respect for Danny and the main topic of LibreChat. As of April, AWS Activate credits can indeed be used for Claude models, despite their availability through the marketplace. This change is part of a special exception, which you can find detailed in the updated terms and conditions of the credits. I've also verified this with AWS support, and our billing reports confirm that the credits are being correctly applied.
For more information, please refer to this article:
AWS Activate credits now accepted for third-party models on Amazon Bedrock

@danny-avila Are you also looking into Bedrock support on the rag_api side? It would be nice to have the ability to use the Titan / Cohere embedding models natively and have everything running in AWS.

@danny-avila Just as an update, I think we are going to move forward with using LiteLLM as suggested in other areas.
I do think this would be a great enhancement though, to natively support Bedrock. We will revisit later, unless you get to it first.

Native AWS Setup is coming, and I've already tested extensively using Bedrock models.

@danny-avila Are there any updates on when is this going to be available. Thanks!

+1

I have faith.

I also would love to see this @danny-avila and look forward to testing. Bedrock is a fantastic service and we have about 6ms latency to the bedrock runtime. Putting litellm in between makes things slower for us. Bedrock is also one of the lowest possible cybersecurity risks  when running cloud based LLM as many organizations already have an existing relationship with AWS and approve of their cybersecurity approach and don't want to deal with new and unknown vendors.

There's an open PR, and I'm hoping to merge it today https://github.com/danny-avila/LibreChat/pull/3935

Fantastic, you are the best !

Basic Setup:
Required environment variables:
BEDROCK_AWS_DEFAULT_REGION=us-east-1 # A default region must be provided
BEDROCK_AWS_ACCESS_KEY_ID=someAccessKey
BEDROCK_AWS_SECRET_ACCESS_KEY=someSecretAccessKey

# Note: This example list is not meant to be exhaustive. If omitted, all known, supported model IDs will be included for you.
BEDROCK_AWS_MODELS=anthropic.claude-3-5-sonnet-20240620-v1:0,meta.llama3-1-8b-instruct-v1:0

# See all Bedrock model IDs here: https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html#model-ids-arns

# Notes on specific models:
# 'ai21.j2-mid-v1', # Not supported, as it doesn't support streaming
# 'ai21.j2-ultra-v1', # Not supported, as it doesn't support conversation history

There are also librechat.yaml options you can configue:
endpoints:
  bedrock:
    availableRegions:
      - "us-east-1"
      - "us-west-2"
    streamRate: 35
    titleModel: 'anthropic.claude-3-haiku-20240307-v1:0'

streamRate - (optional) in milliseconds, like the existing option for other endpoints, determines the rate of processing each new token. This can be helpful to free up the event loop to stabilize processing of many concurrent requests. Can also help provide a smoother UX for frontend stream rendering
titleModel - (optional) sets the model to use for titles. Recommended: anthropic.claude-3-haiku-20240307-v1:0. Omit or set as current_model to use the model used for chat.
availableRegions - (optional) If provided, the parameters will include the regions you set as an optional dropdown. if a region is not selected, the default region is used instead. This is useful to also allow use of many regions, as region availability can vary per model

More info: https://github.com/danny-avila/LibreChat/pull/3935

@aryansaurav
Check your ENDPOINTS environment variable, that would be the only thing limiting access to the endpoint, if you have everything else configured.
That and, making sure you are using the latest image: https://github.com/danny-avila/LibreChat/pkgs/container/librechat-dev
librechat-dev:latest not librechat:latest
I've successfully configured the new endpoint on every instance I manage.

I tried, it works! (You've to uncomment the lines in env)

@danny-avila for me it does not work either. I wipe everything (including removing all the docker images) and just clone the main LibreChat branch. The only thing i am configuring in .env is the 3 required BEDROCK entries, then I edit docker-compose.override.yml with the content below and run the whole thing with docker compose. I can login and the bedrock entry shows up but when I click on it nothing happens, same happens when I explicitly configure ENDPOINTS .

# ---------------------------------------------------

services:

# # USE LIBRECHAT CONFIG FILE
#   api:
#     volumes:
#     - type: bind
#       source: ./librechat.yaml
#       target: /app/librechat.yaml

# # LOCAL BUILD
#   api:
#     image: librechat
#     build:
#       context: .
#       target: node

# # BUILD FROM LATEST IMAGE
   api:
     image: ghcr.io/danny-avila/librechat-dev:latest

# # BUILD FROM LATEST IMAGE (NUMBERED RELEASE)
#   api:
#     image: ghcr.io/danny-avila/librechat:latest

# # BUILD FROM LATEST API IMAGE
#   api:
#     image: ghcr.io/danny-avila/librechat-dev-api:latest

ah, i see that LibreChat from github already has librechat-dev:latest so no override required, not clear on the difference between these 2 though
   api:
     image: ghcr.io/danny-avila/librechat-dev:latest
   api:
     image: ghcr.io/danny-avila/librechat-dev-api:latest

Here are a couple of things you can try, assuming everything is configured like in my comment: https://github.com/danny-avila/LibreChat/issues/2620#issuecomment-2338518412
1. Try opening the app in a private window, or a different browser, to see if it's a browser caching issue.
2. You can also make sure the site data is deleted, this is how it's done on Chrome:

3. If the issue persists, try removing all docker container images with the name "librechat", to be sure docker isn't using a cached version of "latest."
# from project root
docker compose down
docker rmi $(docker images | grep librechat | awk '{print $3}')
docker compose up

Ha, doh, creating a private browser window did the trick ...... has already run docker rmi many times .

Now the testing can start, but it seems it works very well. Thank you for your hard work on this

This is working great, thank you!  I tested using a few models in us-west-2
Of note: I'm not sure if it's a bug but switching Bedrock vendor models (eg. Anthropic -> cohere) mid conversation will result in errrors. It doesn't bother me but I wanted to note it. I'd be happy to file a bug report with more detail if it is a bug.

This is working great, thank you! I tested using a few models in us-west-2

Thanks, good to hear!

Of note: I'm not sure if it's a bug but switching Bedrock vendor models (eg. Anthropic -> cohere) mid conversation will result in errrors. It doesn't bother me but I wanted to note it. I'd be happy to file a bug report with more detail if it is a bug.

So the cohere-text models will error because they don't support conversation history. I will remove them from the default list, and add a note. However, switching from any anthropic model to command-r-* will not experience any issues.

Great to know, thanks @danny-avila
Going direct vs through litellm is faster.
It looks like code interpreter isn't working direct but it does work through litellm.

It looks like code interpreter isn't working direct but it does work through litellm.

I'm curious how this is done through litellm, you are able to proxy an AWS Agent as a chat completion model?

In any case, I don't plan on integrating Bedrock's agent system. I'm actually building something native to LibreChat that is vendor agnostic: https://github.com/danny-avila/LibreChat/issues/3607
In fact, this support for Bedrock uses the new LibreChat agent framework under the hood, so it's already being tested.
This will include our own Code Interpreter on par with OpenAI Assistants API/Bedrock agents.

Yes, code interpretor works with Bedrock - Claude 3.5, Haiku 3, mixtrel-8x7b, and maybe a few others (not Llama 3.1) via litellm proxy.

I'm looking forward to seeing #3607 in action!

Yes, code interpretor works with Bedrock - Claude 3.5, Haiku 3, mixtrel-8x7b, and maybe a few others (not Llama 3.1) via litellm proxy.

I see, you mean the Artifacts feature. This makes more sense now, I will address this soon!

Yes, good catch.
Thanks

@danny-avila I assume that if we are running this on AWS resources with an IAM Role with permission for Bedrock, we should not need to specify the access/secret key, correct?

@marty-sullivan for now, you do have to. I'm new to AWS so I did not realize this was possible. Will push an update for this later this week, to allow you to omit the credentials. Doing so prompts the credential provider to find credentials from the following sources (listed in order of precedence):
Environment variables exposed via process.env
SSO credentials from token cache
Web identity token credentials
Shared credentials and config ini files
The EC2/ECS Instance Metadata Service
The default credential provider will invoke one provider at a time and only continue to the next if no credentials have been located. For example, if the process finds values defined via the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables, the files at ~/.aws/credentials and ~/.aws/config will not be read, nor will any messages be sent to the Instance Metadata Service.

Perfect! yes this would be a requirement for us and may others. Creating IAM Users is not an option for us.

@danny-avila Perfect! yes this would be a requirement for us and many others. Creating IAM Users is not an option for us.

Should now work with this PR! https://github.com/danny-avila/LibreChat/pull/4038/
Updated docs: https://github.com/LibreChat-AI/librechat.ai/pull/123/files i.e. https://www.librechat.ai/docs/configuration/pre_configured_ai/bedrock

Tested and confirmed as working very well here. Thanks for adding the support! I think this is gonna be huge for the project, like others have said using Bedrock is ideal for many organisations as they are already on AWS and therefore don't need any kind of additional vendor checks to get up and running.

An update: A user reported they were still not able to use AWS IAM role policy on their EC2 instance with the changes from https://github.com/danny-avila/LibreChat/pull/4038.
It was found that making the changes in https://github.com/danny-avila/LibreChat/pull/4051 made it work, despite that the previous changes were working with ~/.aws/credentials when access keys were omitted.

I appreciate all the hard work from @danny-avila , but I seem to have encountered a bug that makes the Bedrock endpoint nearly unusable. For whatever reason, when I set the "custom instructions" in the parameter tab, also known as the system message, I almost always have to go back and delete it and re-enter it, or close the right sidebar and reopen it, and after about a second delay, the fields populate. If I don't do this, I get this generic red box error every time:

Something went wrong. Here's the specific error message we encountered: An error occurred while processing your request. Please contact the Admin.

When I look in the console log I get this error:
2024-09-20 20:23:56 LibreChat         | 2024-09-21 00:23:56 error: [api/server/controllers/agents/client.js #sendCompletion] Unhandled error type The additional field system conflicts with the existing field text. Remove system and try again.
2024-09-20 20:23:56 LibreChat         | 2024-09-21 00:23:56 error: [handleAbortError] AI response error; aborting request: The additional field system conflicts with the existing field text. Remove system and try again.****

I'm on the latest version. It basically makes using the system message on bedrock unusable. Not sure if this is my installation or an actual bug, but I'm on the latest version and I've tried updating to no avail.
Anyone else experiencing this?

@NN1985 This can't be replicated on my end, at least how you described.
Can you specify the exact model/region you are trying this with? Can you also do a screen recording showcasing the issue?
See video:
https://github.com/user-attachments/assets/e123d83c-24c0-4c4a-bce3-ace7d4635719

Also, I'm locking this issue as it's becoming a catch-all for reports on the new bedrock endpoint. For any specific issues to bedrock, please make a Discussions thread first, and then I will transfer it as an issue if it concerns a planned enhancement or a bug that can be observed/replicated/reproduced.


include from code.google.com

line 8 of config.go references "code.google.com/p/gcfg" as an include ... however earlier in 2016 code.google.com was shutdown I attempted to switch it for "code.google.com/archive/p/gcfg" however this did not work.
The page references that the project has been migrated to "https://gopkg.in/gcfg.v1"

Hmm. Can you make a PR?

Like I said, my attempt to fix it didnt work, or I would have made a Pull Request.


Add 'restart' command to REPL

Just to restart while using goby -i.

Let me merge this to master by myself.


why can not  download  mac ide for the url

hello ,i can not download the mac ide 2.0  .it is the url resource is error??

Hi, the link works.


Fall back to libibverbs.so.1

What problem does this PR solve?
Issue Number:
Problem Summary:
What is changed and the side effects?
Changed:
Side effects:

Performance effects(性能影响):

Breaking backward compatibility(向后兼容性):

Check List:

Please make sure your changes are compilable(请确保你的更改可以通过编译).
When providing us with a new feature, it is best to add related tests(如果你向我们增加一个新的功能, 请添加相关测试).
Please follow Contributor Covenant Code of Conduct.(请遵循贡献者准则).

libibverbs.so is provided by libibverbs-dev and it is a symbol link to libibverbs.so.1 which is provided by libibverbs1.
We do not need -dev package in production environment. So linking to libibverbs.so.1 is a better solution.
lrwxrwxrwx 1 root root   15 Jan 28  2022 /usr/lib/x86_64-linux-gnu/libibverbs.so -> libibverbs.so.1
lrwxrwxrwx 1 root root   23 Jan 28  2022 /usr/lib/x86_64-linux-gnu/libibverbs.so.1 -> libibverbs.so.<IP_ADDRESS>
-rw-r--r-- 1 root root 139K Jan 28  2022 /usr/lib/x86_64-linux-gnu/libibverbs.so.<IP_ADDRESS>

Here is NCCL usage
https://github.com/NVIDIA/nccl/blob/2f4cb874ba461f67adbd78b6f19fc5bc7c458ab7/src/misc/ibvwrap.cc#L52-L59
Other differs on rdma.md is necessary to enable RDMA on libbrpc

Could you please share how you install libibverbs? At all my servers, libibverbs.so and libibverbs.so.1 exist at the same time and the former one is a symbol link to the latter one. The latter one is also a symbol link to the real so, just like the figure you pasted above.

Could you please share how you install libibverbs? At all my servers, libibverbs.so and libibverbs.so.1 exist at the same time and the former one is a symbol link to the latter one. The latter one is also a symbol link to the real so, just like the figure you pasted above.

On Ubuntu 22.04 , libibverbs1 provides libibverbs.so.1 and the real so.
# dpkg -L libibverbs1
/.
/usr
/usr/lib
/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/libibverbs.so.<IP_ADDRESS>
/usr/share
/usr/share/doc
/usr/share/doc/libibverbs1
/usr/share/doc/libibverbs1/changelog.Debian.gz
/usr/share/doc/libibverbs1/copyright
/usr/share/doc/libibverbs1/libibverbs.md
/usr/lib/x86_64-linux-gnu/libibverbs.so.1

This is enough to use ibverbs in production environment(libibverbs-dev is not necessary).
In development environment, libibverbs-dev provides header files and libibverbs.so as a symbol link.

Could you please share how you install libibverbs? At all my servers, libibverbs.so and libibverbs.so.1 exist at the same time and the former one is a symbol link to the latter one. The latter one is also a symbol link to the real so, just like the figure you pasted above.

On Ubuntu 22.04 , libibverbs1 provides libibverbs.so.1 and the real so.
# dpkg -L libibverbs1
/.
/usr
/usr/lib
/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/libibverbs.so.<IP_ADDRESS>
/usr/share
/usr/share/doc
/usr/share/doc/libibverbs1
/usr/share/doc/libibverbs1/changelog.Debian.gz
/usr/share/doc/libibverbs1/copyright
/usr/share/doc/libibverbs1/libibverbs.md
/usr/lib/x86_64-linux-gnu/libibverbs.so.1

This is enough to use ibverbs in production environment(libibverbs-dev is not necessary). In development environment, libibverbs-dev provides header files and libibverbs.so as a symbol link.

Thanks for your feedback.


Omit Privacy and TOS pages from search

I was searching for "update" keyword to look for instructions to update the librechat deployment but found it strange that the privacy policy is the first result.
I think it shouldn't be indexed/searchable to start with, to avoid cluttering search results from the actual docs pages.

Thanks


Consider support for Passive Event Listeners

Passive Event Listeners can improve scroll performance, if we can guarantee that the scroll event listener will not call preventDefault().
Support currently exists in Chrome 51+ (and Opera 38+) and FireFox 49+.
Feature detection would be required for other browsers.
Is there merit in adding support for Passive Event Listeners?

Hi @jrstanley, I'm no longer developing this polyfill just because there's a much better one done by Das Surma from Google. I suggest taking a look at that version and leave comments and opinions in his PR, they will be well received for sure. Here's the link https://github.com/WICG/IntersectionObserver/pull/121


v0.1.11

Signed-off-by: Eliza Weisman<EMAIL_ADDRESS>

I've already pushed a git tag for this release, as described by the release process in the comments in Cargo.toml:
https://github.com/carllerche/h2/blob/c5642739867679118aed333887866e22ca090852/Cargo.toml#L3-L6
Hopefully that was correct. If not, I can delete the tag.


Version 1.1.1 - cannot find module '@ricokahler/pool'

Upgrade from version 1.0.0 to version 1.1.1 is obviously a breaking change.
Version 1.1.1 on NodeJS 14.15.4 produces the error
 Cannot find module '@ricokahler/pool'"
Please use 2.x as new version or revert breaking change.
Unfortunately I could not find updated documentation which describes migration path or how to upgrade.

My apologies. This was not meant to be a breaking change. Let me check the build. I'll have something out right away.

My apologies. This was not meant to be a breaking change. Let me check the build. I'll have something out right away.

Testing it in alpha first but will promote shortly

@maximilianschmid let me know if this fixes it for you.


Services logged out after sleep/restart

Describe the bug
Since like few weeks now, every time i wake my pc from sleep or after a restart, all services have been logged out. And gmail is having #1771 problem. This didn't happen before.
Expected behavior
Staying logged in in active services
Desktop:

OS: Parrot Linux 4.8
Version: 5.4.1

Current behavior Same problem, after waking from sleep, Franz has shutdown.
Your Environment
Host: XPS 15 9550
Kernel: 5.10.2-2-MANJARO
DE: Plasma 5.20.4

Current behavior Same problem, after waking from sleep, Franz has shutdown.
Your Environment
Host: XPS 15 9550
Kernel: 5.10.2-2-MANJARO
DE: Plasma 5.20.4


[18.03] binutils-wrapper: Try to avoid adding unnecessary -L flags

We should have backported this to 18.03 but it got lost in the merges. This should fix most cases of the Argument list too long bug.
(cherry picked from commit f3758258b8895508475caf83e92bfb236a27ceb9)

Motivation for this change
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

18.03 is no longer supported, closing.


TypeError: Object of type 'Tensor' is not JSON serializable

Thank you for publishing this nice work. It reports an error when I run the code at the function 'train()' in training.py, line 663. The following error displayed:
Traceback (most recent call last):
File "D:/cmWorks/backdoor_federated_learning-master/training.py", line 667, in
is_poison=helper.params['is_poison'], last_weight_accumulator=weight_accumulator)
File "D:/cmWorks/backdoor_federated_learning-master/training.py", line 183, in train
name='Classification Loss', win='poison')
File "D:\cmWorks\backdoor_federated_learning-master\models\simple.py", line 43, in train_vis
opts=dict(showlegend=True, width=700, height=400, title='Train loss_{0}'.format(self.created_time)))
File "E:\Anaconda3\envs\pyTorch\lib\site-packages\visdom_init_.py", line 389, in wrapped_f
return f(*args, **kwargs)
File "E:\Anaconda3\envs\pyTorch\lib\site-packages\visdom_init_.py", line 1715, in line
update=update, name=name)
File "E:\Anaconda3\envs\pyTorch\lib\site-packages\visdom_init_.py", line 389, in wrapped_f
return f(*args, **kwargs)
File "E:\Anaconda3\envs\pyTorch\lib\site-packages\visdom_init_.py", line 1640, in scatter
return self.send(data_to_send, endpoint=endpoint)
File "E:\Anaconda3\envs\pyTorch\lib\site-packages\visdom_init.py", line 711, in send
data=json.dumps(msg),
File "E:\Anaconda3\envs\pyTorch\lib\json_init.py", line 231, in dumps
return _default_encoder.encode(obj)
File "E:\Anaconda3\envs\pyTorch\lib\json\encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "E:\Anaconda3\envs\pyTorch\lib\json\encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "E:\Anaconda3\envs\pyTorch\lib\json\encoder.py", line 180, in default
o.class.name)
TypeError: Object of type 'Tensor' is not JSON serializable
It seems like the function "json.dumps()" can not fit the type of some "kwargs". Is it caused by the version's problems or something else?
Thanks a lot!

I finally fix it. The error is caused by "visdom". It would be alright by adding an "item()" after the "loss" argument when passing it to "visdom".


🐛 Weird last drawn line highlighted glitch

@JideGuru i added some blur to the path and this is what its looks like before the fix
https://user-images.githubusercontent.com/13378059/203263989-0d533669-5dbf-4c77-8166-7df28aa0a73f.mp4


SILOptimizer: avoid stack protection in two cases where it's not needed

ignore pointers with no stores
loadUnaligned doesn't need stack protection for it's temporary

rdar://105231457

@swift-ci test

@swift-ci test


Added ipv6 support

With these changes startup script can create AAAA records
Also prevent querying upstream servers when resolution is done from cache.

We've only had issues with ipv6 so far, DNS is only part of the issue. Currently it's disabled for good reasons.
If this is fully tested with sniproxy and monolithic ipv6 then that might be a different discussion

Well, I currently use it with lancachenet (ipv6 modified) in my ipv6 only network though without sniproxy.
I'm not ready (yet) to make a PR for lancachenet itself.
However I believe that there might more than one use case for lancache-dns. And I believe it should not be tied to lancachenet this hard.

Perhaps a enable/disable flag would be a good enough addition to allay any concerns?

@astrolox basically it works the same way as before if you supply only ipv4 addresses.
Now it just supports ipv6 addresses and dns records as well.

I'm a huge fan of IPv6 and I love that you're contributing this.
Yet I'm not ready to commit this without further testing and spending some time to understand what issues have arisen in the past. I'll leave this open for now and hopefully we can circle back to it when we have some spare time in the future.

Sorry @pamidur, I didn't intend to come accross as negative as that!
I'm also a big fan of ipv6 but we've been caught out numerous times with people not understanding it enough causing huge issues. Currently the DNS disabling ipv6 lookups has been a useful backstop against "my setup is broken" issues.
I agree with astro, I'm interested in this, and we'll definitely keep it open. However I wouldn't be happy to release this without the corresponding generic/monolithic update. If you can submit a WIP PR to the relevant project then we can discuss the route to testing and adoption

@VibroAxe , @astrolox , Alright, I'll try to put everything together and make a PR for ipv6 for lancache-generic soon


Send feedback of status of device(lightbulb) to app, after checking lightbulb actual value

I have a light Blub which is operated through ligthbulb characteristic on off. I also have a feedback mechanism from which I get bulb is actually on or off(may get damage). I want the actual status of bulb in app.
When App Send request for On, ESP32 on the pin, and send feedback of On to app, I want to first check if it is actual on and then send feedback of actual value. Can I do this?

There are a few different ways you can accomplish this, depending on what type of feedback you are looking to provide.  The easiest method would be to check the status of the bulb in the update() method of your lightbulb service whenever the user requests the bulb to turn on.  If the bulb is not operating correctly, you can return a false instead of a true.  The Home App is not very smart about processing different types of error messages, so this will likely lead to a "No Response" status.  Unfortunately there is no way of producing a specific error message that someone using the Home App can digest.  The other downside of this method is that the lightbulb is only checked when the user tries to turn it on, since that's the only time the update() method is called.
As an alternative, you can continuously check the health of the lightbulb by adding a loop() method to your lightbulb service.  Within this method you can check the status of the bulb, and if it not working you can reset the On Characteristic from true to false, which will be reflected in the Home App.  See Example 12 for details on how to use the loop() and setVal() methods to send notifications back to the Home App that changes the value of a characteristic.  Take care if you use this method to only send notifications back to the Home App if the status of the bulb changes.  If you send continuous notifications repeating the same status over and over you will generate a LOT of network traffic and likely cause problems for the Home App.

In update() if I return false, then I get No Response on app side, even if I set the actual value of lightbulb by setVal() function.

Please review the HomeSpan documentation and the tutorials.  The return value of update() has nothing to do with the status of any Characteristics --- in fact there may be 5 Characteristics that are all changed in a single update() call, many of which are numerical in value.
Rather, the return value of update() reflects whether or not the update() was successful, or if there is a problem with your device.  By returning a value of false, you are telling the Home App that there is a problem with your device and the Home App request has failed (hence the Home App indicates no response).  Unless you want to send a fail signal to the Home App you should always return a true to update() as per the docs.

Ok Got it. Thanks for clarification


scale geometric error with tileset

#5330

Thanks @Vineg!  @lilleyse will review this soon

@lilleyse this is ready for review

Can you add some unit tests for the new matrix function?

Those are all my comments. @lilleyse will do the final review.  Thanks @Vineg!

@hpinkos @lilleyse done

@Vineg @geoscan-builder bump =)  let us know when these comments are addressed and this is ready for another look.  Thanks!

Yeah, I hope to get back to this soon

Thanks again for your contribution @Vineg!
No one has commented on this pull request in 30 days. Maintainers, can you review, merge or close to keep things tidy?
I'm going to re-bump this in 30 days. If you'd like me to stop, just comment with @cesium-concierge stop. If you want me to start again, just delete the comment.

Thanks again for your contribution @Vineg!
No one has commented on this pull request in 30 days. Maintainers, can you review, merge or close to keep things tidy?
I'm going to re-bump this in 30 days. If you'd like me to stop, just comment with @cesium-concierge stop. If you want me to start again, just delete the comment.

@cesium-concierge stop

@lilleyse applied proposed changes and moved getRotation to getMatrix3 as discussed in https://github.com/AnalyticalGraphicsInc/cesium/issues/7398

@lilleyse what's the status of this PR?

@hpinkos Waiting on my review which I'll finish up now.

Note to self after merging: this PR fixes both https://github.com/AnalyticalGraphicsInc/cesium/issues/5330 and https://github.com/AnalyticalGraphicsInc/cesium/issues/7398

I'm going to merge this into a branch and take over the rest of the clean up.  Thanks for all the work you did for this @Vineg!


[Subtask][AMS]: Add the config of the spill map in optimize group for the optimizer to enable the spill map

Search before asking

[X] I have searched in the issues and found no similar issues.

Description
I want to add some config of the spill map in optimize group as below for the optimizer to enable the spill map:
1.enable_spill_map(whether enable the spill map, true or false, default value is false)
2.backend_base_dir(the base directory for the disk spill map, default value is System.getProperty("java.io.tmpdir"))
3.max_delete_memory_size_in_byte(max delete map size in memory for optimizer)
Parent issue
#925
Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

unify properties naming as : https://arctic.netease.com/ch/configurations/
recommand:

spillable.map.enabled
spillable.map.dir
spillable.memory.limit

Let's unify properties naming as : https://arctic.netease.com/ch/configurations/
recommand:

spillable.map.enabled
spillable.map.dir
spillable.memory.limit

OK.


Fix: Empty string as template in CustomPlot.init

Fixes #10482 (report in dvc repo) at https://github.com/iterative/dvc/issues/10482

[x] ❗ I have followed the Contributing to DVCLive guide.

[x] 📖 If this PR requires documentation updates, I have created a separate PR (or issue, at least) in dvc.org and linked it here.

Thank you for the contribution - we'll try to review it as soon as possible. 🙏

Thanks @anunayasri! As long as we are fixing it for the empty string, can we also handle any string that's not a supported template?

Thanks @anunayasri! As long as we are fixing it for the empty string, can we also handle any string that's not a supported template?

@dberenbaum how do we recognize that? run Vega parser / validator?

There is a missing None check in my commit and mypy failing the pipeline. I have correct it.
I observed that $ pre-commit run --all-files does not run mypy in the local. Is it by design? May be to speed up the pre-commit exeuction.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 95.18%. Comparing base (602c053) to head (bf8ff74).
Report is 32 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #841      +/-   ##
==========================================
- Coverage   95.47%   95.18%   -0.29%
==========================================
  Files          57       55       -2
  Lines        3889     3904      +15
  Branches      353      349       -4
==========================================
+ Hits         3713     3716       +3
- Misses        124      138      +14
+ Partials       52       50       -2

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@dberenbaum how do we recognize that? run Vega parser / validator?

🤔 I forgot we don't render these anywhere here, and supporting custom templates would not be easy, so forget my initial suggestion.


Pip install GAPI install issues

Some kind of UJSON error:
[tw-172-17-97-153 dove_body_image]$ sudo pip install gapi --upgrade
Password:
Requirement already up-to-date: gapi in /Users/snelson/Python/PyPy-2.2.1/site-packages
Requirement already up-to-date: gnacs>=1.0.0 in /Users/snelson/Python/PyPy-2.2.1/site-packages (from gapi)
Requirement already up-to-date: sngrams>=0.1.5 in /Users/snelson/Python/PyPy-2.2.1/site-packages (from gapi)
Requirement already up-to-date: requests>2.2.0 in /Users/snelson/Python/PyPy-2.2.1/site-packages (from gapi)
Downloading/unpacking ujson>=1.2 (from gnacs>=1.0.0->gapi)
Downloading ujson-1.33.zip (197kB): 197kB downloaded
Running setup.py (path:/private/tmp/pip_build_root/ujson/setup.py) egg_info for package ujson
Installing collected packages: ujson
Running setup.py install for ujson
building 'ujson' extension
cc -arch x86_64 -O2 -fPIC -Wimplicit -I./python -I./lib -I/Users/snelson/Python/PyPy-2.2.1/include -c ./python/ujson.c -o build/temp.macosx-10.10-x86_64-2.7/./python/ujson.o -D_GNU_SOURCE
cc -arch x86_64 -O2 -fPIC -Wimplicit -I./python -I./lib -I/Users/snelson/Python/PyPy-2.2.1/include -c ./python/objToJSON.c -o build/temp.macosx-10.10-x86_64-2.7/./python/objToJSON.o -D_GNU_SOURCE
./python/objToJSON.c:523:13: warning: implicit declaration of function 'PyObject_Malloc' is invalid in C99 [-Wimplicit-function-declaration]
tc->prv = PyObject_Malloc(sizeof(TypeContext));
^
./python/objToJSON.c:523:11: warning: incompatible integer to pointer conversion assigning to 'void *' from 'int' [-Wint-conversion]
tc->prv = PyObject_Malloc(sizeof(TypeContext));
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./python/objToJSON.c:663:17: warning: implicit declaration of function 'PyAnySet_Check' is invalid in C99 [-Wimplicit-function-declaration]
if (PyAnySet_Check(obj))
^
./python/objToJSON.c:722:13: warning: implicit declaration of function 'PyObject_Free' is invalid in C99 [-Wimplicit-function-declaration]
PyObject_Free(tc->prv);
^
./python/objToJSON.c:818:3: error: use of undeclared identifier 'PyObject_Realloc'; did you mean 'PyObject_dealloc'?
PyObject_Realloc,
^~~~~~~~~~~~~~~~
PyObject_dealloc
/Users/snelson/Python/PyPy-2.2.1/include/pypy_decl.h:327:18: note: 'PyObject_dealloc' declared here
PyAPI_FUNC(void) PyObject_dealloc(PyObject *arg0);
^
./python/objToJSON.c:817:3: warning: incompatible pointer types initializing 'JSPFN_MALLOC' (aka 'void ()(size_t)') with an expression of type 'int ()' [-Wincompatible-pointer-types]
PyObject_Malloc,
^~~~~~~~~~~~~~~
./python/objToJSON.c:818:3: warning: incompatible pointer types initializing 'JSPFN_REALLOC' (aka 'void ()(void *, size_t)') with an expression of type 'void (PyObject )' [-Wincompatible-pointer-types]
PyObject_Realloc,
^~~~~~~~~~~~~~~~
./python/objToJSON.c:819:3: warning: incompatible pointer types initializing 'JSPFN_FREE' (aka 'void ()(void *)') with an expression of type 'int ()' [-Wincompatible-pointer-types]
PyObject_Free,
^~~~~~~~~~~~~
7 warnings and 1 error generated.
error: command 'cc' failed with exit status 1
Complete output from command /Users/snelson/Python/PyPy-2.2.1/bin/pypy -c "import setuptools, tokenize;file='/private/tmp/pip_build_root/ujson/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-t0z0Mm-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_ext
building 'ujson' extension
creating build
creating build/temp.macosx-10.10-x86_64-2.7
creating build/temp.macosx-10.10-x86_64-2.7/python
creating build/temp.macosx-10.10-x86_64-2.7/lib
cc -arch x86_64 -O2 -fPIC -Wimplicit -I./python -I./lib -I/Users/snelson/Python/PyPy-2.2.1/include -c ./python/ujson.c -o build/temp.macosx-10.10-x86_64-2.7/./python/ujson.o -D_GNU_SOURCE
cc -arch x86_64 -O2 -fPIC -Wimplicit -I./python -I./lib -I/Users/snelson/Python/PyPy-2.2.1/include -c ./python/objToJSON.c -o build/temp.macosx-10.10-x86_64-2.7/./python/objToJSON.o -D_GNU_SOURCE
./python/objToJSON.c:523:13: warning: implicit declaration of function 'PyObject_Malloc' is invalid in C99 [-Wimplicit-function-declaration]
tc->prv = PyObject_Malloc(sizeof(TypeContext));
        ^

./python/objToJSON.c:523:11: warning: incompatible integer to pointer conversion assigning to 'void *' from 'int' [-Wint-conversion]
tc->prv = PyObject_Malloc(sizeof(TypeContext));
      ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

./python/objToJSON.c:663:17: warning: implicit declaration of function 'PyAnySet_Check' is invalid in C99 [-Wimplicit-function-declaration]
                                                                                if (PyAnySet_Check(obj))

                                                                                ^

./python/objToJSON.c:722:13: warning: implicit declaration of function 'PyObject_Free' is invalid in C99 [-Wimplicit-function-declaration]
                                                                                PyObject_Free(tc->prv);

                                                                                ^

./python/objToJSON.c:818:3: error: use of undeclared identifier 'PyObject_Realloc'; did you mean 'PyObject_dealloc'?
            PyObject_Realloc,

            ^~~~~~~~~~~~~~~~

            PyObject_dealloc

/Users/snelson/Python/PyPy-2.2.1/include/pypy_decl.h:327:18: note: 'PyObject_dealloc' declared here
PyAPI_FUNC(void) PyObject_dealloc(PyObject *arg0);
             ^

./python/objToJSON.c:817:3: warning: incompatible pointer types initializing 'JSPFN_MALLOC' (aka 'void ()(size_t)') with an expression of type 'int ()' [-Wincompatible-pointer-types]
            PyObject_Malloc,

            ^~~~~~~~~~~~~~~

./python/objToJSON.c:818:3: warning: incompatible pointer types initializing 'JSPFN_REALLOC' (aka 'void ()(void *, size_t)') with an expression of type 'void (PyObject *)' [-Wincompatible-pointer-types]
            PyObject_Realloc,

            ^~~~~~~~~~~~~~~~

./python/objToJSON.c:819:3: warning: incompatible pointer types initializing 'JSPFN_FREE' (aka 'void (*)(void *)') with an expression of type 'int ()' [-Wincompatible-pointer-types]
            PyObject_Free,

            ^~~~~~~~~~~~~

7 warnings and 1 error generated.
error: command 'cc' failed with exit status 1

Cleaning up...
Command /Users/snelson/Python/PyPy-2.2.1/bin/pypy -c "import setuptools, tokenize;file='/private/tmp/pip_build_root/ujson/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-t0z0Mm-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /private/tmp/pip_build_root/ujson
Storing debug log for failure in /Users/snelson/.pip/pip.log

ujson is an optimization that is not strictly required for running the application. Unfortunately, it depends on having a compiler installed.  I think the script will still run with the default json compiler included in the standard library.


Bulk operations for garbage collection schedule

A post-endpoint with example body:
{
garbage_type: string,
date: Date,
move_to_date: Date,
region/tour : id
}

So the purpose of this issue is to create an endpoint to move all entities in the table garbage_collection with a given garbage_type, date and tour to a different given date?
Why does it say region/tour and not just tour or region? Is a decision yet to be made?

To reschedule a set of garbage_collections, you could do a POST to http://localhost/api/garbage-collection/bulk-move?garbage_type=PAP&date=2023-03-08&move_to_date=2023-01-11&region=2

Why does it say region/tour and not just tour or region? Is a decision yet to be made?

It should support both region & tour.


[RFE] collect and index node_load1 metric from nodes

Is your feature request related to a problem? Please describe.
CPU usage alone is not enough to understand what kind of load is on a machine.
Describe the solution you'd like
We have a metric node_load1 we simply need to add to metrics profiles.

Not stale. Still needed.


Example replacements for placeholders

When writing text with placeholders, one might want to see how it looks like with "real world" values. Replacing it in the input field directly is cumbersome, so it would be nice to have extra fields for placeholders where you can put actual values in (and e.g., a check box to enable replacement). Not sure how that can be integrated nicely though...

maybe we can do this with #37, have a seperate page for templates where you can save those values, key/value style?


take a look at memory usage

window retains a lot of moment.js (> 2 MB). Take a look at memory profiler and learn it.

programatic injection of moment.js and chance.js ready in branch "memory-opts2

will be in 2.1.2


Fixes for problems in 1.4.0, bug fixes

This PR fixes the problems in 1.4.0, #100 and #101. It also fixes #97 and #98.
There's a lot here, and I'd probably make it 1.5.0 except that 1.4.0 was so completely broken that it seems reasonable just to call this 1.4.1. There are upgrades on new versions of XML Resolver and Saxon XInclude processor, and the build system has been reworked quite a bit.
Fingers crossed!

Nice merge/update! Thanks for providing a fix for #97.


feat: support customized table

fix: https://github.com/casbin/gorm-adapter/issues/180

Let's fix such issues first: https://github.com/casbin/gorm-adapter-ex/blob/ee30a2f0ab244b24b37840c6e20ba573d1e9ae35/.github/workflows/ci.yml#L57


feat: implement basic support for telemetry

Codecov Report
Attention: Patch coverage is 44.68085% with 26 lines in your changes missing coverage. Please review.

Project coverage is 79.70%. Comparing base (77e1cc2) to head (d132d11).

Files
Patch %
Lines

Streamistry.Core/Telemetry/ConsoleTracer.cs
0.00%
11 Missing :warning:

Streamistry.Core/Telemetry/NullTracer.cs
11.11%
8 Missing :warning:

Streamistry.Core/Telemetry/TelemetryAttribute.cs
37.50%
4 Missing and 1 partial :warning:

Streamistry.Core/Telemetry/TelemetryProvider.cs
50.00%
2 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #24      +/-   ##
==========================================
- Coverage   87.65%   79.70%   -7.96%
==========================================
  Files          22       26       +4
  Lines         235      271      +36
  Branches       55       57       +2
==========================================
+ Hits          206      216      +10
- Misses         10       35      +25
- Partials       19       20       +1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


make credo happy again

Current dependencies on/for this PR:

main

PR #10   👈

This comment was auto-generated by Graphite.


[WIP] Refactor AnalysisResult into a dataclass

Summary
Attempt at addressing some of the data issues in #230 inspired by dataclass use in #221
This adds an AnalysisResult data class that is not a subclass of DbAnalysisResultV1, but is a more user-friendly class for python coding that can be converted to and from DbAnalysisResultV1 for data base storage and retrieval.
The structure of this class is
@dataclass
class AnalysisResult:
    name: str
    value: any
    stderr: Optional[any] = None
    unit: Optional[str] = None
    chisq: Optional[float] = None
    quality: Optional[str] = None
    verified: bool = False
    result_id: Optional[str] = None
    tags: List[str] = []
    device_components: List = []
    extra: Dict[str, any] = {}

Details and comments
Starts refactoring experiments to use this class. Most experiments need some re-thinking of their analysis result types under this change. This updates experiments to use this class, but only changes the number of results for a couple of cases including:

breaking tomogrpahy up into separate results for state, fidelity, eigvals, positivity, trace preserving
breaking RB up into separate results for curve fit, alpha, EPC, EPG

Current issues

RB tests are failing, i couldn't figure out how the rb tests worked to fix these
There are issues with result DB integration (i got errors when trying to save some of the updated experiments, and updating results is not supported yet).
The CurveAnalysis based class would need more refactoring to use this data structure and remove the CurveAnalysisResultData class (possibly replacing internal structure with some sort of simple CurveFitData dataclass)
Breaking up analyis results this way makes it more important to be able to filter the returned results by the name or other attributes when using expdata.analysis_results()

Examples
Example of what some updated results look like using
results = exp_data.analysis_results()
for i in results:
    print(i)

T1 experiment
AnalysisResult: T1
- value: 2.3603894490214927e-05 ± 1.801743431504264e-06 s
- chisq: 0.4268894515534577
- quality: good
- extra: <8 items>

The extra field contains all the extra curve fit data  results[0].extra:
{'popt': array([9.63204516e-01, 2.36038945e-05, 3.85656625e-02]),
 'popt_keys': ['a', 'tau', 'c'],
 'popt_err': array([3.51602498e-02, 1.80174343e-06, 3.79972357e-02]),
 'pcov': array([[ 1.23624317e-03,  5.95164703e-08, -1.31594866e-03],
        [ 5.95164703e-08,  3.24627939e-12, -6.68005016e-08],
        [-1.31594866e-03, -6.68005016e-08,  1.44378992e-03]]),
 'reduced_chisq': 0.4268894515534577,
 'dof': 10,
 'xrange': [1e-06, 3.7e-05],
 'circuit_unit': 'us'}

T2 Ramsey Experiment
AnalysisResult: T2
- value: 8.119888734798959e-05 ± 1.1732134066317724e-06 s
- chisq: 1.0009782741856124
- quality: good
- extra: <8 items>
AnalysisResult: Frequency
- value: 19986.28552215432 ± 36.786715681775256 Hz
- chisq: 1.0009782741856124
- quality: good
- extra: <8 items>

the curve fit extra data is duplicated in both results in this case
State tomography
AnalysisResult: state
- value: DensityMatrix([[0.4827474 +0.j        , 0.00211589+0.00797526j,
                0.00292969-0.00846354j, 0.01367188-0.45019531j],
               [0.00211589-0.00797526j, 0.04101563+0.j        ,
                0.00585938-0.00097656j, 0.00292969-0.00651042j],
               [0.00292969+0.00846354j, 0.00585938+0.00097656j,
                0.01660156+0.j        , 0.01188151-0.02034505j],
               [0.01367188+0.45019531j, 0.00292969+0.00651042j,
                0.01188151+0.02034505j, 0.45963542+0.j        ]],
              dims=(2, 2))
- extra: <3 items>
AnalysisResult: eigvals
- value: [0.92231868 0.04652951 0.0267665  0.00438531]
AnalysisResult: state_fidelity
- value: 0.9213867187499998
AnalysisResult: positive
- value: True

the fitter metdata is in the state extra data results[0].extra:
{'fitter': 'linear_inversion',
 'fitter_time': 0.0017287731170654297,
 'trace': 1.0}

Thanks @chriseclectic . Overall the design looks good to me. One thing I'd like to introduce here is a dedicated data class for fit value https://github.com/Qiskit/qiskit-experiments/pull/237. With this class, the analysis class data structure can be
@dataclasses.dataclass
class AnalysisResult:
    """Experiment analysis result dataclass"""

    name: str
    value: any
    chisq: Optional[float] = None
    quality: Optional[str] = None
    verified: bool = False
    result_id: Optional[str] = None
    tags: List[str] = dataclasses.field(default_factory=list)
    device_components: List = dataclasses.field(default_factory=list)
    extra: Dict[str, any] = dataclasses.field(default_factory=dict, hash=False, compare=False)

which makes the logic more cleaner.
Note that value is populated with fitval in https://github.com/Qiskit/qiskit-experiments/pull/237 by curve analysis. This class buffers result data formatting difference among analysis, e.g. curvefit v.s. tomography, since we can put anything to value. You can also drop a special repr logic for data with uncertainty.
For now you still need to write converter logic. However, since https://github.com/Qiskit/qiskit-experiments/pull/237 is serializable, we can remove these mechanisms to handle a set of value, stdev, unit and exceptions for data without uncertainty from front end once the result db supports the new data type.

test_rb_analysis just checks that the pre-calculated outputs and the analysis outputs are the same.
It fails with the error 'AnalysisResult' object has no attribute 'data'  (did it had this attribute previously?)
The following function is failing: https://github.com/Qiskit/qiskit-experiments/blob/c3458fbb5e994e7152d27c092b4459b00ddeef00/test/randomized_benchmarking/test_rb_analysis.py#L134
this functions gets
calculated_analysis_samples_data: list of dictionary containing the analysis result.
expected_analysis_samples_data: list of dictionary containing the analysis expected result.
and checks that they are (almost) the same.

Seems like it is problematic since we may change the data structure of analysis. Can the rb test remove coupling to the file system?

Replaced by #248


[FEATURE] Add 'Powered by Vercel' logo on the footer

Detailed description
Hi, in order to list this repository as Vercel OSS Sponsor program, we need to brand the footer with this logo:

Context
In order to match the requirements of Vercel's sponsorship program, we need to add this changes:

A Vercel banner in the footer of each site page.
A Vercel banner in the source repository's README.md file.
All links back to Vercel should use a UTM tag with the format: ?utm_source=[team-name]&utm_campaign=oss.

Possible implementation
No response
Additional information
More Vercel design content to be found here.
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Hi sure! :wave:

Hey @mkubdev - can you please send me .env.local variables so that I can get rid of this error:
Please define the NEXT_PUBLIC_SUPABASE_URL and NEXT_PUBLIC_SUPABASE_ANON_KEY environment variables inside .env.local

Hey @iambilalriaz, you can use the local data instead of the supabase,
Import it from lib > Data > events.ts

Hey @iambilalriaz ! Yes, sorry we need to update the readme accordingly to explain the steps to create a supabase session and get API keys. As @hariscs (thanks!) said, you can use local data to test.
Sorry again for the inconvenience.

Hey @mkubdev -  please review this PR https://github.com/WebXDAO/DevConf/pull/39

Fixed by #39


Ready for review: Movebank GPS → Darwin Core

I have finished a new use case:

Lossy transformation of GPS data formatted in the Movebank Attribute Dictionary to Darwin Core.

It is similar to the Mahoney use case @sarahcd made, but rather than attempting to map all source data to Darwin Core, it is lossy (as suggested at our TDWG WG session). It extracts the more basic biological occurrence data that can be harvested by GBIF/OBIS. E.g. it does not include tag, deployment end and acceleration data and subsamples the data per hour. The result is 2 occurrence files:

dwc_occurrence_deployment.csv: a HumanObservation for the tag deployment, with sex, lifeStage, etc. of the animal
dwc_occurrence_gps.csv: MachineObservations by the gps tag, with an indication that it is subsampled per hour

There is no Event Core, since there is not really location or time information to group the occurrences by. Occurrence do have an eventID though (a tag-id + animal-id combination) to allow grouping these in deployments (each one containing a single HumanObservation and number of MachineObservations).
Transformations are done in documented sql queries based on a sqlite database derived from the source data (a data package). These transformations have been reviewed by @sarahcd, but you are all welcome to leave comments.

This is great Peter. The layout is super simple and easy to follow.
In entering a brave new world without necessarily using Event Core, I suspect that there we should examine whether there are some terms here that will be particularly useful for identifying different types of machine observations that will fit into the occurrence core. For example, I feel that it would be really useful to differentiate types of study, eg: acoustic telemetry vs gps telemetry vs geolocation vs radio tracking. Is this the right job for samplingProtocol - should we be aiming for a semi formal vocabulary at least for these terms?

Just FYI, we did some work a while back on lossy transformation of GPS data using a spatiotemporal grid. The difference with this approach is that more data are retained when the individual is covering larger distances (e.g. every kilometer in addition to every hour). See https://github.com/iobis/ziptrack but use with caution as we didn't test this extensively.

@peggynewman: controlled vocabulary for samplingProtocol: that is a good idea, will make a new issue.

@pieterprovoost nice, good to know! I have opted for one by hour, because it is simple to explain and implement (basic window function in SQL).

I have updated the README with a summary of the transformation approach, and will now close this issue. I will extend the use case next year to all available Movebank terms and - with the help of @niconoe - make the transformation steps generic so they can run on any Movebank gps dataset expressed as a frictionless data package.


cmd/thanos/receive: fix dialopts TLS heuristic

Currently, the Thanos receive gRPC dialopts are configured to use TLS
when the remote-write HTTP server is configured with a TLS certificate.
This heuristic is not correct: just because the remote-write HTTP server
is exposed over TLS does not mean that the hashring gRPC server is using
TLS. This commit fixes the heuristic for enabling TLS in the gRPC client
for forwarding remote-write requests internally within a hashring.
Fixes: https://github.com/thanos-io/thanos/issues/4382.
Signed-off-by: Lucas Servén Marín<EMAIL_ADDRESS>
[x] I added CHANGELOG entry for this change.

Nice find :muscle: How have you been able to verify this change indeed fixes the problem? I'm not aware of a test that excercises this code path.

Thanks!


chore(deps): update golang:1.19.x Azure DevOps Pipeline

Update golang:1.19.x Azure DevOps Pipeline

Without the x at the end of the Golang version, Azure will use version 1.19.0 which is outdated and have security issues.


Documentation issue: Db2 - (transactional) does not exist in IBM Cloud

On page https://github.com/IBM/FHIR/blob/master/docs/src/pages/guides/DB2OnCloudSetup.md
under Create a DB2 instance section it says:
3. Choose Db2 (transactional).
That does not exist in the catalog. I am not sure which db2 service you want to recommend - I suspect you were thinking "Db2" formerly DashDB for Transactions but am not sure

It's linked above intentionally.  I've moved the link to this resource and Lee has merged