easier way to pass extensions and test support

First off, thanks for making this library!
I've been working on various projects that use xapi/cmi5 and we've built a couple versions of js libraries to support it, but I'd like to switch to your lib.
The main blocker for us making that switch is that we have some apps that depend on passing result extensions with pass/fail/complete statements, and the signatures for those functions in this lib don't support passing extensions.
Would you be open to that feature in principal? If you are, we can make a fork, do the work and PR back to you.
I'm imagining the best and most general purpose way to handle this would be an optional callback function that transforms xapi statments before send, e.g.
// a function that can apply an arbitrary transform
// to an xapi Statement, e.g. add extensions
interface StatementTransformFunction {
  transform: (s: Statement) => Statement;
}

// a catch-all for options that might be
// passed to cmi-send-statement functions
interface SendCmiStatementOptions {
  transform?: StatementTransformFunction;
}

class Cmi5 {
  public complete(s: Partial<Statement>, options: SendCmiStatementOptions = {}) : AxiosPromise<string[]> {
    ...
    this. sendCmi5DefinedStatement({...}, options)
  }

  public sendCmi5AllowedStatement(
    statement: Partial<Statement>,
    options: SendCmiStatementOptions = {}): AxiosPromise<string[]> {
    ...
    const mergedStatement: Statement = deepmerge.all([cmi5AllowedStatementRequirements, statement]) as Statement;
    const sendStatement = (options.transform)
      ? options.transform(mergedStatement)
      : mergedStatement;
    return this.connection.sendStatement(sendStatement);
  }
}

For motivation, the custom extension we most commonly use is one that supplies granular 'skill' scores to the result of an assignable unit. For example, in some interactive-training assignable units we have, the result might look like this:
{
  "result": {
    "score": { "scaled": 0.8 },
    "extensions": {
      "https://ict.usc.edu/xapi/result/extension/skills": {
        "asks": 0.2,
        "evaluates": 0.9,
        "listens": 0.5,
      }
    }
  }
}

Curious if you know of a a more standards-based way to do the equivalent?
The last thing I want to add is that the cmi5 libraries we've developed inhouse have a suite of tests. Since they're testing cmi implementation over axios (via spies), should be not too hard to port our tests over to your library. Would you want those basic tests as a (separate) PR?

Hey @beatthat - thanks for getting in touch, and its nice to know somebody is using the library other than myself! I'd love to see how the library could accommodate the needs of your projects, I haven't worked a great deal with extensions yet so bare with my understanding.
I like your idea of being able to transform requests, but I'm not sure if bundling an extra parameter with each method would be the cleanest approach (at least for now with the lack of other config required per request). From using libraries like Axios, you have the ability to set a default transformRequest for all calls when initialising the library instance.
This could be an optional options parameter passed in the constructor, containing a transformRequest/transformStatement option. This option can then be picked up within the sendCmi5AllowedStatement method (exactly like you have demonstrated above).
One downside I can think of going for this approach would be having to identify what type of statement/request is being made and only modifying the intended statements.
Let me know what you think!
PS - tests sound good. I have some very basic tests implemented on the @xapijs/xapi and @xapijs/scorm libraries using Jest, but haven't brought anything into this library yet.

Thanks for getting back so quickly. I don't know if the global option would work for our case because we really just want to pass specific extensions on specific statements.
I definitely understand the concern about cluttering the API. Just to make sure clear though, if we went with the options route, that would be an optional parameter so the api would be backwards compatible. The reason I suggested making that new (optional) param an options object instead of just the transform function as a standalone was to leave space for any other options you might want to add in the future without breaking backwards compatibility in user call signatures
RE tests: let me take a look next couple days at forking this repo and just see how hard it would be to apply the tests we already have

After sleeping on the idea, I think your approach is the most suitable 👍 I'm happy for you to proceed with a PR of these improvements!
For some of the methods e.g. interaction helpers, they have quite a lengthy list of parameters already. I'm considering combining these into a single object instead whilst the library is still in pre-release stages in case any additional parameters for statements need to be added in the future.

Great! I'll have some time to fork and start that PR this afternoon. Will send you a draft when ready.

Nice one. If you want to take a look at how I've tested the other xapijs libraries, each has a .spec.ts file:
https://github.com/xapijs/xapi/blob/master/src/XAPI.spec.ts
https://github.com/xapijs/scorm/blob/master/src/SCORM.spec.ts
In terms of CI, GitHub Actions is preferred and the other libs use that too! You can see my crude attempt here:
https://github.com/xapijs/xapi/blob/master/.github/workflows/nodejs.yml

Great. Have an initial draft PR here. I think repo setting maybe don't allow me to request review?
I don't have github actions in yet, but was going to add that next.
Let me know what you think


Extended  M3U

I got an M3U link from my IPTV service, but anytime I load It, xteve asks for an "extended M3U". I ve configured one before and it did not ask me for that. Please help

m3u_plus

i have the same issue. my m3u url already has type=m3u_plus and its still happening


Moving example description

Moving example description under the code block trying to fix the formatting error which prevents the whole parameter section from showing in the published version of the article.
The output of cmdlet is not normally shown on cmdlet documentation so that could be removed later too.
@Kateyanne @yogkumgit this is regarding the email I sent.

Docs Build status updates of commit c65d60b:
:clock10: Full build: the average full build time is 17 min(s) 37 sec(s), this is based on the last 1000 build(s) for this repository.

Docs Build status updates of commit c65d60b:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

teams/teams-ps/teams/Remove-CsGroupPolicyAssignment.md
:white_check_mark:Succeeded
View (teams-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

@Kateyanne Please merge this PR. Thanks

This has been merged.
<EMAIL_ADDRESS>
Thanks,
Kateyanne
From: Yogesh Kumar<EMAIL_ADDRESS>Sent: Monday, August 10, 2020 8:00 AM
To: MicrosoftDocs/office-docs-powershell<EMAIL_ADDRESS>Cc: Kateyanne Unullisi (Aquent LLC<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [MicrosoftDocs/office-docs-powershell] Moving example description (#6294)
@Kateyannehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FKateyanne&data=02|01|v-kaunu%40microsoft.com|b2087672df1b4834fc1708d83d3e0679|72f988bf86f141af91ab2d7cd011db47|1|0|637326683903804916&sdata=bIUzQ9TKTXnaxixxQcBBWrlSgJ4aqXnucQgDYbgWUlk%3D&reserved=0 Please merge this PR. Thanks
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Foffice-docs-powershell%2Fpull%2F6294%23issuecomment-671407725&data=02|01|v-kaunu%40microsoft.com|b2087672df1b4834fc1708d83d3e0679|72f988bf86f141af91ab2d7cd011db47|1|0|637326683903804916&sdata=0xfJlxQrvSP0EzqWnP%2FggA1uASN4HkvMu37%2FU97qgS0%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAQD2HUSAQGQ6YPP42KYN7TDSAADOHANCNFSM4PYUPA7A&data=02|01|v-kaunu%40microsoft.com|b2087672df1b4834fc1708d83d3e0679|72f988bf86f141af91ab2d7cd011db47|1|0|637326683903814909&sdata=Ve6GBk2v%2BcMBoPYfpJhmNHDyChVCaKEKqNJUWmzlTTo%3D&reserved=0.


minor changes to monorepo dockerfile to facilitate building on mac si…

The test for ARM does not seem to be working correctly - grep -q is not doing what we expect it to? Grep -i seems a partial fix, but the if () is still not executing properly.
Also added the missing --platform linux/amd64 flag for Docker in case of Mac silicon

Codecov Report

Merging #34 (d9db560) into develop (1d3ee98) will increase coverage by 40.68%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##           develop      #34       +/-   ##
============================================
+ Coverage    46.03%   86.72%   +40.68%
============================================
  Files           91       34       -57
  Lines         3669     1122     -2547
  Branches       699      162      -537
============================================
- Hits          1689      973      -716
+ Misses        1980      149     -1831

Flag
Coverage Δ

boba-contracts
?

contracts
86.72% <ø> (ø)

core-utils
?

data-transport-layer
?

message-relayer
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...ontracts/contracts/oracle/payment/Subscription.sol

packages/core-utils/src/batches.ts

...acts/contracts/oracle/payment/AccessController.sol

packages/core-utils/src/provider.ts

...contracts/contracts/standards/L2StandardERC721.sol

...ontracts/DAO/governance/GovernorBravoDelegator.sol

...contracts/DAO/governance/GovernorBravoDelegate.sol

.../contracts/libraries/Lib_ResolvedDelegateProxy.sol

...boba/contracts/contracts/oracle/FluxAggregator.sol

...ba/contracts/contracts/DAO/governance/SafeMath.sol

... and 47 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1d3ee98...d9db560. Read the comment docs.

This isn't really working as expected. based on logs:
2022-03-23T03:59:01.3123908Z #7 4.320 Building for Mac (linux/amd64)
2022-03-23T03:59:01.3124256Z #7 4.320 /bin/sh: 1: [[: not found

it builds for mac on a ubuntu host.

on debian arm64:
inomurko@ip-172-31-5-5:~/opt/boba/ops$ docker-compose build builder
Building builder
Sending build context to Docker daemon  421.5MB
Step 1/44 : FROM node:16-buster-slim as node
 ---> d2dde0aca1e9
Step 2/44 : ARG TARGETPLATFORM
 ---> Running in 90a77408e15a
Removing intermediate container 90a77408e15a
 ---> 99c5c1c97041
Step 3/44 : ARG BUILDPLATFORM
 ---> Running in a2d9491089a1
Removing intermediate container a2d9491089a1
 ---> 63301954cb91
Step 4/44 : RUN echo "I am running on $BUILDPLATFORM, building for $TARGETPLATFORM"
 ---> Running in 6d9558e55396
I am running on , building for
Removing intermediate container 6d9558e55396
 ---> fbb6dcbc49a2
Step 5/44 : ARG TARGETPLATFORM
 ---> Running in 408d32525fd8
Removing intermediate container 408d32525fd8
 ---> c5d767a8af70
Step 6/44 : RUN echo "Platform: $TARGETPLATFORM"
 ---> Running in 214c9e67249a
Platform:
Removing intermediate container 214c9e67249a
 ---> 4baa90fcb196
Step 7/44 : RUN if [[ "$TARGETPLATFORM" == "linux/amd64" ]] ; then   echo "Building for x86_64" ;   apt-get update -y && apt-get install -y git ;   lse if [[ "$TARGETPLATFORM" == "linux/amd64" ]]   echo "Building for Mac ($TARGETPLATFORM)" ;   apt-get update -y && apt-get install -y git python3 build-essential ;   fi
 ---> Running in 2baf2fd85260
/bin/sh: 1: [[: not found

interesting - seems like ARG TARGETPLATFORM is not supported on debian?

Darwin Boyuans-MBP 21.4.0 Darwin Kernel Version 21.4.0: Mon Feb 21 20:35:58 PST 2022; root:xnu-8020.101.4~2/RELEASE_ARM64_T6000 x86_64

Darwin Inos-MacBook-Pro.local 21.4.0 Darwin Kernel Version 21.4.0: Mon Feb 21 20:34:37 PST 2022; root:xnu-8020.101.4~2/RELEASE_X86_64 x86_64

Linux ip-172-31-5-5 5.10.0-12-cloud-arm64 #1 SMP Debian 5.10.103-1 (2022-03-07) aarch64 GNU/Linux

but on debian aarch64 buildplatform is not defined, so I propose we stick with uname -a amd fix for sillicon:
RUN if echo "\"`uname -a`\"" | grep -q -E -i "(arm64|aarch)" ; then \
     apt-get update -y && apt-get install -y git python make gcc libudev-dev libusb-1.0-0-dev g++ pkg-config ; \
     else \
     apt-get update -y && apt-get install -y git; \
     fi


Collect Location Method

Collect the method in which the location was determined as string under the "location_method" field.
Known values: "wifi", "cellular", "fused", "gps"
"location_method": "gps",

Not possible on iOS.


⚠ Bofrost.lu has degraded performance

In 2b3dc5f, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 999 ms

Resolved: Bofrost.lu performance has improved in c6825b5.


Mobile's back button not working - backbone.js 0.5.3 with jquerymobile 1.0 RC2

When using the navigate method with triggerRoute = true; the page navigates fine to the new page but history is not updated.
When hitting the back button on an android device (nexus S), the history pops, but the navigation does not happen.
If I pass "false" for the  triggerRoute and then call "Backbone.history.loadUrl();"  Then the back button does work but sort of erratically.
The navigate method has this comment ....
// URL-encoding the fragment in advance. This does not trigger
// a hashchange event.
After reading several posts, it seems to me that using the navigate method is the right way and should update the history...
Code snippet ...
er.getApp().getController().navigate('home', true);
OR
er.getApp().getController().navigate('home');
Backbone.history.loadUrl();
Is there a known routing issue with jquerymobile and backbone combination ?

I haven't heard of one before, but I wouldn't exactly be surprised. I think that lots of folks have had trouble integrating with jQuery mobile. I'm also afraid that I can't do more to diagnose this for you, without a smartphone or a jQuery mobile app. But if you come up with a change in the future that you think should be made -- let us know.

Here's the stackoverflow link with the resolution:
http://stackoverflow.com/questions/8100846/mobiles-back-button-not-working-backbone-js-0-5-3-with-jquerymobile-1-0-rc2/8106520#8106520
I am noticing that the render method is getting called twice ... after a window.history.back(); ...
Will look into it and see if its my code or if there is a known issue / solution


Document list of template-able fields

Summary
At the moment, folks have to guess which fields are template-able or infer from examples. Ideally, folks could look in a list of field names indicating which version of Argo first supported templating in that field.
Use Cases
Whenever I'm trying to template a field that I haven't successfully templated before.

Message from the maintainers:
Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.

The Workflow Variables Reference has pretty good coverage of this, so any improvements I think can go into that section.

@agilgur5 that covers what variables you can use, but not where you can use them.

It does say what type of templates they are available in, but not to the granularity of what fields they can be used in. So I think it partially covers that.


net: support DNS-over-HTTPS

(This is feature request for an, as of Sept 2018, experimental protocol and is meant as a placeholder. It's likely that this protocol will become unexperimental and having thoughts and work on it early would be useful)
DNS-over-HTTPS is an evolution of DNS that allows us to secure the stream of DNS requests from a system or user in the same way we secure HTTP traffic with TLS. As of Sept 2018, it's currently deployed in two of the major browsers (Mozilla[1], Chrome) and 2 major service owners (Cloudflare[2], Google[3])  and has the backing of lot of folks in the field as what we need to fix DNS.
There are a few Go implementations[4] already on GitHub, but those require buy-in and awareness to be used. Having it baked into the Go standard library would be an incredible boon in securing our systems and users.
[1] https://blog.nightly.mozilla.org/2018/06/01/improving-dns-privacy-in-firefox/
[2] https://developers.cloudflare.com/<IP_ADDRESS>/dns-over-https/
[3] https://developers.google.com/speed/public-dns/docs/dns-over-https
[4] e.g. https://github.com/coredns/coredns/pull/1619

I'd imagine that it would be best to first implement this outside the standard library, much like how HTTP2 was first implemented in x/net. So perhaps https://github.com/golang/go/issues/16218 should be done first.
I also wonder if https://github.com/golang/go/issues/12503 would be a requirement for this, to be able to swap between different resolvers.
I am also personally a bit confused by DNS over TLS versus DNS over HTTPS. Do we want to support both? If not, which one should we prefer and why?

I think much of #16218 is in https://godoc.org/golang.org/x/net/dns/dnsmessage
DNS over TLS is a different protocol, but, confusingly, is sometimes used to refer to both DNS over TLS and DNS over HTTPS (since HTTPS relies on TLS). The trick with DNS over TLS is that it requires a new port to be opened up in people's firewalls, and the HTTP/2 and SPDY Chrome experiments determined that new ports would prevent adoption (and so they were kept to 443). So, tunneling over 443 and reaping the benefits of solid HTTP clients has been taking the mindshare.

I'd imagine that it would be best to first implement this outside the standard library, much like how HTTP2 was first implemented in x/net. So perhaps #16218 should be done first.

I've been playing with a toy cli-client that uses net/http and  x/net/dns/dnsmessage. it was quite pleasant.

@jmhodges while I'd rather not turn this thread into a protocol debate, "TLS on port 443" appears to leave ALPN on the table. I'm unsure what specifically narrows things to HTTP/2 on that front..?

I am also personally a bit confused by DNS over TLS versus DNS over HTTPS. Do we want to support both? If not, which one should we prefer and why?

@mvdan DNS over TLS is already supported with trivial amount of code: https://github.com/artyom/dot/blob/master/dot.go

As side note, another DoH implementation in Go is also available at https://godoc.org/github.com/shuLhan/share/lib/dns#DoHClient .

net.Resolver instances with caching, opportunistic encryption, DNS-over-TLS and DNS-over-HTTPS: github.com/ncruces/go-dns


The cursor becomes very large

Hyprland Version
 System/Version info
Hyprland, built from branch main at commit 3529fbc6d4db298c4d50a39732e59ca38b463c9f dirty (compositor: fix getMonitorFromVector getting wrong monitor (6010)).
Date: Sat May 11 17:35:20 2024
Tag: v0.40.0-60-g3529fbc6, commits: 4666

flags: (if any)

System Information:
System name: Linux
Node name: Arch
Release: 6.8.9-1-cachyos-echo-lto
Version: #1 SMP PREEMPT_DYNAMIC Fri, 03 May 2024 15:07:15 +0000

GPU information:
64:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Phoenix1 [1002:15bf] (rev c7) (prog-if 00 [VGA controller])

os-release: NAME="Arch Linux"
PRETTY_NAME="Arch Linux"
ID=arch
BUILD_ID=rolling
ANSI_COLOR="38;2;23;147;209"
HOME_URL="https://archlinux.org/"
DOCUMENTATION_URL="https://wiki.archlinux.org/"
SUPPORT_URL="https://bbs.archlinux.org/"
BUG_REPORT_URL="https://gitlab.archlinux.org/groups/archlinux/-/issues"
PRIVACY_POLICY_URL="https://terms.archlinux.org/docs/privacy-policy/"
LOGO=archlinux-logo

plugins:

Bug or Regression?
Bug
Description
Cursor gets big in some software.
hyprland.conf
env = HYPRCURSOR_THEME,Bibata-Modern-Ice
env = HYPRCURSOR_SIZE,24
env = XCURSOR_THEME,Bibata-Modern-Ice
env = XCURSOR_SIZE,24

exec = hyprctl setcursor Bibata-Modern-Ice 24
exec = gsettings set org.gnome.desktop.interface cursor-theme 'Bibata-Modern-Ice'
exec = gsettings set org.gnome.desktop.interface cursor-size 24

How to reproduce

Installing hyprland built with the main branch
Cursor gets big on desktop or chromium

Crash reports, logs, images, videos
https://github.com/hyprwm/Hyprland/assets/39702328/79dba9ef-27b2-448f-a166-07cedac3ddce

#6017

#6017

With fractional scaling, the cursor size becomes very large, without fractional scaling it is normal.


Feature request: in precompiled functions, support anonymous arguments instead of an object

It would be sweet to be able to write this:
var welcome = mf.compile("Welcome back, {$1}! It's {$2}.")
welcome("John", "Monday") // Welcome back, John! It's Monday.

This sprintf-like notation is shorter and easier to use, since naming stuff is one of the hardest things in programming.
Compiled function can look like this:
function (a1, a2) { return "Welcome back, " + a1 + "! It's " + a2 + "."; }

.. or like this:
function () { return "Welcome back, " + arguments[0] + "! It's " + arguments[1] + "."; }

Though the first option looks nicer.
It should be compatible with the current object-style notation:
var welcome = mf.compile("Welcome back, {name}! It's {$2}.")
welcome({name: "John"}, "Monday") // Welcome back, John! It's Monday.

I'd consider that an anti-pattern, but the point is - you can use different styles for different messages, so old code won't break.

As a workaround, it's currently possible to use an array instead of an object, and then use indexes as keys:
var welcome = mf.compile("Welcome back, {0}! It's {1}.")
welcome(["John", "Monday"]) // Welcome back, John! It's Monday.

But it's confusing to read, especially if there's only one argument.

I'm not convinced that this would be a pattern that would prove "easier to use". Especially as we're talking about replacing words, it's hard to think of an actual example where an appropriate name could not be found for the variable in question. Your example, for instance, could work well as:
var welcome = mf.compile("Welcome back, {name}! It's {day}.")
welcome({ name:"John", day:"Monday" }) // Welcome back, John! It's Monday.

... which has the added benefit of making it clear what the replacements are for. If using numerical indices only, "Welcome back, {$1}! It's {$2}." could just as easily turn into "Welcome back, visitor! It's four o'clock."
Also, given that the generated function requires some {object} or [array] notation for its input parameter, and that numerical indices already work with the latter, this seems like syntactic sugar that just adds complexity.

True. Now that I've found that arrays work as well, this feature is practically already implemented.
Thanks for your response. :)


Pay expense to a Collective budget (in the same host)

I mentioned this in #3374 but I think it needs its own issue.
User story
As an expense submitter, I want to be able to select a Collective under the same host as the payee for an expense (or the host budget if it has an integrated budget).
This is effectively the same thing as going to a Collective and going through the contribution flow and selecting another Collective balance as the payment source - it moves money between Collectives without any money actually moving in bank accounts.
The reason we should enable this via expenses in addition to contributions is for transparency and good accounting practice. If one Collective does work for another, they should be able to submit an expense to reflect that accurately.
Moving money they way should not trigger the tax form because no money is being paid out. It's still sitting in the same host bank account, just credited to a different Collective budget.
Probably at first we should only allow people to select payee Collectives that they are admin of.
The expense should go through the same approval process as any other expense.
Ideally this would work across hosts but that's way more complicated so I'm not suggesting it.

Note this would also apply to host budgets, as Hosts and their Collectives sometimes do work for each other too.

This is done, right?

@znarf this is done right?

@znarf this is done right?

@alanna Can you try it? Does it work for you?

@alanna Can you try it? Does it work for you?

This one worked so I guess it's good to go https://opencollective.com/opensource/expenses/30480

This one worked so I guess it's good to go https://opencollective.com/opensource/expenses/30480


hubble: add a couple of "any interface" filter test cases

Explicit how a Hubble API user can request flows with "any interface" set, by setting a non-empty repeated interface filter (e.g. non-empty slice from Go) with a zero values filter. Like other filters, a logical inverse of this "any interface" filter can be queried using the same construction under the GetFlowsRequest.blacklist set of filters instead of GetFlowsRequest.whitelist.

/test

/test


Parameter file is updated, error comes out when I run deploy.ps1

Here is the error message that I received:
ADAppUpdateDisplayName : Cannot bind argument to parameter 'appId' because it is null.
At C:\temp\microsoft-teams-apps-company-communicator-main\Deployment\deploy.ps1:1085 char:26

    ADAppUpdateDisplayName $userAppCred.appId $currentAppName $us ...

                           ~~~~~~~~~~~~~~~~~~

CategoryInfo          : InvalidData: (:) [ADAppUpdateDisplayName], ParameterBindingValidationException
FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,ADAppUpdateDisplayName

Please advise, thanks.

Hi @chan228228 ,
Thanks for raising the query. Please follow the manual deployment guide to unblock yourself. There is a issue with PowerShell deployment which will be addressed in the future releases of CC


picture format question

Hi!
The pictures in my datasets are '.bmp' format,and it doesn`t work in voc_form .What can i do to train the pictures?
Thx

You may train without any modification as long as the images can be opened with cv2.imread().

thx a lot， i`ll try

@hellock
I find the following codes in xml_style.py
            filename = 'JPEGImages/{}.jpg'.format(img_id)
            xml_path = osp.join(self.img_prefix, 'Annotations',
                                '{}.xml'.format(img_id))

it seems that for VOC format only jpg is supported?
thank you co much.


Highlight the search results in scrollbar

I hope in pdf viewer, the search results could be shown with highlighted small bars in the scrollbar. That will be very convenient to overview the search results.
Chrome, Edge, those browsers have this functionality, both for webpage and pdf. But Firefox has this feature only for the webpage, not for pdf. I do not know if it is an issue with pdf.js. PDF Xchange also has this feature. Hope this feature could be implemented in pdf.js.

Duplicate of https://bugzilla.mozilla.org/show_bug.cgi?id=1693784

Please note that searching in PDF.js is completely asynchronous, and that only a limited number of pages rendered at once, hence this feature will unfortunately not be possible to implement particularly well for most PDF documents (or it'd be very inefficient).


Add icon to nao

the icon is checked in to nao's source directory, but need to update package file and draft file


Create animal table

This project involves the creation of a database with an animal table, inserting it's data, and making queries for the table.

Thanks for this!🙏
On Wed, 20 Sep 2023 at 20:48, Bantealem Geto @.***>
wrote:

@.**** approved this pull request.
Status: Approved
Hi @Wils-FOURR-speed https://github.com/Wils-FOURR-speed
Your project is complete! There is nothing else to say other than... it's
time to merge it [image: :shipit:]
Congratulations! 🎉
Highlights

Good implementation
Correct Git flow
Professional Readme.

Optional suggestions
Every comment with the [OPTIONAL] prefix won't stop the approval of this
PR. However, I strongly recommend you to take them into account as they can
make your code better. Some of them were simply missed by the previous
reviewer and addressing them will really improve your application.
Cheers and Happy coding!👏👏👏
Feel free to leave any questions or comments in the PR thread if something
is not 100% clear.
Please, remember to tag me in your question so I can receive the
notification.
As described in the Code reviews limits policy
https://microverse.zendesk.com/hc/en-us/articles/1500004088561 you have a
limited number of reviews per project (check the exact number in your
Dashboard). If you think that the code review was not fair, you can request
a second opinion using this form https://airtable.com/shrQAqnBwek5a0O0s.
—
Reply to this email directly, view it on GitHub
https://github.com/Wils-FOURR-speed/vet-clinic/pull/1#pullrequestreview-1636383465,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A5TIO5TDTWQAV6GTBBOLE53X3NCBTANCNFSM6AAAAAA5AMWFAA
.
You are receiving this because you were mentioned.Message ID:
@.***>


Remove HashedElGamalCiphertext.numBytes from class and proto

numBytes == c1.size

Not sure this is right, closing


Conflicting documentation - Management Groups

Hello,
Conflicting documentation regarding Mangement Group strutucture.
https://docs.microsoft.com/en-us/azure/cloud-adoption-framework/ready/landing-zone/design-area/resource-org-management-groups:
"Don't create management groups for production, testing, and development environments. If necessary, separate these groups into different subscriptions in the same management group. To review further guidance on this topic, see:"
https://docs.microsoft.com/en-us/azure/cloud-adoption-framework/ready/azure-best-practices/organize-subscriptions:
"Do you expect your Azure environment to grow? Then create management groups for production and nonproduction now, and apply appropriate policies and access controls at the management group level. As you add new subscriptions to each management group, those subscriptions inherit the appropriate controls."
Best Regards,
Rasmus

Thank you for your feedback.  We have created internal ticket 24464 to track and investigate the issue.

#please-close


Rename environment prefix, app name and test database name to fleetdb

We are using FLEETDB_ instead of SERVERSERVICE_ environment variable prefix for configuring fleetdb.
Hi @DoctorVin and @joelrebel ,
We also need to update corresponding environment variables in https://github.com/equinixmetal/k8s-fleetdb/tree/main
It may be simpler for us to merge this PR and change k8s-fleetdb before launching fleetdb on production?
Keep using ServerService_ is also convenient. We can rename the prefix later after fleetdb has already been launched.
I am fine with either. I may need to update the template on the sandbox work depending on the decision.
Do we want to sync up or create a document to discuss the migration plans?

Since we have almost finished migration, we can start to push new releases.


Promote VisibilityOnDemand to Beta

What would you like to be added:
Promote the VisibilityOnDemand to Beta.
Why is this needed:

the feature is assumed to be enabled by kueuectl "POSITION IN QUEUE" column in the kueuectl list workload command
The feature is in alpha quite long already
The enablement involves 2 steps which is sometimes confusing to users

Completion requirements:
This enhancement requires the following artifacts:

[ ] Design doc
[ ] API change
[ ] Docs update

The artifacts should be linked in subsequent comments.

/cc @mwielgus @mwysokin @PBundyra @alculquicondor

From the graduation criteria:

Release the API in beta and guarantee backwards compatibility,

This is the objective of the change.

Reconsider introducing a throttling mechanism based on user and review feedback,

I'm not aware of users' feedback to justify the effort. We can move it to GA criteria.

Consider introducing FlowScheme and PriorityLevelConfiguration to allow admins to easily tune API priorities.

Similar, I'm not aware of the feedback to justify the effort, I would suggest moving it to criteria for GA.
There were also ideas for new endpoints like https://github.com/kubernetes-sigs/kueue/pull/2145, but it is not clear they are needed for now. I think we should be ok to graduate the API we have. We should be good to still add the new endpoints while in Beta if needed.

An important question when graduating is whether we want to include visibility-api.yaml as part of the main YAML.
I suppose there are no downsides.

/assign

An important question when graduating is whether we want to include visibility-api.yaml as part of the main YAML.
I suppose there are no downsides.

I see no downsides, so I would embed it for simplicity of deployment.

@mimowo @alculquicondor What about QueueVisibility? Should we graduate it too?

No, we aim to deprecate it, and remove in the long run.

Shouldn't we add some guard that we planned in the design phase?
https://github.com/kubernetes-sigs/kueue/tree/main/keps/168-2-pending-workloads-visibility#graduation-criteria
IIUC, the current visibility endpoint, is not guarded by any mechanism. So, at least, I think it would be better to provide APF configurations, and it would be nice to have a throttling mechanism.

If we do not provide any guards for the kube-apiserver, I guess that the internet-exposed kube-apiserver faces some dangers.
Indeed, if we want to use the MultiKueue, we need to create a public k8s cluster, not a supported private cluster, right?

IIUC, the current visibility endpoint, is not guarded by any mechanism. So, at least, I think it would be better to provide APF configurations, and it would be nice to have a throttling mechanism.

I think there is a basic protection mechanism by APF, assigning one unit (seat) to every request. This may not be very accurate as listing workload may take proportionally more time.

Indeed, if we want to use the MultiKueue, we need to create a public k8s cluster, not a supported private cluster, right?

I'm not sure how MultiKueue changes that. The single regular clusters might be public, but there is authentication which would prevent external actors.

But I agree it requires some extra research. In particular, I it would be good to refresh the knowledge on the default protection mechanism, and see how much effort there is to provide a dedicated FlowSchema.
So, I would suggest to record the findings in the KEP first before the code PR.

I think there is a basic protection mechanism by APF, assigning one unit (seat) to every request. This may not be very accurate as listing workload may take proportionally more time.

I guess that it would be better to provide the kueue-sysytem namespace dedicated FlowSchema so that we can prevent the kueue-manager occupies the kube-apiserver seats when the visibility endpoints are attacked by malicious people.
But, before we introduce the FlowSchema, we need to investigate a little bit if the mechanism improves the problems.

The single regular clusters might be public, but there is authentication which would prevent external actors.

Yes, that's right. But, even if the malicious people do not have the credentials to access the kube-apiserver, they can give high load to kube-apiserver via visibility endpoints.

I guess that it would be better to provide the kueue-sysytem namespace dedicated FlowSchema so that we can prevent the kueue-manager occupies the kube-apiserver seats when the visibility endpoints are attacked by malicious people.
But, before we introduce the FlowSchema, we need to investigate a little bit if the mechanism improves the problems.

@mbobrovskyi is investigating this, and will summarize the findings, but from what we see the default global-default schema of the P&F mechanism protects the endpoint. Yes, we can introduce a custom FlowSchema or Kueue, but it is a complication and I'm not sure it is needed. We don't have any feedback from users that the built-in mechanism is not enough.

Yes, that's right. But, even if the malicious people do not have the credentials to access the kube-apiserver, they can give high load to kube-apiserver via visibility endpoints.

I don't think you can query the visibility endpoints without credentials, I would assume this is bad, but we may do an experiment on a life cluster.

Anyway, we can discuss this after the summary is made public.

I have confirmed that on GKE (and @mbobrovskyi checked on Kind) that the requests to the custom endpoint are authorized. In particular, I run:
curl -kH "Authorization: Bearer $ACCESS_TOKEN" -H "Accept: application/json" "https://$CLUSTER_ENDPOINT/apis/visibility.kueue.x-k8s.io/v1alpha1/clusterqueues/cluster-queue/pendingworkloads"

returns me results, while if I remove the Authorization header I get:
{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {},
  "status": "Failure",
  "message": "nodes is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope",
  "reason": "Forbidden",
  "details": {
    "kind": "nodes"
  },
  "code": 403
}

I get the analogous error for other built-in endpoints, such as https://$CLUSTER_ENDPOINT/api/v1/nodes

Yes, the Visibility API endpoints are protected by default with global-default FlowSchema and PriorityLevelConfiguration. Below is the configuration for both:
apiVersion: flowcontrol.apiserver.k8s.io/v1
kind: FlowSchema
metadata:
  annotations:
    apf.kubernetes.io/autoupdate-spec: "true"
  name: global-default
spec:
  distinguisherMethod:
    type: ByUser
  matchingPrecedence: 9900
  priorityLevelConfiguration:
    name: global-default
  rules:
  - nonResourceRules:
    - nonResourceURLs:
      - '*'
      verbs:
      - '*'
    resourceRules:
    - apiGroups:
      - '*'
      clusterScope: true
      namespaces:
      - '*'
      resources:
      - '*'
      verbs:
      - '*'
    subjects:
    - group:
        name: system:unauthenticated
      kind: Group
    - group:
        name: system:authenticated
      kind: Group
---
apiVersion: flowcontrol.apiserver.k8s.io/v1
kind: PriorityLevelConfiguration
metadata:
  annotations:
    apf.kubernetes.io/autoupdate-spec: "true"
  name: global-default
spec:
  limited:
    lendablePercent: 50
    limitResponse:
      queuing:
        handSize: 6
        queueLengthLimit: 50
        queues: 128
      type: Queue
    nominalConcurrencyShares: 20
  type: Limited

I found it when I tried executing a large number of parallel requests to kubectl get --raw /apis/visibility.kueue.x-k8s.io/v1alpha1/clusterqueues/cluster-queue/pendingworkloads.
The kubectl get --raw /debug/api_priority_and_fairness/dump_priority_levels return a table with column DispatchedRequests that show us how much requests we have for this configuration.

The LimitResponse type in PriorityLevelConfiguration can be set to one of two options:

Reject: Excess traffic is immediately rejected with an HTTP 429 (Too Many Requests) error.
Queue: Requests exceeding the threshold are placed in a queue."

The global-default configuration uses the Queue type. This means that if there is a high volume of requests, they will be added to a queue and wait for their turn to be processed.
This is an example that what I have:
kubectl get --raw /debug/api_priority_and_fairness/dump_priority_levels
PriorityLevelName, ActiveQueues, IsIdle, IsQuiescing, WaitingRequests, ExecutingRequests, DispatchedRequests, RejectedRequests, TimedoutRequests, CancelledRequests
catch-all,         0,            true,   false,       0,               0,                 39,                 0,                0,                0
exempt,            0,            true,   false,       0,               0,                 2290,               0,                0,                0
global-default,    4,            false,  false,       57,              368,               23614,              1533,             0,                609
leader-election,   0,            true,   false,       0,               0,                 1170,               0,                0,                0
node-high,         0,            true,   false,       0,               0,                 743,                0,                0,                0
system,            0,            true,   false,       0,               0,                 557,                0,                0,                0
workload-high,     0,            true,   false,       0,               0,                 1706,               0,                0,                0
workload-low,      0,            true,   false,       0,               0,                 4595,               0,                0,                0

And also the apiserver_flowcontrol_rejected_requests_total metrics that what I have:
# This one rejected because queue full
apiserver_flowcontrol_rejected_requests_total{endpoint="https", flow_schema="global-default", instance="<IP_ADDRESS>:6443", job="apiserver", namespace="default", priority_level="global-default", reason="queue-full", service="kubernetes"} | 924
-- | --
# This one rejected due to time-out
apiserver_flowcontrol_rejected_requests_total{endpoint="https", flow_schema="global-default", instance="<IP_ADDRESS>:6443", job="apiserver", namespace="default", priority_level="global-default", reason="time-out", service="kubernetes"} | 719

Possible rejection:

queue-full, indicating that too many requests were already queued.
concurrency-limit, indicating that the PriorityLevelConfiguration is configured to reject rather than queue excess requests.
time-out, indicating that the request was still in the queue when its queuing time limit expired.
cancelled, indicating that the request is not purge locked and has been ejected from the queue.

I guess that it would be better to provide the kueue-sysytem namespace dedicated FlowSchema so that we can prevent the kueue-manager occupies the kube-apiserver seats when the visibility endpoints are attacked by malicious people.
But, before we introduce the FlowSchema, we need to investigate a little bit if the mechanism improves the problems.

I think you right, it should help. We will have different queues that allow us to do not block kube-system or another namespaces when kueue-system namespace is overloaded.
Just a question. Should we have dedicated FlowSchema for kube-system namespace or just for Visibility API. WDYT?

I think you right, it should help. We will have different queues that allow us to do not block kube-system or another namespaces when kueue-system namespace is overloaded. I also tested this scenario, and it worked well.
Just a question. Should we have dedicated FlowSchema for kube-system namespace or just for Visibility API? WDYT?

If we can configure the FlowScheme only for the visibility endpoint, it would be great since we want to continue the Kueue operation when the visibility endpoints are attacked by malicious people.
Is it possible to configure the FliowScheme only for visibility endpoints?

The main problem is that the visibility endpoints use the global-defaults FlowScheme since most of the controllers and workloads belong to the global defaults. So, when the global-defaults are occupied by the visibility endpoints, the cluster admin and batch users will lose way to operate cluster and themselves Jobs.

Is it possible to configure the FliowScheme only for visibility endpoints?

Yes, we can configure for visibility.kueue.x-k8s.io API group. I can prepare PR for it.

Is it possible to configure the FliowScheme only for visibility endpoints?

Yes, we can configure for visibility.kueue.x-k8s.io API group. I can prepare PR for it.

Thank you for letting me know. Could you wait for a while?
Actually, we are investigating the cloud provider specifications for FlowScheme. After we resolve some questions, @mimowo will comment here or KEP PR.

@mimowo @tenzen-y
It's a lot of task related to this issue. We should merge it in this order:

[ ] https://github.com/kubernetes-sigs/kueue/pull/3083
[ ] https://github.com/kubernetes-sigs/kueue/pull/3043
[ ] https://github.com/kubernetes-sigs/kueue/pull/3084
[ ] https://github.com/kubernetes-sigs/kueue/pull/3008

We still have a pending discussion about including P&F config: https://github.com/kubernetes-sigs/kueue/pull/3043#discussion_r1760626660. I think we may need to rework the PR to put it in docs, and update the KEP.


Adjust line length to favor readability

I've adjusted it to 120 in eslint and 119 in prettier. After some tests I concluded that with this configuration we would be able to keep using VSCode keybindings to automatically format the code.
Please feel free of share your opinions about the size, or raise any problem you may encounter formatting the code.

Coverage increased (+0.2%) to 88.052% when pulling 31948ad0f70f16e3918440d9608c8f14d635ecb8 on configure-prettier-eslint-line-length into f1d434ae173bc809d5560f04cadebd13eb011f90 on main.

@fmrsabino I just removed plugins and integrations between eslint and prettier. Now I'd expect both tools to do their job separately. As we talked prettier will keep formatting the code on a pre-commit hook, but lets keep in mind we could remove this behaviour anytime if we're not confortable with it. 🙂


Calculator in Standard wont won't keep a multiline entry ex previously 1+1+1=3 now it shows 1+1=2, 2+1=3 despite being entered as 1+1+1=

Describe the bug

Steps To Reproduce

Expected behavior

Screenshots

Device and Application Information

OS Build:
Architecture:
Application Version:
Region:
Dev Version Installed:

Additional context

Requested Assignment

If possible, I would like to fix this.
I'm just reporting this problem.  I don't want to fix it.

This is your friendly Microsoft Issue Bot. I've seen this issue come in and have gone to tell a human about it.

This was a really useful featurette, surely it's disappearance is a regression?

 Help please .thanks

This used to give buggy display. If you entered sequentially 1+1x2, it would actually display 1+1x2 but give you 4 as result.

I really liked this feature and I'd like to see it come back, at least as an option.

In issue #138, some people are confused about the display of standard mode and think the way it has been displayed is wrong.
But this one-line display method is no problem for us senior accustomed to using traditional tabletop calculators.
Hope the one-line display can be just an optional switch.
(To be honest, I communicated with some young people. Their experience of calculators all comes from Android phones or iPhone.

I need this feature back. Scientific & Programmer calculator still have this feature. I'm not sure why remove it from Standard calculator.

Standard Calculator mode works in immediate-execution mode, just as a simple pocket calculator would. This led to confusion and a significant amount of feedback around this mode appearing to not follow order of operations, because we were displaying expressions in a way that was inconsistent with evaluation in immediate-execution mode.
As part of the fix for #138, the output now more accurately captures the operations being performed in this mode. We will continue to monitor feedback on this issue, but I am going to close this issue as by design for now.

Surely a shame to lose a useful feature! Is it logically impossible to reconcile the two issues, otherwise could you raise an issue for doing this?


Disable the need for registry edits

Although it is nice that with a registry edit you can enable signature override but this might have implications for other games and certain anti-cheats.
It is possible to enable signature override within Cyberpunk by setting a single byte to 0x1 at the address 0x4B9AA0C and any registry modifications will not be needed.
You would most likely need another DLL to make the modifications before nvngx.dll is loaded.
Address can be found by searching for referenced string "OVERRIDE DETECTED" and can be found at the start of the function where a pointer to that address can be found.

Its a bit more nuanced than that, not all game uses same check, also some games apply different checks even before the  "OVERRIDE DETECTED" part, like in HZD the dll loads fine for Nvidia users (after the registry patch of course), but does not load on AMD even with the reg patch, so I had to find another check that happens before the "OVERRIDE DETECTED" part, thus we have to consider that the patching is going to be different for some games. So the mod needs to be flexible enough to consider such cases.

Yes, I got the idea of bypassing the driver check from your repo, so thanks for that, as I never done any IDA related work before. And I wasn't saying we need the registry, I was just chiming in to say, while we are at it (patching the exe instead of reg) we should also find a flexible way to patch game specific stuffs, maybe creating a separate issue for this would be better.

Yes, I got the idea of bypassing the driver check from your repo (changing JNZ to JZ), so thanks for that, as I never done any IDA/assembly related work before. And I wasn't saying we need the registry, I was just chiming in to say, while we are at it (patching the exe instead of reg) we should also find a flexible way to patch game specific stuffs, maybe creating a separate issue for this would be better.

A flexible method of applying patches pre-nvngx load would be useful. Right now the best method is to find the name of another DLL the game would load and patch with that but it might be possible to write a service program to auto-patch a game upon launch. Even better, could be done automatically if you can grab the signature for the function that contains a pointer to the signature override byte and just find the instructions offset and get the pointer from the mov instruction.

That's a game specific check, not the override im referring to. The check you are referring to can be bypassed by switching the jz to jnz or nopping out the responsible instruction entirely which is what my fork does for RDR2.

Any hint on which function to look for? I'm trying to do SOTTR.


Remove duplicated documentation test code that's been moved to the driver

Fixes #637
We've already removed the actual documentation, and other tests should already be covering the use cases in this test.
@sbellem I'm not sure if you're still using this file for the driver, if you are then we can wait (or you can refer to a previous commit when adding more usage cases for the driver).

:+1:

@sbellem I'm not sure if you're still using this file for the driver, if you are then we can wait (or you can refer to a previous commit when adding more usage cases for the driver).

We can remove it.

AFAIK @diminator wrote that file of tests to test the BigchainDB Python API by Example page --- a page that is now gone from the docs.

@ttmc we'll have to revise that entire test file using the new driver ofc

@diminator :
@sbellem copied all those examples over to the Python driver docs page about "Advanced Usage" with notes about what's not supported yet: see https://docs.bigchaindb.com/projects/py-driver/en/latest/advanced-usage.html
Maybe you can share some ideas with him about how to test those.


THEEDGE-1635 Pod crashes in uploader.go + log flushes + initial loggi…

…ng unit tests
Description
Change pod crash in uploader.go to log the error, then flush the logs before panic'ing.  Also added initial logging unit tests.
Fixes # THEEDGE-1635
Type of change
What is it?

[X] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Documentation update
[X] Tests update
[ ] Refactor

Checklist:

[X] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[X] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[X] I run go fmt ./... to check that my code is properly formatted
[X] I run go vet ./... to check that my code is free of common Go style mistakes

Codecov Report

Merging #656 (99ba222) into main (4db32ff) will increase coverage by 0.17%.
The diff coverage is 12.50%.

@@            Coverage Diff             @@
##             main     #656      +/-   ##
==========================================
+ Coverage   32.86%   33.03%   +0.17%
==========================================
  Files          37       38       +1
  Lines        4032     4068      +36
==========================================
+ Hits         1325     1344      +19
- Misses       2519     2537      +18
+ Partials      188      187       -1

Impacted Files
Coverage Δ

pkg/services/files/downloader.go
0.00% <ø> (ø)

pkg/services/files/uploader.go
14.78% <0.00%> (ø)

logger/logger.go
2.77% <14.28%> (ø)

pkg/clients/fdo/client.go
96.42% <0.00%> (+21.42%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4db32ff...99ba222. Read the comment docs.

/retest


[Import Genesis] Speeding up the import genesis time

Background
We need to understand what is happening during the import genesis and come up with a solution to improve time.
Acceptance Criteria

Import genesis time is well understood
A plan to mitigate the problem is created

Hi @moniya12 . Are you on this? If yes, what is the status? Thanks!

@p0mvn, I did some work on this and so far I've reduced the importing time by maybe a half. I did the profiling by running [this test file] (https://github.com/notional-labs/osmosis/blob/1fdff279e3a2ad816f56bb0fd1b4e4662ba2f55e/simapp/sim_test.go#L39) which contains the code needed for importing genesis used in the osmosisd start cmd.

@catShaark out of curiosity, how much time does it now take compared to what it did before? What changes did you make to achieve this?

Changes I did :

Improve the DelegatorSharesInvariant invariant check in the sdk. It reduces runtime from 241 to 161 sec in one of my computer.
makes the unmarshaling of modules' genesis state parallel, before we unmarshal modules' genesis sequentially. This reduces runtime for about 30-40 sec.

@catShaark So just to give you some more context, this issue was in reference to state exported testnets. Basically, when we take a state export, modify the values, then start a new testnet off this new genesis the following happens:

Init genesis starts, this process takes roughly 15 minutes
We hit the block after the snapshot was taken
We then pause on this block for about an hour and a half, using a large amount of RAM (almost 64GB)
After we hit the next block, we hit blocks like normal

I think your changes will help with number 1, but number 3 is where the most time is spent. @moniya12 was working on some database changes to potentially help with this.

take a state export = running osmosisd export right ?

@catShaark correct! That takes ~5-10 mins max

I import a 1.8G genesis file ( exported from an osmosis mainet node ) and it takes only 200-300 sec. Do I do something wrong ?

In our scenario, we are modifying the genesis the mint new coins to a wallet, change a validator's consensus key to the one that we have locally, and give it 99 percent of the voting power. You can read a little more about the process here https://github.com/osmosis-labs/osmosis/tree/main/tests/localosmosis#localosmosis-with-mainnet-state and see the python script being used to modify the genesis here https://github.com/osmosis-labs/osmosis/blob/main/cmd/osmosisd/cmd/testnetify/testnetify.py
Tomorrow I can provide you with a modified one with the required consensus key if desired

thanks, I'd love to work on this to improve our test runtime overall
also, can you tell me what you think happened on step 3, why is it taking so long

Import genesis time is well understood

, this is importing a 1.8G genesis file, I run in this test file.

@catShaark thanks for the pprof. From what I remember, we had a specific case where our pprof was taking more than an hour, halting for over 40 minutes due to processing data.
@czarcas7ic do you happen to remember how to reproduce this?

ohh thank you guys, this is something I really want to put my hand on.

Yeah honestly the more eyes on this the better. While it started off being just an annoying side problem, its gotten in the way a lot more than one would expect

@czarcas7ic, this is still relevant right ? now that the incident is over, should we continue working on this ?

Yes very relevant, will get back to it tomorrow, thanks for the ping

Closing this for now. We've temporarily mitigated the problem by pruning IBC state.
Further improvements should come from IAVL improvements and better atomicity structure in the SDK's usage of the database


UXPLATFORM-9209: Add ability to dive into cells with focusable elements.

Summary

What was changed:
Added the ability to dive into cells that have focusable elements.  The change does not account for skipping focusable elements when tabbing through the Worklist Data Grid.
Why it was changed:
The Worklist Data Grid design requires that there is a single tab stop within the component.  In order to access focusable elements within a cell, the keyboard user is expected to denote that they want to interact with a specific cell via the Enter key.  This action will allow the keyboard user to traverse and interact with the elements within the designated cell.
Testing

This change was tested using:

[x] WDIO
[ ] Jest
[ ] Visual testing (please attach a screenshot or recording)
[ ] Other (please describe below)
[ ] No tests are needed

Reviews
In addition to engineering reviews, this PR needs:

[x] UX review
[x] Accessibility review
[ ] Functional review

Additional Details
N/A
This PR resolves:
UXPLATFORM-9209

Thank you for contributing to Terra.
@cerner/terra

@cm9361 - the Focusable Cell example looks good. The keyboard navigation works well. JAWS also works well with the component. There are some things that can improved with how JAWS handles the components within the cells. Here are some observations:
Cell 2,2 - when I clicked the Alert button, the modal opened and JAWS read the modal content 3 times.
Cell 2,4 - when I clicked the Visit link, the URL opened within the same browser tab and took out of the Terra site. It would be better if the link opened in a new browser tab.
Cell 3,4 - The text area has the expander icon in the lower right corner of the text box. It works with the mouse, but it is not keyboard accessible.

@cm9361 - the Focusable Cell example looks good. The keyboard navigation works well. JAWS also works well with the component. There are some things that can improved with how JAWS handles the components within the cells. Here are some observations: Cell 2,2 - when I clicked the Alert button, the modal opened and JAWS read the modal content 3 times. Cell 2,4 - when I clicked the Visit link, the URL opened within the same browser tab and took out of the Terra site. It would be better if the link opened in a new browser tab. Cell 3,4 - The text area has the expander icon in the lower right corner of the text box. It works with the mouse, but it is not keyboard accessible.

@mjpalazzo - Thank you so much for your review.  I just created some raw examples for testing.  The behavior of the components does not factor into the accessibility of this JIRA.  Their true purpose was to prove that they would be recognized as focusable elements.  I had to create the anchor tag a specific way to be considered a focusable element so that is why the behavior is as it is.  If you are agreeable to that, I think we can move forward.  I will check the modal issue, maybe we need to log something for that component.

This is working so well! Nice job!
Please include in the root PR comment that this PR doesn't attempt to remove the focusable elements from the page's tab order but that will come in a later PR.

I made this update to the PR template.

@cm9361 - I agree that the navigation of the cell dive is working well. Please proceed. Thanks for all your hard work on WDG!


Video colorization

Hi, great work!
Any chance to use your approach with videos? I've tried a little bit and now it's flickering a lot.

@chekalsky I think you would have problems with temporal consistency for videos, would probably need some sort of temporaral consistency loss.

https://github.com/phoenix104104/fast_blind_video_consistency A plug-and-play solution to video flickering.

Hi, great work!
Can you please update the code to work with CPU ?
Thanks
Regards

https://github.com/phoenix104104/fast_blind_video_consistency A plug-and-play solution to video flickering.
Hi,Do you have a model of this algorithm? Can't download there

https://github.com/phoenix104104/fast_blind_video_consistency A plug-and-play solution to video flickering.
Hi,Do you have a model of this algorithm? Can't download there

i think this is better https://github.com/ChenyangLEI/deep-video-prior


TS 3.9.x disallow spread overwrite?

TypeScript Version:  3.9.3

Search Terms: spread overwritten
Code
const a = { spm: '123', test: 1 };
const b = { spm: '456', ...a };

Expected behavior:
No Error
Actual behavior:

Playground Link:
https://www.typescriptlang.org/play/index.html#code/MYewdgzgLgBAhjAvDA3jCAHAtgLhgcgEYAmAZnwBp49CYBfAbgChRJYAjJVdbPfAFgCsANkowAdJISMgA
Related Issues:
#37559 maybe?

Why would you want to do this?  The error is correct and at runtime, the value spm: '456' is completely irrelevant and can never be observed

You need to do:
const a = { spm: '123', test: 1 };
const b = { ...a, spm: '456' };

If you want spm: '456' to override spm: '123'.

I provide a bad example. Let me explain it in detail.
consider that I wrote a function handleQuery
interface Query {
    spm: string;
    test: number;
}

type HandleQuery = (q: Query) => void;

I would like user provide all the keys explicitly. (in this case spm & test)
But in the real world some one would call it like handleQuery({ test: 1 }).(js / no type check)
so for these cases I would provide a default spm value
const handleQuery: HandleQuery = (q) => {
  const finalQuery = { spm: '123', ...q };
}

I understand this error. In most cases it should not occur like that.
A simple type cast would fix that error, so just be it
const finalQuery = { spm: '123', ...q };

Typing q as Partial<Query> is another way to express your intent here

Typing q as Partial<Query> is another way to express your intent here

Be aware that it's possible to explicitly set undefined to a property, which will override the default value as well. Then the property can be set to undefined, even if the type explicitly states that it should not be undefined.
Example:
interface Query { spm: string; test: number; }

const query: Partial<Query> = { spm: undefined };
const finalQuery: Query = { spm: '123', test: 123, ...query };

// Type: string // value: undefined
console.log(typeof finalQuery.spm);

Playground link

Why not just let me overwrite the property? Who cares if a property gets overwritten on a spread? I don't.
TypeScript is making things more difficult than they need to be.

Who cares if a property gets overwritten on a spread? I don't.

I do. 🤷
You can use // @ts-ignore to prevent the error. But removing this check and there's nothing I could add to re-gain the safety.

Overwriting a property is not an error though. It's overwriting a property, something that we all do every day.

Unreachable code is not an error either, yet I like that the compiler warns me when I have clearly nonsensical code. { value: 1, value: 3 } is in the same ball-park here.
The error occurs when, according to the type-system, a property will always be overwritten. This is not something that we do every day, that's a bug.
The actual issue is the lack of #13195, which would allow to express the intent properly in the type-system.

Unreachable code is also never useful, whereas overwriting a property during a spread is extremely common because it's so useful.
Your example is showing an object literal that has the same prop twice. Spreading an object that happens to have the same prop is completely different.
I want to set a default value on my object and overwrite it with the object I'm spreading. Partial<> doesn't work here because then TypeScript complains that it's missing properties. So, I have to change my entire type or resort to using Object.assign just because TypeScript can't express the proper intent? That's garbage.
There are many, many, many complaints about this new error - what concrete problem does it actually solve for you?

I'd have to agree.  This add a tremendous amount of "typing" overhead for certain use cases.


Add source map to build

See: https://vitejs.dev/config/#build-sourcemap

Also minify the iife build since its directly downloaded by the user. If the user needs to debug, he has the sourcemap


New Module: genome-properties

Part of the series of modules we need for V6 Assembly Analysis

Nice. Some minor typos only. But I was also wondering; would it make sens (i.e. produce results) if instead of 1000 lines test file you fed it a 10 line one? Less storage space, faster test.

Thank you! I'd rather keep it like this, the test output is more realistic and the test finishes almost instantly, also the input file is very small (223kb), but good to think about for the future


Actix 4 incoming

Currently in beta but works. Good news is that it is on Tokia 1.0. So, either we can have it with beta or wait for full version. I ll try to keep this branch up to date in any case.

I was unable to get subscription support to latest version. I might need some help to fix that. @engylemure  @mihai-dinculescu
Current subscription support was in draft / beta mode. If things are alright we can get it as fully supported with latest actix 4.0 version.

Closing this in favour of #912


Add systemd icons

These icons were created by taking the systemd logomark (available at https://brand.systemd.io) and doing the following:

resizing to match other catppuccin-rEFInd items
coloring to match other catppuccin-rEFInd items

Since the original logomark is licensed under CC-BY-SA 4.0, these modified versions are also licensed under CC-By-SA 4.0

Thanks, it looks great, but the colors don't seem to be a little bit off:
Latte: #4d4f69 instead of #4c4f69
Frappe: #cdd6f4 instead of #c6d0f5
Macchiato: #cdd6f4 instead of #cad3f5

Thanks, it looks great, but the colors don't seem to be a little bit off:
Latte: #4d4f69 instead of #4c4f69
Frappe: #cdd6f4 instead of #c6d0f5
Macchiato: #cdd6f4 instead of #cad3f5

Fixed! I tried to match colors using the eyedropper tool, but I guess it didn't work. These new versions of the image are colored directly from the SVG and converted to PNGs with ImageMagick, so they should be correct.

Weird, using a color picker I still get different colors as result

The R and G value seem to be 1 too low for every image

Huh, I'm not sure what to do about that — I already tried my best strategy for getting the colors exactly right. I'll leave this PR open in case anyone wants to fix the colors on these icons

Thank you anyways. I fixed it now.
fyi I used Inkscape to change the color and export it to a pdf


Fix Chinese punctuation in utils.py

There are some Chinese punctuation in utils.py. It looks strange, and Python2 users couldn't import it because of these non-ascii characters.

Thank you!


Feat: Weaviate delete with filter

Support for deleting objects based on either their ids or a filter condition in weaviate vectorstore.

One small nit

Added the static method. I named it deleteIndex for consistency.

Should we add deleteIndex as a generic function in vectorstore base because most vector databases has the feature of deleting an index?

@jacoblee93 Let me know if it looks good or it needs changes. Thanks

Looking at it now, if this renders the vector store unusable (which it does, right?) without another API call, it seems a bit out of scope for the class. It's a one-liner that someone can do with the client directly anyway.

Good point. It would render an existing index unusable but since it returns void so it can be used to delete a single index with the name. But for now I removed it.

Thank you!


Scala fast test support

Description of the feature request:
Java IJWB has fast test support for many years.
We want to extend it to also support scala.
What underlying problem are you trying to solve with this feature?
From discussions with googlers it was said to be very valuable for developer velocity.
We (at Wix.com) are trying to improve the edit-compile-test flow times and during our research we identified fast test as a potential strong win.
We have implemented this in our fork https://github.com/wix-playground/intellij/commit/16ddedba46ac2bfe4ff2a13616bd4501c3f011d7 and would like to discuss upstreaming it.
We have 2 alpha users using the feature for a few months and they have reported amazing results with it.
What operating system, Intellij IDE and programming languages are you using? Please provide specific versions.
macos, linux, IJ 2022.2, Scala
Have you found anything relevant by searching the web?
No response
Any other information, logs, or outputs that you want to share?
No response

@tpasternak Could you have a look at this?

This is an interesting proposal.
I'm not sure who will create and maintain such a separate plugin.
I don't know if we (@wix engineering) can maintain another plugin but I'll reach out to @liucijus when he gets back from Christmas vacation and see what he thinks.
Maybe someone else can maintain this 🤷
If such a plugin doesn't happen then practically for most scala users this option will still be blocked.
We'll update.
Btw,
WDYT about having in this repository maybe another ext/contrib plugin? That has more relaxed governance?
I'm trying to capitalize on the infra that this repo has with respect to building, deployment and more

I like the idea of contrib plugin(s) that use the same infrastructure, do you think this can be done with reasonable effort? I wouldn't like helpful features like this to be blocked too long on setting this up. Also making the features available to users will require more actions on their behalf. But for features that only affect a handful of users, this can be fine.

@jastice you're asking me re effort? I'm not sure. I'd imagine not huge but it's been a few years since I deep dived into the build mechanics of IJWB.
Re the other point- I agree that it will make the feature a lot less accessible when users need to install another plugin just for it. There is indeed a trade off between this value and high iterations. I think that if there is willingness on plugin maintainers side to allow some iterations then perhaps this should go through the main door.
Mainly because the main body of work to support bloop is in the scala plugin and not my contribution.

As far as I'm concerned, if we don't need to restructure the build in a way that messes up future merges, adding a contrib build would be great. Maybe a single contrib plugin for any future feature additions would be a good compromise. Up to you which way you find more promising!

Who is the build, release and deploy expert of the plugin right now? We need that person to try and roughly estimate the effort and hopefully give me a few pointers on what needs to be done.
We can also decide to not upstream it but I'd say that's a bit of a shame...
We're seeing a lot of user satisfaction from this feature :)

Right now I manage the build and release of the plugin. But the ask here is not clear to me, do we want to have another branch in this repo to which features like scala fast test will be added?

I think the ask is to have another folder on main branch that builds, tests and releases another plugin

Will it be similar to how the repo currently has a ijwb, clwb and aswb plugins? If so, I think we can extend the scripts we use for releases to release another plugin and we can also add more pipelines in CI for testing the new plugin. But it will be helpful if you can come up with a proposal for what this new plugin will serve, how it will play with the rest and also how relaxed we want the governance to be for this new part?

@ittaiz please update.

🤷🏽♂
I'm not sure.
I understand the concerns but at the same time I'm not sure that the complexity of another plugin (for maintenance and for discoverability) are worth it.
@liucijus wdyt?

Thanks for the clarification.
Two questions:

Would you be open to kickstarting an empty skeleton of such a plugin? Hooking its build to this repo CI and such. It might be small but it's been a few years since I opened a new IntelliJ plugin
Are other maintainers ok with having a gradle based plugin?

If the answer to both is yes I'll happily do the relevant changes (in the existing plugin as well as in the skeleton one you'll create)

I understand.
We (Wix) don’t have the bandwidth for 2.
If this is part of this repo and there is a skeleton then I think I can
make the contribution.
To be honest I don’t know if we’ll move since requiring everyone to install
another plugin has challenges.
We can also close this and decide that Scala fast test is a wix only
feature.
That would not be ideal IMHO but maybe they is the best option available
for now
On Tue, 2 May 2023 at 13:39 Tomasz Pasternak @.***>
wrote:

Oh, so this was not what I meant 😅 What I thought about was:

Merging a small PR to the Bazel plugin that would allow to plug
Bloop-related logic into it. (via extension point)
Move the bloop logic to a separate plugin under a separate
organisation and just release it fully independently

This is of course only the initial idea, simplest from our perspective.
This does not mean we cannot do anything else
—
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/intellij/issues/4130#issuecomment-1531248872,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAKQQF2KIGFEJP6PYMOOTRDXEDP5PANCNFSM6AAAAAASLAHAHE
.
You are receiving this because you were assigned.Message ID:
@.***>


Send message to multiple devices

I want to send notifications to specific users using tokens. firebase-admin can't send multiple tokens so I'm using:
for obj in queryset:
            token = obj.app_token
            message = messaging.Message(android=android_config, data=data, token=token)
            try:
                messaging.send(message=message, dry_run=False, app=default_app)
            except Exception as ex:
                print('Error %s' % ex)

But it seems that "FCM" is blocked due to the fact that I send a lot of requests per second. Because after that I can not get a notification for a while.
Can this be solved?

I believe there's some on going work to support multicast messaging in the new FCM REST API. This will be supported once that is implemented (although I don't know what the exact timeline is).

In the meantime, can you use topic messaging to implement your use case?

No. The list is generated dynamically and I will need to create multiple topics

Update: as mentioned in #315, we are working on a new batch-send API for Admin SDK.

Further updates will be posted on https://github.com/firebase/firebase-admin-node/issues/315

I believe this issue was fixed by #283 and can be closed.

Thanks for the reminder!


Any Change for EA Origin Chat?

Any Change for EA Origin Chat?

@TomberWolf Provid me more info and I can help you code it. Thanks


python311Packages.velbus-aio: 2023.11.0 -> 2023.12.0

Diff: https://github.com/Cereal2nd/velbus-aio/compare/refs/tags/2023.11.0...2023.12.0
Changelog: https://github.com/Cereal2nd/velbus-aio/releases/tag/2023.12.0
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 277692 run on x86_64-linux 1

5 packages built:

home-assistant-component-tests.velbus
python310Packages.velbus-aio
python310Packages.velbus-aio.dist
python311Packages.velbus-aio
python311Packages.velbus-aio.dist


In mobile it should say "tap" not "hover"

The map subtext should say "tap" or "click" the map rather than hover in mobile devices.

Will take this up.

Will using regex work?
https://stackoverflow.com/questions/11381673/detecting-a-mobile-browser

Fixed in patch #62


Paralell pushing to multiple registries

With dockerhubs move to strict rate-limiting when pulling images, deploying container-based systems can be a struggle.
Therefore I propose adapting the pipelines to push to multiple repositories, namely ghcr.io (github) and quay.io (redhat) to give the users a choice where to pull from. This would make setting up a cache registry obsolete in most cases.
Also I suggest to pick one of the two proposed registries as a default in the documentation.
/cc @mmattel

Thanks for informing us, I am going to make an internal fixme note in the container documentation with a link to this issue to track progress.

Ping to unstale :)

Closing, no real priority, no resources.


NuGet NPOI 2.6.0 PackagePropertiesPart.cs Set the Created Date.

I am getting an error on a .NET6 project when attempting to read from a file stream as an xssf workbook and the XML has a string date with a timezone stamp. ie: -5:00. System.FormatException String '2023-02-16T 8:47:18-0500' Only happening from the NUGet package and was not able to reproduce after cloning the project. Am I missing a step in order to read from an .xlsx file?

Can you upload the excel file?

2023-03-05XWalk.xlsx

How is this file generated? Third-party library? The datetime format looks not to be generated by Microsoft Office.


Separate backtrace and std features?

In Cargo.toml, the std feature depends on backtrace. Is there a technical reason this is required, or would it be possible to implement Fail for StdError without depending on backtrace?
We have a use case in dotenv for using failure crate with std error types as #[cause]s, but it would be ideal if we didn't pull in all of the backtrace crate.
Would it be possible to either:

add a new feature for impl<E: StdError + Send + Sync + 'static> Fail for E {} separate from backtrace and std
or have the std feature not depend on backtrace?

I'd be willing to submit a PR doing this if it's something you'd want to add, but there might be some other thing blocking it that I'm not aware of?

Could this be re-opened as of https://github.com/rust-lang-nursery/failure/commit/3d5b8f9172579fc14ac748190e0f4c8fc6103fe2?
This is still important to me and I'd really like it not to fall between the cracks. I love what failure does for error handling, but this is still blocking effective usage of failure without backtraces.

Could this be re-opened per https://github.com/rust-lang-nursery/failure/commit/3d5b8f9172579fc14ac748190e0f4c8fc6103fe2?
This is still important to me and I'd really like it not to fall between the cracks. I love what failure does for error handling, but this is still blocking effective usage of failure without backtraces.

Gentle ping to @mitsuhiko and @rust-lang-nursery/failure. I'm also again in need of this and it would be better if re-opened.

If we can find a way to make this backwards compatible I'm accepting PRs :)

@mitsuhiko I am not sure there is a way to do that, thus this was mostly so that it doesn't get forgotten for next breaking release.
Speaking of that, given that https://github.com/rust-lang/rust/pull/53533 landed in nightly already, perhaps 0.2 work (in no-backtrace mode, for now) can already start somewhere?

Agreeing with @lucab here that doing this right would be a breaking change.
We could implement this without a breaking change, but the only way I can think to do it is to add an extra std2 feature identical to std but without depending on backtrace. This wouldn't change the defaults, but if someone depended on failure = { version = "0.1", default-features = false, features = ["std2"] } they could get std features without backtrace (which is what this issue is about).
If 0.2 is anywhere close, adding std2 with plans to remove it at 0.2 probably wouldn't be worth it. Especially because it'd probably mean asking any number of libraries to switch to depending on std2 with no-default-features then them having to undo that change for when 0.2 is released.

I too would appreciate if backtrace was optional.


Novels Support

Describe your suggested feature
Plz add novels support
Other details
No response
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a short but informative title.
[X] I have updated the app to version 0.16.3.
[X] I will fill out all of the requested information in this form.

#314


Unknown complex types break Druid SQL

Affected Version
0.16.0
Description
After upgrading our clusters to 0.16.0, we noticed that the Druid SQL wasn't working because the metadata updates in DruidSchema were failing because of unknown aggregators in the segment metadata.
Turns out, in 0.16.0, I made a backward incompatible change in t-digest aggregator extension where I removed the aggregator types buildTDigestSketch and mergeTDigestSketch and replaced it with a single aggregator tDigestSketch.
We had some users ingest their data using the buildTDigestSketch aggregator. However, when the 0.16.0 cluster was deployed, it wasn't able to deserialize the aggregator metadata as they were removed. This caused problems for the Druid SQL metadata cache as SegmentMetadataRequests started failing which in turn completely broke the Druid SQL functionality on the cluster.
I think we should fix this and prevent future cases of someone making incompatible changes, especially in extensions-contrib modules, from breaking a core Druid functionality.

I suppose in this case it would be best to ignore the column rather than the entire table. What were you seeing? Where'd the failure occur?

I was seeing SegmentMetadata queries fail on the historical nodes with an NPE.
Caused by: java.lang.NullPointerException
@400000005de84b7507eae4c4 	at org.apache.druid.segment.column.SimpleColumnHolder.getColumn(SimpleColumnHolder.java:68) ~[druid-processing-0.16.0.jar:0.16.0]
@400000005de84b7507eb0404 	at org.apache.druid.segment.QueryableIndexColumnSelectorFactory.lambda$getCachedColumn$2(QueryableIndexColumnSelectorFactory.java:175) ~[druid-processing-0.16.0.jar:0.16.0]
@400000005de84b7507eb07ec 	at java.util.HashMap.computeIfAbsent(HashMap.java:1127) ~[?:1.8.0_231]
@400000005de84b7507eb07ec 	at org.apache.druid.segment.QueryableIndexColumnSelectorFactory.getCachedColumn(QueryableIndexColumnSelectorFactory.java:171) ~[druid-processing-0.16.0.jar:0.16.0]
@400000005de84b7507eb0bd4 	at org.apache.druid.segment.QueryableIndexColumnSelectorFactory.lambda$makeColumnValueSelector$1(QueryableIndexColumnSelectorFactory.java:146) ~[druid-processing-0.16.0.jar:0.16.0]
@400000005de84b7507eb0bd4 	at org.apache.druid.segment.QueryableIndexColumnSelectorFactory.makeColumnValueSelector(QueryableIndexColumnSelectorFactory.java:160) ~[druid-processing-0.16.0.jar:0.16.0]
@400000005de84b7507eb0fbc 	at org.apache.druid.query.aggregation.tdigestsketch.TDigestSketchAggregatorFactory.factorizeBuffered(TDigestSketchAggregatorFactory.java:117) ~[?:?]
@400000005de84b7507eb178c 	at org.apache.druid.query.aggregation.AggregatorAdapters.factorizeBuffered(AggregatorAdapters.java:103) ~[druid-processing-0.16.0.jar:0.16.0]
@400000005de84b7507eb178c 	at org.apache.druid.query.groupby.epinephelinae.GroupByQueryEngineV2$HashAggregateIterator.newGrouper(GroupByQueryEngineV2.java:550) ~[druid-processing-0.16.0.jar:0.16.0]
@400000005de84b7507eb1b74

These metadata queries were generated by DruidCache over here:
https://github.com/apache/druid/blob/master/sql/src/main/java/org/apache/druid/sql/calcite/schema/DruidSchema.java#L508
The reason behind NPE was  columnSupplier being null in SimpleColumnHolder. This was happening  because the metadata deserialization was failing here:
https://github.com/apache/druid/blob/master/processing/src/main/java/org/apache/druid/segment/IndexIO.java#L578
which was "silently" setting the columnSupplier to null.

The above NPE is unhandled in DruidSchema which causes the metadata refresh to fail and restart and fail at the same point over and over again. Maybe, DruidSchema should also be fixed to report only and swallow any such failures and proceed ahead with refreshing other datasources.

Hi @samarthjain, thank you for the report.

Turns out, in 0.16.0, I made a backward incompatible change in t-digest aggregator extension where I removed the aggregator types buildTDigestSketch and mergeTDigestSketch and replaced it with a single aggregator tDigestSketch.

I'm wondering this incompatibility issue exists with Apache Druid extension. AFAIT, the TDigest extension was first added in #7331 and those aggregators were merged in #8100, but both PRs are in 0.16.0.

I think you are right. I had backported #7331 to our internal fork based off of 0.12.2 and users started using it. Then when the cluster was updated to 0.16, things broke because 0.16 had #8100 which removed the aggregators that were added in #7331.
Having said that, I think this PR is still worth while. Aggregator modules, especially in the extensions-contrib module, can make some changes which may or may not be compatible with each other. I am actually not even sure if we have any such guidelines around backward and forward compatibility when it comes to extension modules. This change is making sure that an incompatibility like the one I inadvertently introduced doesn't end up breaking a major Druid functionality (Druid SQL in this case).

AFAIK, we have different compatibility policies in different areas. There are two areas I'm aware of.

For APIs, we have PublicApi and ExtensionPoint annotations. For those APIs annotated with one of them, there could be incompatible changes in those APIs between major releases. For non-annotated APIs, they can change in breaking ways in any releases.
For segment read, we should be able to read any format of segments no matter how old Druid created them (I didn't know this policy when I raised https://github.com/apache/druid/pull/9246). This applies to segment format, all shardSpecs, and all loadSpecs.

These policies apply to across all Druid modules including contrib extensions.
Based on these policies, if the aggregator type has changed inadvertently, that's a compatibility bug and should be fixed by supporting both old and new types.
However, I think your PR might be still useful since people sometimes might want to run Druid without some extensions (for example, they just want to stop using those extensions). Even in that case, as you mentioned, it would be nice that Druid could be still functioning. Your PR could address this use case. I'll review it soon.


Getting 400 While inserting data

Hi when I try to insert 10 records using postman runner. Out of 10 only 2 or 3 records are getting added in cosmos db rest are getting 400 bad request.
Here is my code.
Log.Logger = new LoggerConfiguration()
                    .WriteTo.Console()
                    .WriteTo.AzureCosmosDB(
                   endpointUri: new Uri(cosmosDbConfig.Account),
                    authorizationKey: cosmosDbConfig.Key,
                    partitionKey: "PartitionKey",
                    partitionKeyProvider: new PartitionKeyProvider(),
                    databaseName: cosmosDbConfig.DatabaseName,
                    collectionName: cosmosDbConfig.ContainerName,
                    batchSize: 100,
                    timeToLive: TimeSpan.FromDays(7)
                    ).CreateLogger();

I have simple post api.
[HttpPost]
public IActionResult Post([FromBody] LogInfoModel logInfo)
{
    try
    {
        if (logInfo!= null)
        {
            Log.ForContext("logInfo", logInfo, true).Information($"xyz");
            return Ok();
        }
    }
    catch (Exception ex)
    {
        return StatusCode(500, ex.Message);
    }
}

Is there any configuration or setting which i missed over here?
Can you also guide me how I can enable detail level logging for capturing more information about failure.

@mahdighorbanpour I am getting below error
Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: d84019bf-37a2-4bde-9c88-acdb95b0ee56; Reason: (Message: {"Errors":["Encountered exception while executing function. Exception = TypeError: Object doesn't support property or method 'warn'\r\nStack trace: TypeError: Object doesn't support property or method 'warn'\n   at Anonymous function (script.js:21:7)\n   at Anonymous function (script.js:687:29)"]}

@kgparmar739 are you using v 1.0 or 2.0?

@kgparmar739 I've created a PR that will bubble the insert exceptions you are encountering up to the Serilog SelfLog where you can get more information about what the issue is.  Once accepted and pushed you can diagnose the issue.  Please follow that PR for updates.

@kgparmar739 I've created a PR that will bubble the insert exceptions you are encountering up to the Serilog SelfLog where you can get more information about what the issue is.  Once accepted and pushed you can diagnose the issue.  Please follow that PR for updates.

Thanks @tghamm, i wil check with latest version.
I have a question like is bulk import will work if all records are having individual partition key value?
As in my case I have records which has different partition key value for each record..

@kgparmar739 that's probably your problem.  This sink assumes all logs in a single batch will have the same partition key value.  It's derived from the first event in the batch, so if others in the same batch don't have the same value, it might cause your error.

@kgparmar739 this can probably be worked around as well.  Can you provide more insight into your partition provider and value?

@tghamm my partition provider is custom one which will read the value from message context. and value will be new GUID.

@kgparmar739 ok thanks, give me a bit and I'll see what I can cook up to deal with it.  The easy fix is to group by the key that's created but in your case that means losing bulk insert as it's written so I want to see if there's any way to rewrite the bulk query to handle your scenario.

PartitionKey

@kgparmar739 I'm having trouble reproducing.  Even when using a guid, it seems to insert, it just uses the first guid it comes across in the batch for all in the batch, they don't actually fail to insert.  Would you have the ability to run my PR version and enable Serilog.Debugging.SelfLog.Enable(Console.WriteLine) to see what error message is being returned to help further diagnose?

@kgparmar739 is "PartitionKey" an object on your LogEvent? I notice you are using it as your partition key.

It is not object. Actually we don't want to use that partition key as our partition key resides inside the context.
so, it is something like this properties/track/partitionkey where track is our custom object which contains information.
Model look like this
public class Log
{
	public Int64 EventIdHash { get; set; }
	public DateTime TimeStamp { get; set; }
	public string Level { get; set; }
	public string Message { get; set; }
	public string MessageTemplate { get; set; }
	public Properties Properties { get; set; }
	public Guid Id { get; set; }
	public string PartitionKey { get; set; }
}
public class Properties
{
	public Track Track { get; set; }
}
public class Track
{
	public string PartitionKey { get; set; }
}

While inserting PartitionKey value will be guid or you can say always unique.
While inserting I am doing this
Log.ForContext("Track", trackObj, true).Information($"{trackObj.PartitionKey}")

Let me know if you need more information.

@kgparmar739 I'm having trouble reproducing.  Even when using a guid, it seems to insert, it just uses the first guid it comes across in the batch for all in the batch, they don't actually fail to insert.  Would you have the ability to run my PR version and enable Serilog.Debugging.SelfLog.Enable(Console.WriteLine) to see what error message is being returned to help further diagnose?

I will try and let you know by tomorrow eod

Thanks!