Improve api_responsiveness_prometheus

Improvements:

Added unit tests to cover all functionality
Refactor code, so it's IMO structured better.

/assign @mm4tt
One remaining thing is to move each measurement into it's own package. Currently, due to similarity of measurements, I've hit names collision few times while working on this PR.

ref https://github.com/kubernetes/perf-tests/issues/986

/test pull-perf-tests-verify-all

/approve
/lgtm
/hold
/assign @jkaniuk
Adding current oncall as FYI, please void the hold whenever you want

/retest

/unhold


Delete method signatures not in V4

See

https://github.com/senzing-garage/g2-sdk-go/issues/130

Pull requests:

https://github.com/senzing-garage/serve-grpc/pull/252


TS-Jest should ignore private methods from the coverage report

Issue

TS-Jest should exclude all private methods from the coverage report.

Expected behavior

When running jest --coverage the coverage report should not complain about private methods not being tested

Output from your debug log

// type-class.ts
export class MyClass {
  public method1() {
    return `I'm public`;
  }

  private method2() {
    return `I'm private`;
  }
}

// myclass_spec.ts
import { MyClass } from '../type-class';

describe('MyClass', () => {
  it('test public method', () => {
    const x = new MyClass();
    const expectation = x.method1();
    expect(expectation).toEqual(`I'm public`);
  })
});

$ jest --coverage

---------------|----------|----------|----------|----------|-------------------|
File           |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
---------------|----------|----------|----------|----------|-------------------|
All files      |    66.67 |      100 |       50 |    66.67 |                   |
 type-class.ts |    66.67 |      100 |       50 |    66.67 |                 7 |
---------------|----------|----------|----------|----------|-------------------|

I think.. it makes perfectly sense to track coverage on private methods?

this would be a good feature though. Any plan to implement this?

I can't think of a single good reason not to get coverage from private methods unless you're not actually testing them and trying to claim coverage numbers you don't actually have.
I can't think of any good reason not to collect coverage from private methods. Am I missing something?

Sorry to resurrect a long-forgotten discussion, but I was looking for a solution to the same problem and stumbled upon this thread.
Regarding testing private methods. There are several reasons why I'd not recommend testing private methods:

private methods shouldn't be accessed outside of the class (breaks encapsulation), so there should be no way to test them in the first place
private methods are not exposed to the user and therefore can be invoked from the public or protected methods only. This implies that when you test public methods, you also test private methods in an indirect way (assuming that public method would fail if one of the private methods fail)
private methods are great when it comes to refactoring. If you plan to change any implementation detail in your class (but keep public interface unchanged), your test shouldn't be changed. Instead, they should be used to debug the code you refactored.

Not sure if this question is relevant to anyone, but there were two questions on the same matter so I decided to take a minute and try to address these concerns. Cheers 😊


Possibility to add half a day

In Sweden we regularly have half days off, and members of a team can also have an hour or so off because of for instance other works.
It would be nice to be able to add half a days or hours to be off instead of only full days off.

Add my vote for a half day.  We use this for PTO.

We're closing old issues so we can get back to actively maintaining the extension. Feel free to reopen if this is still an issue for you.


Vault - Getting Started Demo/Test

This PR is for a demo / test deployment of Vault that can be used to confirm Vault is working in any of the namespaces attached to a project set. Further examples will eventually be placed her to go with the Service Definition and docs being added to the openshift-wiki
Explanations of the annotations used in the example will be fully documented in the other repository and a link will be added through another PR at a later time.

TIL console is a codeblock type. Thanks!!


feat(theme-classic): new navbar item linking to a sidebar

Motivation
This PR resolves #6117
Have you read the Contributing Guidelines on pull requests?
Yes.
Implementations
For the plugin-content-doc, I wrote 2 functions:

getFirstDocIdOfSidebar in plugin-content-doc/src/sidebars/utils.ts, which receives a Sidebar object and returns the docId of that object. If it's a category with autogenerated index, the slug of that index page will be returned instead.
toGlobalSidebar in plugin-content-doc/src/globalData.ts, which map the sidebar name and docId returned from getFirstDocIdOfSidebar function to a GlobalSidebar object, which is then attached to a GlobalVersion object in toGlobalVersion function

For theme-classic/src/theme/NavbarItem, I added a new item called DocSidebarNavbarItem that handle the new navbar item type. It does so by fetching the GlobalVersion object, and retrieve its GlobalSidebar data that already contain a GlobalSidebarLink pointing to its first document
Test Plan
To test my changes, I modified the sidebar and navbar objects in the docusaurus.config.js and sidebar.js files in /website/. These changes are not included in this PR, but I have created a separate test branch on my fork with these changes so that you can test it there.
The detailed changes:
//docusaurus.config.js
navbar: {
  ...
  items: [
    { // test sidebar with a category as first item
      type: 'docSidebar',
      position: 'left',
      id: 'docs',
      label: 'Docs',
    },
    { // test sidebar with a doc as first item
       type: 'docSidebar',
       position: 'left',
       id: 'api',
       label: 'API',
    },
  ...
  ]
}

//sidebar.js
const sidebars = {
  docs: [
    // 'introduction', commented out so that first item would be category with generated-index link
    {
      type: 'category',
      label: 'Getting Started',
      link: {
        type: 'generated-index',
      },
      collapsed: false,
      items: [
        ...
      ],
    },
  api: [ //no changes here, first item is a doc
    'cli',
    'docusaurus-core',
    {
      type: 'autogenerated',
      dirName: 'api',
    },
  ],
}

I also created a demo video with the exact changes above: https://youtu.be/BtJ3306urjI
I also tested category with a doc link, as well as without a category link object, and they also seem to be working.
Related PRs
Might require updating the documentation to include this new feature if this PR is merged.
Feel free to let me know if you want any changes to be made!

Our current implementation differs slightly from what @slorber envisioned in that issue. The global data is set as {[sidebarId: string]: {link: string}} because the label can be set on every navbar item and fall back to the sidebar ID, so a plain string would work.

@lmpham1 Thanks! Now it's just docs left. You can just go add docs in this PR instead of creating a new one.

@Josh-Cena hey, I just added the documentation for our new navbar item type in website/docs/api/themes/theme-configuration. Feel free to let me know if I need to change anything, or if I should add more examples for category cases.

LGTM, thanks !


Reconfigure OpenAM docker image

Users are user.a, user.b, and user.x with authorities edit-ade1, edit-ade2,
and superuser, and passwords gumby123A, gumby123B, and gumby123X,
respectively.
RSA key pair is the default OpenAM test keypair.

:+1:  :apple:


Make triage tabs do page reloads

Make the triage tabs do full page reloads instead of using JS to hide/show tabs.
Currently the consent, triage and vaccination sections load up all the tabs and use JS to switch between them. The current implementation has a couple downsides:

back-buttons are harder to implement
the page is a mess on mobile and non-JS
and to a lesser extent, there will be performance risks as we have sessions with more users

This PR is the first step to changing them all to page reloads. Current issues with this implementation:

nhsuk-frontend currently breaks because it depends on discovering the tab label by examining the link for the tab and using the fragment from it (see https://github.com/nhsuk/nhsuk-frontend/blob/main/packages/components/tabs/tabs.js#L306). This means that the role and other aria-related attributes are broken, although the behaviour seems to work
scroll position is lost since, predictably, the page reload loses the position. Maybe this could be fixed if we implemented content-replacement?

Work remaining after this PR:

[ ] convert consents and vaccinations sections
[ ] move the patient pages into the new URL paths, e.g. /sessions/1/triage/needed/patients/2 and change the back buttons to rely on the path instead of cookies
[ ] re-implement the tabs themselves, these are being redesigned and we won't be able to use the govuk tabs components (see above issues)

Agreed with the two comments. I was going to just get the three sections here converted, and then finesse the component some more after.


Updated the Document URL's

Jira Ticket: https://new-relic.atlassian.net/browse/NR-255133
Description: Some of the documentation links are redirected to a 404 error page, so they have been updated with new documentation links.
Summary
A concise description of the changes being introduced. Please review the pre-merge checklist section to validate this pull request is ready for review and merge. If it is not ready, please mark the pull request as a draft.
The owners of this repo are not experts in the subject matter of the quickstarts. We review for the quickstart to be functional and for security risks. If you are seeking feedback on the content of the quickstart, please seek out a subject matter expert. If you are not an internal NR contributor, we can do our best to connect you with a content reviewer.

Pre merge checklist

[ ] Did you check you NRQL syntax? - Does it work?
[ ] Did you include a Data source and Documentation reference?
[ ] Are all documentation links publicly accessible?
[ ] Did you check your descriptive content for voice and tone?
[ ] Did you check your descriptive content for spelling and grammar errors?
[ ] Did you review your content with a subject matter expert? (e.g. a Browser agent quickstart is reviewed with a member of the Browser Agent team)

Dashboards

[ ] Does the PR contain a screenshot for each of your dashboards?
[ ] Do your screenshots show data?
[ ] Has the sanitization script been run on each dashboard?

Alerts

[ ] Did you check that your alerts actually work?

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 2 committers have signed the CLA.:white_check_mark: RamanaReddy8801:x: stevulaYou have signed the CLA already but the status is still pending? Let us recheck it.


Fixing engine release note header navigation

Original version: https://docs.docker.com/engine/release-notes/
Signed-off-by: Adrian Plata<EMAIL_ADDRESS>

Deploy preview for docsdocker ready!
Built with commit 76a62869af083bbc6a09575e14dab2854fe29607
https://deploy-preview-9606--docsdocker.netlify.com


Allow setting a list of constant tags for Tracing

Title: Allow setting a list of constant tags for Tracing
Description:

Currently there is an option for adding tags based on http request headers (request_headers_for_tags). However, there is no option for passing a list of hardcoded tags. It would be great to have such a feature. For example variables like node_id, host_ip, or pod_name could be added to the traces as follows, enriching the traces at will.

"tracing": {
  "operation_name": "egress",
  "request_headers_for_tags": [
    "x-cdp-sdk",
    "x-cdp-app",
    "x-cdp-clienttag"
   ],
   "constant_tags": [
     "node_ip=x.x.x.x",
      "pod_name=pod-xyz"
    ]
 },

It should be an easy task for someone that knows the codebase. Thank you in advance.

@mattklein123 Does this mean that this feature is accepted, I think it's a proper beginner task to get involved in, I would like to start from it if no one else is working on it.

@yittg yes please go for it.

@yittg One additional requirement, if I may - it would be good to also enable the tag's value to be obtained from environment variables. So if you could support some form of syntax like "tagName1=tagValue1,tagName2=${envvar2},tagName3=${envvar3:defaultVal3}". So tagName2 would be set if the environment variable envvar2 has a value, and tagName3 would be set to the env variable envvar3 if it exists else use the value defaultVal3.
If my explanation is not clear, let me know.

@objectiser Got your point. I'm not quite sure why we need it, would you give some scenarios about it? 😄

For example, when deploying in Kubernetes, we may want to record the pod name - which is only available in an environment variable.

@objectiser Got it.
Would we introduce a struct like following to describe it instead of parsing variable.
CustomTag {
  string tag;
  Type type={Literal, Environment, Header,...};
  string value;
  string default_value;
}

@yittg Ok sounds good.
Does this mean you are looking to replace the request_headers_for_tags as well? If so,

Consolidating into the one configuration seems reasonable as long as enough time is given to deprecate the old mechanism.
Previously I remember someone requesting the ability to rename a tag based on a request header - which wasn't possible with the current approach - but if the value field is used to specify the header name, then this mechanism would also support that requirement.

@objectiser Thanks for your reply.
I think it's a good idea to unify the inner implementation. But I'm not quite sure whether to expose user with clear extendable structure or simplified expression.

@mattklein123 any thoughts on this?

@objectiser I gave a first try about this issue, though still some more work to do, :).

Guzman HDTV chip

![robot](https://user-images.githubusercontent.com/52799430/69012494-6db74000-096e-11ea-9d0b-83cdfe4fc028.png)mooch


proto: Rename packages from tendermint -> cometbft

The packages of all Protobuf definitions in the proto folder of on branch main must be renamed from tendermint to cometbft. We intend on making substantial changes to the architecture and interfaces in the coming year and, as such, the protos could diverge substantially from Tendermint Core over time (only across major releases, of course). We also have no intention of remaining compatible with Tendermint Core in future major releases (this is already actually the case with our v0.37 release). As such, it makes very little sense, from a product perspective, to continue producing interface/data structure changes for Tendermint Core.
Having our proto package as cometbft will make publishing these protos to registries, e.g. https://buf.build/cometbft/cometbft, easier for us and for integrators who consume them (especially once we version our protos, in #95).
e.g. https://github.com/cometbft/cometbft/blob/754f1ae9e29a84754aa42ac969f0cd369266f019/proto/tendermint/types/block.proto#L2
must be renamed to:
package cometbft.types;

This includes regenerating all of the Go code and resolving any import errors.
Branches v0.37.x and v0.34.x will stay as they are:

in order to minimize disruption in the ecosystem
all protos in those branches are reused verbatim from the Tendermint Core repo

Further, we cannot do this for the v0.34.x branch because we would be breaking the message URLs associated with these messages, meaning CometBFT would potentially be incompatible with Tendermint Core, meaning that networks would need to perform a coordinated upgrade to CometBFT. Ideally we want our first v0.34-series release to be a drop-in replacement, so that a coordinated upgrade isn't necessary.

There are gRPC services defined in these files:

File
Service name

abci/types.proto
ABCIApplication

rpc/grpc/types.proto
BroadcastAPI

Changing the package name will result in changing the gRPC endpoints for these services as used by the generated code.
Do we want to go forward with it as part of this renaming, or should I keep these packages in the tendermint namespace?

Changing the package name will result in changing the gRPC endpoints for these services as used by the generated code. Do we want to go forward with it as part of this renaming, or should I keep these packages in the tendermint namespace?

We have no intention of keeping the tendermint namespace for anything Protobuf- or gRPC-related from our v0.38 release onwards. We also intend on implementing a full gRPC interface to the node ASAP, as per #81 (@mzabaluev, if you want, this is something you could take on once #94 and #95 are done).

It would be good to talk to users and get a list of what this breaks. From my understanding this could break ibc tendermint clients and potentially apps in cosmos that use these types would need to migrate.
We can only do so much from our end in the sdk.

Pet peeve time: while we are at breaking things, could we rename BroadcastAPI?

It would be good to talk to users and get a list of what this breaks. From my understanding this could break ibc tendermint clients and potentially apps in cosmos that use these types would need to migrate.

Yeah we're definitely going to try to solicit as much feedback as possible. My recommendation to @mzabaluev has been to do this work in a feature branch (feature/proto-upgrade, which is currently based on the vote extensions feature branch) so we can choose when exactly to incorporate this.
My understanding at present is that this primarily breaks:

Anyone serializing tendermint package types into Protobuf Any fields, which require the full type URL as part of the serialization.
Anyone importing the generated Go code from github.com/cometbft/cometbft/proto/tendermint (where this would eventually change to github.com/cometbft/cometbft/api after #95).
Anyone relying on any kind of serialization (e.g. JSON) that embeds the full path to the Protobuf type.

What other kinds of breakage do folks expect?
Wire-level encoding/decoding should be 100% compatible, except if you're somehow embedded type names in your encoded data.

Pet peeve time: while we are at breaking things, could we rename BroadcastAPI?

It seems to me as though while working on #81 would be the best time to do this, would it not?

It would also be interesting to investigate whether it's possible to offer both packages' protos simultaneously.
For instance, could we internally switch to using cometbft package protos and their generated code, while still offering tendermint protos (no longer hooked into our code anywhere) for external consumers, with these tendermint types being wire-level compatible with their cometbft counterparts?

for sdk i thought of a way we can avoid the breakage but allow you to break, i still need to test.
For go the main thing we care about is the option_go_package call because that defines how we import the generated files, but the proto url is what matters for state compatibility, we already have the idea to generate our own protos (this idea was around before the rename). We would keep the same proto urls and rename the option_go_package names to cosmos-sdk imports. That way things stay the same for us but allow you to break for future users

Moved to Q3. It is part of v0.39.x release plan tracking issue

After examination with @mzabaluev , we consider all these activities complete. The final feature branch merge into main is tracked in the parent issue. Reason: the feature branch is also being used for versioning.


rbac: Fix name shadowing

Currenly, there is a variable name in CreateRoles() with the same name as createRole().
Rename createRole variable to createdRole.
Signed-off-by: Orel Misan<EMAIL_ADDRESS>

Change: Fixed commit message.


fixed few glTF Spec urls anchors

The in page links anchors to a mesh.primitive, camera.orthographic, and camera.perspective were wrong and didn't scroll down to the section
#reference-mesh.primitive -> #reference-primitive
#reference-camera.perspective -> #reference-perspective
#reference-camera.orthographic -> #reference-orthographic

Thanks for these fixes!
(There's still a pending task to update the auto-generated spec. I assume that this will similar updates. But it's good that the current state now works as intended)


Create a chat screen layout

Need to create a frontend layout for chatting.
Features to focus on while working on the frontend

Chat system should be a team-based chat feature with multiple sub-channels feature
Chat UX should be understandable such as (user should be able to differentiate between sender and receiver's responses)
Use modular structures and components to implement this UI

This feature has been implemented!


⚠ Bofrost.fr has degraded performance

In 580a7ef, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 2207 ms

Resolved: Bofrost.fr performance has improved in 83a8ee5 after 10 minutes.


Cloud Voicemail Requirements - On-Premises environment

The section Phone System with on-premises environments should include a step for the network requirements for edge server to include all six geographic sites (countries, e.g. Hong Kong [APAC]). Also, the network URLs and IP address ranges for SfBO and Teams.
This is because certain functionality may break with Cloud Voicemail when migrating from Unified Messaging without confirming or expanding the above.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 8228f9b2-a7de-abb4-6f41-70db0b3af155
Version Independent ID: 8fd9fafb-dd76-59a8-e609-56f06d539a31
Content: Set up Cloud Voicemail - Microsoft Teams
Content Source: Teams/set-up-phone-system-voicemail.md
Service: msteams
GitHub Login: @dstrome
Microsoft Alias: dstrome

This issue doesn't belong here.
Better fit for here or here


Add JavaScript-like record syntax.

How hard / easy would it be to have convenience syntax for this case:
  return { params := params
           motives := motives
           witness := witness
           indHyps := indHyps }

written as
  return { params
           motives
           witness
           indHyps }

or, for this case then:
  return { params, motives, witness, indHyps }

I find myself writing it often enough that I miss this JavaScript syntax.

The best syntax @leodemoura and I could come up that does not overlap with set notation was
{ := params, := motives, := witness, := indHyps }

Personally, I'd be happy with it, since it's quite compact. I don't care too much about the concrete syntax rather than just not liking to write xxx := xxx. That's all.

The best syntax @leodemoura and I could come up with that does not overlap with set notation was
{ := params, := motives, := witness, := indHyps }

Yeah, I recall that we were not very happy with this syntax.
BTW, the support for overloading notation in Lean 4 is more robust. in the following example, I have overloaded it and it works reasonably well.
import Lean

def Set (α : Type u) : Type u :=
  α → Prop

def Set.empty : Set α :=
  fun a => False

def Set.insert (s : Set α) (a : α) : Set α :=
  fun x => x = a ∨ s a

syntax (name := finSet) "{" term,* "}" : term

syntax (name := jsSet) "{" ident,* "}" : term

macro_rules (kind := finSet)
  | `({ $as,* }) => do
     as.getElems.foldlM (init := ← `(Set.empty)) fun s a => `(Set.insert $s $a)

#check { 1, 2, 3 }

macro_rules (kind := jsSet)
  | `({ $[$is],* }) => do
     let fields ← is.mapM fun i =>
       `(Lean.Parser.Term.structInstField| $i:ident := $i:ident)
     `({ $[$fields:structInstField]* })

structure S where
  x : Nat
  b : Bool

def f (x : Nat) (b : Bool) : S :=
  let r := { x, b } -- expected type is not known here
  r

#print f

#check { 1, 2, 3 }

#check { true }

def g (s : S) : S :=
  { s with b := false }

#check fun x b => g { x, b }
#check fun x b => g <| id { x, b }
#check fun x b =>
  let aux := id { x, b }
  g aux

The main drawback is confusing error messages.
However, mathlib users can avoid this issue by using a higher priority for the Set notation.


UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf6 in position 1: invalid start byte

Downloads from youtube-channel TeletubbiesaufDeutsch are not possible, either single files, nor playlists nor the whole videos-list.
I used the latest version of youtube-dl. Another channel I tried to download from worked as expected, so it seems there is a difference.
root@user-vm2:/media/user/yt/Teletubbies# youtube-dl -v https://www.youtube.com/c/TeletubbiesaufDeutsch/videos
[debug] System config: []
[debug] User config: ['-i', '-f', 'best', '--write-thumbnail', '--all-subs', '--download-archive', '_ids.txt', '--abort-on-unavailable-fragment', '--write-description', '--no-overwrites', '--retries', '24']
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.youtube.com/c/TeletubbiesaufDeutsch/videos']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.06.06
[debug] Python version 3.8.10 (CPython) - Linux-5.11.0-34-generic-x86_64-with-glibc2.29
[debug] exe versions: ffmpeg 4.2.4, ffprobe 4.2.4, rtmpdump 2.4
[debug] Proxy map: {}
[youtube:tab] TeletubbiesaufDeutsch: Downloading webpage
[download] Downloading playlist: Teletubbies auf Deutsch - WildBrain - Videos
[youtube:tab] Downloading page 1
[youtube:tab] Downloading page 2
[youtube:tab] Downloading page 3
[youtube:tab] Downloading page 4
[youtube:tab] Downloading page 5
[youtube:tab] Downloading page 6
[youtube:tab] Downloading page 7
[youtube:tab] Downloading page 8
[youtube:tab] Downloading page 9
[youtube:tab] Downloading page 10
[youtube:tab] Downloading page 11
[youtube:tab] Downloading page 12
[youtube:tab] Downloading page 13
[youtube:tab] Downloading page 14
[youtube:tab] Downloading page 15
[youtube:tab] Downloading page 16
[youtube:tab] Downloading page 17
[youtube:tab] Downloading page 18
[youtube:tab] Downloading page 19
[youtube:tab] Downloading page 20
[youtube:tab] Downloading page 21
[youtube:tab] Downloading page 22
[youtube:tab] Downloading page 23
[youtube:tab] Downloading page 24
[youtube:tab] Downloading page 25
[youtube:tab] Downloading page 26
[youtube:tab] Downloading page 27
[youtube:tab] Downloading page 28
[youtube:tab] playlist Teletubbies auf Deutsch - WildBrain - Videos: Downloading 863 videos
[download] Downloading video 1 of 863
ERROR: 'utf-8' codec can't decode byte 0xf6 in position 1: invalid start byte
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/youtube_dl/YoutubeDL.py", line 815, in wrapper
return func(self, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/youtube_dl/YoutubeDL.py", line 847, in __extract_info
return self.process_ie_result(ie_result, download, extra_info)
File "/usr/local/lib/python3.8/dist-packages/youtube_dl/YoutubeDL.py", line 933, in process_ie_result
return self.__process_playlist(ie_result, download)
File "/usr/local/lib/python3.8/dist-packages/youtube_dl/YoutubeDL.py", line 1062, in __process_playlist
reason = self._match_entry(entry, incomplete=True)
File "/usr/local/lib/python3.8/dist-packages/youtube_dl/YoutubeDL.py", line 756, in _match_entry
if self.in_download_archive(info_dict):
File "/usr/local/lib/python3.8/dist-packages/youtube_dl/YoutubeDL.py", line 2157, in in_download_archive
for line in archive_file:
File "/usr/lib/python3.8/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf6 in position 1: invalid start byte

The download archive is meant to be UTF-8 but contains a 0xf6 byte (presumably Latin-1 ö). In UTF-8, that should be 0xc3 0xb6. This is quite irritating when all the character set encodings on your system are UTF-8.
Either some yt-dl code wrote data to the file without making it UTF-8 first, or some other tool did so.
You could try converting the archive file to UTF-8 using a suitable text or (advanced) hex editor or the iconv utility.
Perhaps lines that can't be read should be skipped with a warning.


Add missing htmleditor.css import

Checklist

[x] the contributor license agreement is signed
[x] commit message follows commit guidelines
[x] screenshots are included showing significant UI changes

Description of change
This CSS file was missing from the EBP work, causing a few things about the legacy TinyMCE editor to work strangely - the height of the editor would be too short, and also the "toggle full screen" button to be
inaccessible when in fullscreen (old version in wizards, not the version within the login notice editor),
effectively trapping users in full screen HTML editor until they refresh.
Before fix:
https://user-images.githubusercontent.com/24543345/131623326-e5622d81-8ffc-4263-bf2f-79a4152b9c9d.mp4
After fix:
https://user-images.githubusercontent.com/24543345/131623300-b2d7d506-7d57-43bc-9ede-3f6324e0ad6e.mp4
Change to address #3018

You'll need to merge in develop with NVM fix from #3369


logging: support logging with SourceLocation

Fixes #963

Consider that using Entry.Operation field already requires the caller to import logpb package, adding SourceLocation field with logpb.LogEntrySourceLocation doesn't make this client more difficult to use.

Thanks, we use gerrit so I put your changes into CL https://code-review.googlesource.com/c/gocloud/+/30091.

Cool. Will remember to send CL on Gerrit next time.


Add Support for Phi-3-small

Hi all, hopefully I'm doing this right, new to contributing to projects.
Noticed this project did not work with Phi-3-small-8k-instruct, which uses tiktoken as the tokenizer unlike phi-3-mini. Likewise, the triton_flash_blocksparse_attn only supports a q_len of 1 which would break even if the tokenizer was fixed.
This PR provides two changes which hopefully do not cause much issues elsewhere: first is adding support for the tiktoken tokenizer, second is adding a fallback token-by-token section to the get_logits which will process the tokens 1 token at a time if doing otherwise would cause some sort of error. Doing so will allow phi-3-small to work (seemingly) without breaking support for other models.

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
Attention: Patch coverage is 10.00000% with 18 lines in your changes missing coverage. Please review.

Project coverage is 57.47%. Comparing base (6e4ee06) to head (36bb6d1).

Files
Patch %
Lines

guidance/models/transformers/_transformers.py
10.00%
18 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #935      +/-   ##
==========================================
+ Coverage   55.54%   57.47%   +1.93%
==========================================
  Files          63       63
  Lines        4733     4748      +15
==========================================
+ Hits         2629     2729     +100
+ Misses       2104     2019      -85

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@ambisinister is Phi3-Small going to require a GPU (seeing the errors about flash-attention)? If so, we can transfer to that side of testing (along with updating its instantiation arguments).

It does seem so; all of my local testing has been with GPU, not sure what additional lift would be necessary to make it work via cpu

@ambisinister I've just tweaked how Phi3 is hooked into the tests, so that it should now load onto the GPU, and run as part of the GPU tests

Remaining cases are byte-level tokens that are being encoded as strings (and colliding), I'll update shortly

@ambisinister I've made some progress, getting flash-attention properly installed.

The smoke test failure is a bug but I'll try to run the rest of the tests locally to see how much of the failures are just environment related

======================================================== short test summary info =========================================================
FAILED tests/model_integration/library/test_gen.py::test_metrics_smoke - assert (3 == 1 or 3 == 2)
 +  where 3 = GuidanceEngineMetrics(engine_input_tokens=3, engine_output_tokens=3).engine_output_tokens
 +    where GuidanceEngineMetrics(engine_input_tokens=3, engine_output_tokens=3) = <guidance.models.transformers._transformers.TransformersEngine object at 0x7ff3862370d0>.metrics
 +      where <guidance.models.transformers._transformers.TransformersEngine object at 0x7ff3862370d0> = <guidance.models.transformers._transformers.Transformers object at 0x7ff31e94b040>.engine
 +  and   3 = GuidanceEngineMetrics(engine_input_tokens=3, engine_output_tokens=3).engine_output_tokens
 +    where GuidanceEngineMetrics(engine_input_tokens=3, engine_output_tokens=3) = <guidance.models.transformers._transformers.TransformersEngine object at 0x7ff3862370d0>.metrics
 +      where <guidance.models.transformers._transformers.TransformersEngine object at 0x7ff3862370d0> = <guidance.models.transformers._transformers.Transformers object at 0x7ff31e94b040>.engine
==================================== 1 failed, 820 passed, 33 skipped, 1 warning in 140.89s (0:02:20) ====================================

Ok after some digging it seems that the smoke test failure is an unrelated bug:
import guidance
from guidance import gen, select
from guidance import models

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_str = "microsoft/phi-2"
model_obj = AutoModelForCausalLM.from_pretrained(
    model_str,
    device_map="cuda",
    torch_dtype="auto",
    load_in_8bit=True,
    load_in_4bit=False,
    trust_remote_code=True,
    use_flash_attention_2=True
)
tokenizer = AutoTokenizer.from_pretrained(model_str,
                                          trust_remote_code=True)
model = models.Transformers(model_obj, tokenizer, echo=False)

@guidance
def smoke_test(lm):
    lm.engine.reset_metrics()
    lm += "\t\t\t\t\t\t\t"
    lm += gen("first", max_tokens=1)
    print(lm.engine.metrics.engine_output_tokens)
    lm += "\t\t\t\t\t\t\t"
    lm += gen("second", max_tokens=1)
    print(lm.engine.metrics.engine_output_tokens)
    return lm
model += smoke_test()

Stdout:
6
11

the problem is that "\t\t\t\t\t\t" here (and "abcd" in the smoke test) are a single token. So, in get_logits, we have token_ids of just [eos_token], and forced_bytes of b'\t\t\t\t\t\t'. What happens here is that the model will make the next token '\t', and then it will call get_logits again with token_ids [eos_token, '\t'] and forced_bytes of b'\t\t\t\t\t', and it will repopulate the original sequence one at a time until it has b'' again, after which it does normally.
Not sure I am explaining this right but I think this would be a good test to add (and fix) independent of this PR -- intuitively it feels like all the logits for tokens which do not start with forced_bytes should be masked so that they cannot be selected, that way this will work properly again, but that would be a bigger change that touches everything else in the repo so maybe outside the scope of this PR? Really interesting in any case.

Do we know what gpu is being used for these tests?

I may be out of tricks, modeling_phi3_small.py has a check suggesting flash attention 1 is supported but it then goes on to constantly use functions which are only imported in flash attention 2, so the tests all fail. Not sure if there's an avenue to make this run on a t4 -- very annoying

Thanks. We're just trying to obtain a local machine so we can run the tests manually, and then we'll merge.

Really appreciate all your help on this! Excited to see this get merged.

I have finally gotten access to a machine with a suitable GPU, and was able to run the tests successfully (although I do have to enable 8-bit, or even an A6000 runs out of memory). The only failure for python -m pytest --selected_model transformers_phi3_small_8k_instruct tests/model_integration/ tests/model_specific/ was in test_metrics_smoke and given the various issues with token counting, I can live with that.
Thank you very much @ambisinister  for this contribution. I made a few linting changes; once the gates complete, I will merge.


Move selenium tests into it tests

Closing, this is not a thing. The Browser tests are actually completely separate, my habit of running the testAll command alias means I'm catching them, but that's just a habit I'll need to change.


Use a custom TrustManager when proxying requests

Use a custom SSL trust manager when proxying requests to the current
leader. Until now Marathon was using a trust manager that contained only
Marathon's SSL certificate, which would work for self-signed
certificates, but not certificates signed by a CA.
cc/ @lloesche @jgehrcke

Way to pull this off so quickly!   I have 2 comments beyond the semicolons :)

it would be great to have a test with this
I would add a reference in the comments to: http://stackoverflow.com/questions/24555890/using-a-custom-truststore-in-java-as-well-as-the-default-one
so that the next developer to as WTH will have a reference and context.

Nice work!

👍  Thanks @gkleiman!

:rocket:

Thanks @gkleiman 🚀
Always a pleasure working with the Marathon team!


Improve skip parameter query filter

Filter parameter can work only when  limit is also specified and so sdk should extend skip parameter with limit as well.

SDK should not force this kind of behaviours, it should be up to users to set all parameters themselves and sdk should not tamper with it.


Intermediate routing screen

Create an intermediate screen to reduce loading time for deep link

This is the pre-HS/ landing/ routing screen we are talking about @CypherPoet

@antuz123 I don’t think this is an issue anymore

@AliMeer - changed the title. This is still an issue. We need:

Ability to lock thee app when the app goes to the background for > x secs
Not to do all HS activity when you click on deep link, it takes a long time, if we host the deep link modal on this screen then there are no actions here
This screen can also take the user to the last screen they were on or last step they were doing e.g. creating an account. It will jump the flow past HS so it does not have to wait for all HS stuff before they can take a step.

I have moved it to 2.0.0, please think about the solution till then

-Verified this issue as discussed with dev.

@cakesoft-swati - loading time reduced for the link?


Are const enums not supported now ?

Are const enums now not supported ?
Sandbox: https://codesandbox.io/s/great-fermat-y1pup?file=/src/App.tsx

Yes, see https://create-react-app.dev/docs/adding-typescript/ / https://babeljs.io/docs/en/babel-plugin-transform-typescript#caveats.

Yes, see https://create-react-app.dev/docs/adding-typescript/ / https://babeljs.io/docs/en/babel-plugin-transform-typescript#caveats.

Looks like it supports inlining constant enums now.


feat(EMS-3182): No PDF - Export contract - Agent charges - Data model/schema

Introduction :pencil2:
This PR introduces a data model/schema for "Agent charges" in the "Export contract" section of a no PDF application.
Resolution :heavy_check_mark:

Update SQL dump.
Create new EXPORT_CONTRACT.AGENT_SERVICE_CHARGE.METHOD constants.
Update mocks/fixtures.
Create new API helper function, createAnExportContractAgentServiceCharge.
Update createAnExportContractAgent to call createAnExportContractAgentServiceCharge.
Update the keystone schema to include ExportContractAgentServiceCharge, with a relationship to ExportContractAgentService.
Update types.

Miscellaneous :heavy_plus_sign:

Destructure various APPLICATION constants in the keystone schema.
Fix some typos.

Is this part of another table? If not, it should be added to getPopulatedApplication

Yes, however getPopulatedApplication is for XLSX integration. There is currently no agent related data in the XLSX, will be its own ticket/PR


spring.cloud.consul.config.prefixes documentation update

The documentation for spring-cloud-consul has no mention of the deprecation of spring.cloud.consul.config.prefix in favor of spring.cloud.consul.config.prefixes. I had to do dig up the code to see if the new (plural) property is a YAML list or if it is a comma separated String. The documentation at the below URL should be updated with an explanation.
https://docs.spring.io/spring-cloud-consul/docs/current/reference/html/#customizing

up - I wasted few hours trying to figured out why my configuration has not been resolving because of this confusing doc... 👎

This typically wouldn't go in the documentation but in release notes. This was missed, our bad. Both properties were supported in 3.x and was removed in 4.0

My mistake, prefix support hasn't been removed and it should still work. @mario45211 prefix should still work.


Update doc for CA1801 to mention exclusion of serialization methods

Fix #4987

@Evangelink : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Hi @Evangelink - Could you respond to the questions when you have a chance? Thanks.

@Evangelink : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@Evangelink : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@Mikejo5000 : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#sign-off


Added Coin flip and Dice roll

This is a clear duplicate of #87. Every feature should have only 1 corresponding Pull Request. If you want to address the requested changes you can simply push to your branch, this Pull Request is basically mirroring that branch and every new commit to that branch will appear here, there is no reason to create new pull requests for new features, I'm closing this as a duplicate of #87, continue working on that Pull Request instead.


pcre does not build

==> Installing neko dependency: pcre
==> Downloading https://downloads.sourceforge.net/project/pcre/pcre/8.36/pcre-8.
Already downloaded: /home/andy/.cache/Homebrew/pcre-8.36.tar.bz2
==> ./configure --prefix=/home/andy/.linuxbrew/Cellar/pcre/8.36 --enable-utf8 --
checking whether pthreads work with -pthread... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
checking for PTHREAD_PRIO_INHERIT... no
** Cannot --enable-pcregrep-libbz2 because bzlib.h was not found

READ THIS: https://github.com/Homebrew/linuxbrew/blob/master/share/doc/homebrew/Troubleshooting.md#troubleshooting

It built after I removed "--enable-pcregrep-libbz2", (line 33 of pcre.rb).

It may require a bz library to be installed - everything's directly ported from mac where a lot of these libraries are pre-installed, so adding dependencies is needed for some formulae.
For this one in particular, it looks like it may be lbzip2, but I'll play around with it and see what I can do.

Try brew install homebrew/dupes/bzip2

Thanks for the bug report, Andy.


Applying labels and annotations to secrets

This example includes the following regarding labels and annotations:
  # labels and annotations are copied over to the
  # secret that will be created
  labels:
    acme.org/owned-by: "q-team"
  annotations:
    acme.org/sha: 1234

I did not find this to be true. To have the annotation/label applied to the secret, I had to add it to the secret template metadata directly. Not sure if this is an error in the example or if I was doing something wrong.

If you specify spec.target.template this will take precedence.  If you do NOT specify spec.target.template, ESO copies the labels from the external secret.
The intention of this snippet is to provide an overview of the CRD structure with all the inline comments etc.

Ok, well the exact workings of labels and annotations is not really described anywhere in the docs that I can find. Using the example as provided does not result in the labels and annotations being copied as suggested in the comment. Maybe because the template explicitly sets them to empty?
      metadata:
        annotations: {}
        labels: {}

Maybe because the template explicitly sets them to empty?

Exactly! If you specify spec.target.template then annotations and labels are implicitly an empty map.
If you leave out spec.target.template then it does copy the label/annotations from the ExternalSecret resource.
https://github.com/external-secrets/external-secrets/blob/main/docs/snippets/full-external-secret.yaml#L47-L53

I see. Thanks for the clarification. I'll go ahead and close this. Might want to clarify this behavior somewhere as it wasn't that clear to me. :)


[NPU]squared_l2_norm support bf16.

squared_l2_norm support bf16.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Why is eval logic so complicated?

Hi,
Two major questions cross my mind when looking over this repository
1.) Would you like access to an optimized compressed FAISS index? I ran a simple optimization this morning and shrunk my index from ~60gb to 5.5gb without any obvious degradation. This might help others in replicating the retrieval pipeline.
2.) Will I be able to reproduce your results in production if I elect to just retrieve on '[Retrieval]' tokens, rather than performing this calculation -
        if threshold is not None:
            score_dict = {}
            for tok, id in ret_tokens.items():
                if id not in pred_log_probs[0]:
                    score_dict[tok] = -100
                prob = pred_log_probs[0][id]
                score_dict[tok] = float(prob)
            do_retrieve = score_dict["[Retrieval]"] / (
                score_dict["[Retrieval]"] + score_dict["[No Retrieval]"]) > threshold

E.g. how different is this in practice retrieving only when the retrieval token is the sampled result?

Re 1:
That sounds awesome! I tried the HNSW index to reduce the inference time efficiency (at the additional storage costs) and it gives us some performance deterioration, possibly due to bad hyperparameters. Reducing the index size is another issue that is helpful. Do you mind sharing more details?
Re 2:
Sections 3.3 and Appendix Section A.3 explain why we did that, but it's essentially because we want to adjust retrieval frequency for different downstream tasks. As shown in Figure 4, retrieving more helps a lot on PopQA, which is dominated by many rare entities, while it doesn't affect much for PubHealth (claim verification task). Our Table 3, Hard constraints baselines show the performance when we only retrieves when the retrieval tokens are generated.

Re 1:
Awesome.
Another thing I did was move away from loading all passages into memory (this + the original FAISS index blew up my instance with ~80gb of mem). Further, I saw that inference times were on the order of 20s with the implementation in this repo, slimming the FAISS index to ~5gb reduced this to ~10ms. I haven't extensively checked the evaluation performance, but doing some spot checks it all looked reasonable.
The script for generating the compressed index is here. The db implementation is here.
I am very inspired by this work, and so I am building a simplified approach to using models which follow this format. I have built a simple framework that allows one to attach a local LLM to a remote vLLM provider + rag db server (shown at sciphi-infra).
The interface for creating a self-rag LLM is then reduced to the below, as implemented here.
    llm = SciPhiLLM(
        SciPhiConfig(
            server_base="http://localhost:8000/v1",
            rag_provider_base="http://localhost:8001/search",
            rag_provider_token="",
        )
    )

Lastly,
I am re-running your fine-tune on Mistral to see how impacts your findings. I have generally found Mistral to be much better than llama-7b, so I'm excited to see how that shakes out.

Great discussion about optimization. I’m hoping there is limited performance regressions with the optimized code and when trying it out on Mistral!
FYI, added issue on license with your inference code @emrgnt-cmplxty

Thanks, that's Apache 2.0 - I uploaded the license.

Hi @emrgnt-cmplxty thank you so much for all of the contributions!! I will try to reduce the inference time memory requirement for retrieval part following your suggestions & snippets!

Closing this issue now, but feel free to reopen it if you want! Thank you so much for such an amazing followup, @emrgnt-cmplxty !

Re 1:
Awesome.
Another thing I did was move away from loading all passages into memory (this + the original FAISS index blew up my instance with ~80gb of mem). Further, I saw that inference times were on the order of 20s with the implementation in this repo, optimizing the FAISS index reduced the size 90% and inference speed to ~10ms. I haven't extensively checked the evaluation performance, but doing some spot checks it all looked reasonable.
The script for generating the compressed index is here. The db implementation is here. I am planning on hosting the index + a self-rag model today that others can access once I have smoothed out a few rough edges around the infra.
I am very inspired by this work, and so I am also building a simplified approach to using models which follow this format. I have built a simple framework that allows one to attach a local LLM to a remote vLLM provider + rag db server (boiler plate implementation is in sciphi-infra). And I have worked out a simple way to do inference.
My goal is to reduce the interface for creating a self-rag LLM to that shown below:
    llm = SciPhiLLM(
        SciPhiConfig(
            server_base="http://localhost:8000/v1",
            rag_provider_base="http://localhost:8001/search",
            rag_provider_token="",
        )
    )

w/ the option to just use local vllm if one doesn't want to make an server on their instance. This is something which I have working in my local setup and just need to cleanup / push.
Lastly,
I am re-running your fine-tune on Mistral to see how impacts your findings. I have generally found Mistral to be much better than llama-7b, so I'm excited to see how that shakes out.
Re 2:
Ah, thank you for the thoughtful explanation. I hope it is not a terrible approximation for now to simply stop on '' tokens and do the retrieval when the retrieval token precedes this. I will look into implementing the more complicated logic at a later date.

Hi, I guess the link is out-of-date, could you please update this link? I would like to look into details about the optimization in memory during the wiki-reader and FAISS embedding process.


Adds support for my location with always authorization status

Currently, apple_maps_flutter will only show the user's location (i.e., a blue dot on the map) when the authorization status is authorizedWhenInUse.
https://github.com/LuisThein/apple_maps_flutter/blob/f787ab1bd3bc65cac023e4df5c14e4963fd37298/ios/Classes/MapView/FlutterMapView.swift#L198-L208
This PR enables the user's location when the authorization status transitions from authorizedWhenInUse to authorizedAlways. Closes LuisThein/platform_maps_flutter#28.

Hi, I totally missed that! Thank you for your contribution and linking the underlying issue.

Nice. Thanks for the merge, @LuisThein! We were worried when you haven't been responsive.

No worries, I was just really busy, but I will have more time for the plugins now :)


Doesn't build on GHC 7.6

It'd be good to update the lower bound on "base" to exclude GHC 7.6 and to edit that constraint onto hackage.

The compile error on GHC 7.6 is the following:
[1 of 4] Compiling Control.Monad.Loop.Internal ( src/Control/Monad/Loop/Internal.hs, dist/dist-sandbox-26c26b96/build/Control/Monad/Loop/Internal.o )

src/Control/Monad/Loop/Internal.hs:187:39:
    parse error on input `where'
xcabal: Error: some packages failed to install:
loops-<IP_ADDRESS> failed during the building phase. The exception was:
ExitFailure 1

The code fragment in question is
type family UnTL (n :: TL.Nat) :: Nat where
    UnTL 1 = S Z
    UnTL n = S (UnTL ((TL.-) n 1))

I've tightened the lower bound on the three affected 0.2.* releases on Hackage


v1.0.0 modules from dev to prod

Finished stories related to v1.0.0

[x] Setup base React app
[x] Add a Loader component
[x] Add fake boot sequence when page loads
[x] Add pseudo bash history to terminal
[x] Add a Grid component to handle display on terminal
[x] Add content

No reviewers available ... cause I am just one man 😔


Failed to reproduce the results

This is an interesting work, by reading the paper. Therefore, I tried to run and test the code. Since README says "The default settings of the parameters in this repo can achieve 90% detection accuracy on TrojAI round3 training and testing datasets", I ran the code on TrojAI round 3 test set but only achieved around 71% accuracy, mainly because of vast false negatives (i.e. many trojan models were not detected).
Every stage contributed to the false negatives:

A significant number of trojan models were filtered out by pre-screening.
It didn't manage to optimize a small-enough trigger on many trojan models. The final trigger sizes even stayed at the inial value 1e+10, which means they were not optimized at all.
Similar to 2, during the symmetry check, the optimizer also gave a trigger size of 1e+10, and thus the ratio between the trigger sizes of both direction is 1, which will not pass the symmetry check.

From the above observations, it seems like there are problems with the pre-screening phase and the trigger optimization. Do you think they were caused by some sub-optimal hyper-parameters?

Hi, thanks for your interest of our work.  Our optimization component still follows the Neural Cleanse optimization which mainly handles patch(polygon) triggers. However, TrojAI R3 dataset has two types of triggers: polygon and filter(Instagram) triggers. 90% accuracy here means the detection accuracy on backdoor models with polygon triggers and benign models.  For your testing results, you can first check the 'model-id' of your failure cases and check their corresponding 'trigger_type' in the METADATA.csv file to see if most of them fall into the type of Instagram trigger.
If it is the case, you can refer line 2379-2387 in the following repo, which is a simple method to handle filter triggers.
If not, we can further discuss if it's caused by some hyper-parameters settings.
Thanks,
Guangyu

I used the METADATA file and simply ignored the ones with Instagram filters, then evaluated on the remaining models. For round 3 test set, the accuracy reached 91.7% which is quite impressive, with a true positive rate of 79.2% and true negative rate of 97.9%.  Do these stats look reasonable? The TN rate is very high, also since I removed the models with Instagram filters, the remaining dataset is not balanced anymore (roughly 2/3 of the models are benign). These lead to a good accuracy albeit a relatively humble TP rate.
I tried the same (i.e. removing models with Instagram filters) on round 4, but the TP rate is around 50%, which is a little bit low? The TN rate is still very good. Adjusting the trigger size bound and the symmetry ratio bound can help slightly improve TP rate to above 60%, but meanwhile significantly worsens the TN rate. Also, for many trojan models with patch triggers, the optimizer didn’t find a good trigger (the mask size stayed at 1e+10), does it mean I should increase the number of max round on an arm, or probably the total number of steps?
Also another question (sorry for bothering you so much): Since your code produces a definite result (either benign or trojan, not softmax logits), how was the cross entropy loss and AUC computed? Did you give a fixed probability (e.g. 0.9 when the results shows trojan, 0.1 when it is benign)?

I used the METADATA file and simply ignored the ones with Instagram filters, then evaluated on the remaining models. For round 3 test set, the accuracy reached 91.7% which is quite impressive, with a true positive rate of 79.2% and true negative rate of 97.9%. Do these stats look reasonable? The TN rate is very high, also since I removed the models with Instagram filters, the remaining dataset is not balanced anymore (roughly 2/3 of the models are benign). These lead to a good accuracy albeit a relatively humble TP rate.
I tried the same (i.e. removing models with Instagram filters) on round 4, but the TP rate is around 50%, which is a little bit low? The TN rate is still very good. Adjusting the trigger size bound and the symmetry ratio bound can help slightly improve TP rate to above 60%, but meanwhile significantly worsens the TN rate. Also, for many trojan models with patch triggers, the optimizer didn’t find a good trigger (the mask size stayed at 1e+10), does it mean I should increase the number of max round on an arm, or probably the total number of steps?
Also another question (sorry for bothering you so much): Since your code produces a definite result (either benign or trojan, not softmax logits), how was the cross entropy loss and AUC computed? Did you give a fixed probability (e.g. 0.9 when the results shows trojan, 0.1 when it is benign)?

I think the round3 results are reasonable. For round4, there are several differences. For examples, there is no global trigger anymore, all triggers are label-specific. Besides, there is a new type of trigger called spatial trigger. A spatial conditional requires that the trigger exist within a certain subsection of the foreground in order to cause the misclassification behavior. You can refer more in https://pages.nist.gov/trojai/docs/data.html#round-4. Therefore, I guess you might need to fine-tune some of the parameters a lit bit to get a better result.  For example, initial values for trigger mask and pattern, probability for epsilon greedy and max steps, etc.
Regarding to your last question, for round 1-4, we just simply give a fixed probability based on the inversion result. This value can be different for different type of triggers. We tune it on the training set.  For later rounds(r8,r9), we train a classifier(decision tree or random forest) based on some features of the inverted trigger(size, asr, etc). Based on our observation, training classifier usually gives you a slightly better accuracy compared on the naive rule-based method.

Closed since the issue is not active for months. Please reopen it if the issue remains in the code base.

Closed since the issue is not active for months. Please kindly reopen it if the issue remains in the code base.


Update lib requirements to make androdd.py work properly

I don't know if this is the right way to draft a PR - pardon me if I got it wrong!
Issue: androdd.py will fail for two different reasons
Environment: py 3.6, virtualenv, ubuntu 18.04

Decompilation ... End
Dump Landroidx/core/net/ConnectivityManagerCompat; getNetworkInfoFromBroadcast (Landroid/net/ConnectivityManager; Landroid/content/Intent;)Landroid/net/NetworkInfo; ... jpg ... Traceback (most recent call last):
  File "/home/user/Documents/testvirtualenv/test/bin/androdd.py", line 63, in <module>
    args.jar, args.decompiler, args.format)
  File "/home/user/Documents/testvirtualenv/test/lib/python3.6/site-packages/androguard/cli/main.py", line 259, in export_apps_to_format
    method2format(filename + "." + form, form, None, buff)
  File "/home/user/Documents/testvirtualenv/test/lib/python3.6/site-packages/androguard/core/bytecode.py", line 368, in method2format
    import pydot
ModuleNotFoundError: No module named 'pydot'

I was installing via pip, so I guess pip got the required lib wrong.
Anyway, after manually executing pip install pydot, I got another error:

Decompilation ... End
Dump Landroidx/core/net/ConnectivityManagerCompat; getNetworkInfoFromBroadcast (Landroid/net/ConnectivityManager; Landroid/content/Intent;)Landroid/net/NetworkInfo; ... jpg ... Traceback (most recent call last):
  File "/home/a29988122/Documents/testvirtualenv/test/lib/python3.6/site-packages/pydot.py", line 1915, in create
    working_dir=tmp_dir,
  File "/home/a29988122/Documents/testvirtualenv/test/lib/python3.6/site-packages/pydot.py", line 136, in call_graphviz
    **kwargs
  File "/usr/lib/python3.6/subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'dot': 'dot'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/Documents/testvirtualenv/test/bin/androdd.py", line 63, in <module>
    args.jar, args.decompiler, args.format)
  File "/home/user/Documents/testvirtualenv/test/lib/python3.6/site-packages/androguard/cli/main.py", line 259, in export_apps_to_format
    method2format(filename + "." + form, form, None, buff)
  File "/home/user/Documents/testvirtualenv/test/lib/python3.6/site-packages/androguard/core/bytecode.py", line 393, in method2format
    getattr(g, "write_" + _format.lower())(output)
  File "/home/user/Documents/testvirtualenv/test/lib/python3.6/site-packages/pydot.py", line 1734, in new_method
    encoding=encoding)
  File "/home/user/Documents/testvirtualenv/test/lib/python3.6/site-packages/pydot.py", line 1817, in write
    s = self.create(prog, format, encoding=encoding)
  File "/home/user/Documents/testvirtualenv/test/lib/python3.6/site-packages/pydot.py", line 1922, in create
    raise OSError(*args)
FileNotFoundError: [Errno 2] "dot" not found in path.

Apparently, this was caused by another issue in pydot.
It ca be seen here: https://stackoverflow.com/questions/35177262/importerror-no-module-named-pydot-unable-to-import-pydot
So I guess add graphviz as a requirement lib is the solution to this?
Correct me if I'm wrong, 1st time doing PR! Thank you.

Codecov Report

Merging #599 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #599   +/-   ##
=======================================
  Coverage   71.48%   71.48%
=======================================
  Files          49       49
  Lines       15228    15228
=======================================
  Hits        10886    10886
  Misses       4342     4342

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7c64adc...e9098f0. Read the comment docs.

Graphviz is an executable which needs to be installed separately, see https://github.com/pydot/pydot#dependencies

It looks like this feature is so heavily used, we should make pydot a direct dependency, not via extra.

I added the dependecy no into the core dependencies and described the process in the documentation! See #600

Oh, now I get it, Thank you for solving this really fast!
This is such a great project, as a newbie in the security field, I appreciate all your efforts in creating this masterpiece.
Thank you again, reox!


Browsing scoped packages is broken

Browsing scoped packages seems to be broken:
e.g. only one example:
https://unpkg.com/browse/<EMAIL_ADDRESS>

I think it's not broken you are just missing a trailing slash at the end.
Try this https://unpkg.com/browse/<EMAIL_ADDRESS>

I think it's not broken you are just missing a trailing slash at the end.
Try this https://unpkg.com/browse/<EMAIL_ADDRESS>


Update release notes for 3.0.6

Coverage decreased (-0.1%) to 90.713% when pulling cb7cb1df037b1f05a2dfd286926f5929f61e2baf on evildmp:update_release_notes into 2fa7c67818df7e5379c19d1186dc012a2f2bde80 on divio:develop.

Coverage decreased (-0.1%) to 90.713% when pulling cb7cb1df037b1f05a2dfd286926f5929f61e2baf on evildmp:update_release_notes into 2fa7c67818df7e5379c19d1186dc012a2f2bde80 on divio:develop.

Coverage decreased (-0.1%) to 90.713% when pulling cb7cb1df037b1f05a2dfd286926f5929f61e2baf on evildmp:update_release_notes into 2fa7c67818df7e5379c19d1186dc012a2f2bde80 on divio:develop.

Coverage decreased (-0.1%) to 90.713% when pulling cb7cb1df037b1f05a2dfd286926f5929f61e2baf on evildmp:update_release_notes into 2fa7c67818df7e5379c19d1186dc012a2f2bde80 on divio:develop.


GitHub organization access via GitHub app vs OAuth app

I developed a static web app for my organization. When working with my organization's developer support team, there was concern with integrating with SWA directly to our GitHub organization as the integration uses an OAuth app, rather than a GitHub app.  For multiple reasons, including more granular security, our team prefers using GitHub apps.  To complete the project, we pivoted to using Azure DevOps as a bridge to the GitHub repo.
We are now attempting to consolidate our developer toolchains to a single platform, moving to GitHub Actions for pipelines.  Unfortunately, this could negatively impact the current static web app, if an alternative approach to the OAuth integration is not available.
Has there been any consideration of implementing a GitHub App integration for SWA? Are there any alternatives that would enable granular access to a single organizational repository for this use case?

Hi @achalmers-lly Are you talking about the integration that happens during the SWA creation in Azure Portal?
We use it only temporarily to add the workflow yml file automatically to your repro while creating SWA app.
If you don't want this, you can choose other as the source in the deployment details section, and then add the workflow file manually to your repo.
You can follow these instructions partly to do this: https://learn.microsoft.com/en-us/azure/static-web-apps/bitbucket?tabs=vanilla-javascript and then use any other github based SWA app's workflow yml file as a reference to create your own workflow file. Hope this helps!


Update to Terraform Provider 1.51.0 and latest pulumi-terraform

This pull request updates the Terraform Provider to v1.51.0 and incorporates @pgavlin's changes to pulumi-terraform, including those which allow an explicit provider to be passed to get methods.

@pgavlin Yes, it pulls in all changes to date in pulumi-terraform.

Two failures:
[ pulumi/pulumi-aws/examples/api ] Updating (moolumi/p-it-travis-job-api-771d8004):
[ pulumi/pulumi-aws/examples/api ]
[ pulumi/pulumi-aws/examples/api ]     pulumi:pulumi:Stack apigateway-api-p-it-travis-job-api-771d8004 running
[ pulumi/pulumi-aws/examples/api ]     aws:apigateway:x:API myapi
[ pulumi/pulumi-aws/examples/api ]  +  aws:iam:Role myapi04e7e42c creating
[ pulumi/pulumi-aws/examples/api ]  +  aws:iam:Role myapi04e7e42c created
[ pulumi/pulumi-aws/examples/api ]  +  aws:iam:RolePolicyAttachment myapi04e7e42c-32be53a2 creating
[ pulumi/pulumi-aws/examples/api ]  +  aws:lambda:Function myapi04e7e42c creating
[ pulumi/pulumi-aws/examples/api ]  +  aws:iam:RolePolicyAttachment myapi04e7e42c-32be53a2 created
[ pulumi/pulumi-aws/examples/api ] @ Updating....
[ pulumi/pulumi-aws/examples/api ]  +  aws:lambda:Function myapi04e7e42c created
[ pulumi/pulumi-aws/examples/api ]  ~  aws:apigateway:RestApi myapi updating [diff: ~body]
[ pulumi/pulumi-aws/examples/api ]  ~  aws:apigateway:RestApi myapi updated [diff: ~body]
[ pulumi/pulumi-aws/examples/api ]  ++ aws:apigateway:Deployment myapi creating replacement [diff: ~variables]
[ pulumi/pulumi-aws/examples/api ]  ++ aws:apigateway:Deployment myapi created replacement [diff: ~variables]
[ pulumi/pulumi-aws/examples/api ]  +- aws:apigateway:Deployment myapi replacing [diff: ~variables]
[ pulumi/pulumi-aws/examples/api ]  +- aws:apigateway:Deployment myapi replaced [diff: ~variables]
[ pulumi/pulumi-aws/examples/api ]  ++ aws:lambda:Permission myapi-344da574 creating replacement [diff: ~function,statementId]
[ pulumi/pulumi-aws/examples/api ]  ++ aws:lambda:Permission myapi-344da574 created replacement [diff: ~function,statementId]
[ pulumi/pulumi-aws/examples/api ]  +- aws:lambda:Permission myapi-344da574 replacing [diff: ~function,statementId]
[ pulumi/pulumi-aws/examples/api ]  +- aws:lambda:Permission myapi-344da574 replaced [diff: ~function,statementId]
[ pulumi/pulumi-aws/examples/api ]  ~  aws:apigateway:Stage myapi updating [diff: ~deployment]
[ pulumi/pulumi-aws/examples/api ]  ~  aws:apigateway:Stage myapi updated [diff: ~deployment]
[ pulumi/pulumi-aws/examples/api ]  -  aws:lambda:Permission myapi-08d84682 deleting
[ pulumi/pulumi-aws/examples/api ]  -- aws:lambda:Permission myapi-344da574 deleting original [diff: ~function,statementId]
[ pulumi/pulumi-aws/examples/api ]  -  aws:lambda:Permission myapi-08d84682 deleted
[ pulumi/pulumi-aws/examples/api ]  -- aws:lambda:Permission myapi-344da574 deleted original [diff: ~function,statementId]
[ pulumi/pulumi-aws/examples/api ]  -- aws:apigateway:Deployment myapi deleting original [diff: ~variables]
[ pulumi/pulumi-aws/examples/api ]  -- aws:apigateway:Deployment myapi deleting original [diff: ~variables]; error: Plan apply failed: deleting urn:pulumi:p-it-travis-job-api-771d8004::apigateway-api::aws:apigateway:x:API$aws:apigateway/deployment:Deployment::myapi: error getting referenced stage: InvalidParameter: 1 validation error(s) found.
[ pulumi/pulumi-aws/examples/api ]  -- aws:apigateway:Deployment myapi **deleting failed** [diff: ~variables]; error: Plan apply failed: deleting urn:pulumi:p-it-travis-job-api-771d8004::apigateway-api::aws:apigateway:x:API$aws:apigateway/deployment:Deployment::myapi: error getting referenced stage: InvalidParameter: 1 validation error(s) found.
[ pulumi/pulumi-aws/examples/api ]
[ pulumi/pulumi-aws/examples/api ] Diagnostics:
[ pulumi/pulumi-aws/examples/api ]   aws:apigateway:Deployment (myapi):
[ pulumi/pulumi-aws/examples/api ]     error: Plan apply failed: deleting urn:pulumi:p-it-travis-job-api-771d8004::apigateway-api::aws:apigateway:x:API$aws:apigateway/deployment:Deployment::myapi: error getting referenced stage: InvalidParameter: 1 validation error(s) found.
[ pulumi/pulumi-aws/examples/api ]     - minimum field size of 1, GetStageInput.StageName.
[ pulumi/pulumi-aws/examples/api ]
[ pulumi/pulumi-aws/examples/api ] Outputs:
[ pulumi/pulumi-aws/examples/api ]     url: "https://jtcv4210dd.execute-api.us-west-2.amazonaws.com/stage/"
[ pulumi/pulumi-aws/examples/api ]
[ pulumi/pulumi-aws/examples/api ] Resources:
[ pulumi/pulumi-aws/examples/api ]     + 3 created
[ pulumi/pulumi-aws/examples/api ]     ~ 2 updated
[ pulumi/pulumi-aws/examples/api ]     - 1 deleted
[ pulumi/pulumi-aws/examples/api ]     +-2 replaced
[ pulumi/pulumi-aws/examples/api ]     4 changes. 2 unchanged
[ pulumi/pulumi-aws/examples/api ]
[ pulumi/pulumi-aws/examples/api ] Duration: 34s
[ pulumi/pulumi-aws/examples/api ]
[ pulumi/pulumi-aws/examples/api ] Permalink: https://app.pulumi.com/moolumi/p-it-travis-job-api-771d8004/updates/5
[ pulumi/pulumi-aws/examples/api ] error: update failed

Which is probably fixed by the upstream change https://github.com/terraform-providers/terraform-provider-aws/pull/6807
The other failure:
[ -aws/examples/multiple-regions ] Invoke '/home/travis/.pulumi/bin/pulumi refresh --non-interactive --skip-preview --expect-no-changes' failed: exit status 255
[ -aws/examples/multiple-regions ] Refreshing (moolumi/p-it-travis-job-multiple-r-f90333b3):
[ -aws/examples/multiple-regions ]
[ -aws/examples/multiple-regions ]  ~  pulumi:pulumi:Stack multiple-regions-p-it-travis-job-multiple-r-f90333b3 refreshing
[ -aws/examples/multiple-regions ]  ~  pulumi:providers:aws provider-us-west-2 refreshing
[ -aws/examples/multiple-regions ]  ~  pulumi:providers:aws provider-us-east-1 refreshing
[ -aws/examples/multiple-regions ]  ~  aws:ec2:SecurityGroup secgrp-us-west-2 refreshing
[ -aws/examples/multiple-regions ]  ~  aws:ec2:SecurityGroup secgrp-us-east-1 refreshing
[ -aws/examples/multiple-regions ]  ~  aws:ec2:Instance web-server-www-us-west-2 refreshing
[ -aws/examples/multiple-regions ]  ~  aws:ec2:Instance web-server-www-us-east-1 refreshing
[ -aws/examples/multiple-regions ]     pulumi:pulumi:Stack multiple-regions-p-it-travis-job-multiple-r-f90333b3
[ -aws/examples/multiple-regions ]     pulumi:providers:aws provider-us-west-2
[ -aws/examples/multiple-regions ]     pulumi:providers:aws provider-us-east-1
[ -aws/examples/multiple-regions ]     aws:ec2:SecurityGroup secgrp-us-east-1
[ -aws/examples/multiple-regions ]     aws:ec2:SecurityGroup secgrp-us-west-2
[ -aws/examples/multiple-regions ]  ~  aws:ec2:Instance web-server-www-us-east-1 updated [diff: ~instanceState]
[ -aws/examples/multiple-regions ]     aws:ec2:Instance web-server-www-us-west-2
[ -aws/examples/multiple-regions ]
[ -aws/examples/multiple-regions ] Outputs:
[ -aws/examples/multiple-regions ]     servers: {
[ -aws/examples/multiple-regions ]         us-east-1: {
[ -aws/examples/multiple-regions ]             publicDns: "ec2-18-234-213-82.compute-1.amazonaws.com"
[ -aws/examples/multiple-regions ]             publicIp : "<IP_ADDRESS>"
[ -aws/examples/multiple-regions ]         }
[ -aws/examples/multiple-regions ]         us-west-2: {
[ -aws/examples/multiple-regions ]             publicDns: "ec2-34-221-216-146.us-west-2.compute.amazonaws.com"
[ -aws/examples/multiple-regions ]             publicIp : "<IP_ADDRESS>"
[ -aws/examples/multiple-regions ]         }
[ -aws/examples/multiple-regions ]     }
[ -aws/examples/multiple-regions ]
[ -aws/examples/multiple-regions ] Resources:
[ -aws/examples/multiple-regions ]     ~ 1 updated
[ -aws/examples/multiple-regions ]     6 unchanged
[ -aws/examples/multiple-regions ]
[ -aws/examples/multiple-regions ] Duration: 5s
[ -aws/examples/multiple-regions ]
[ -aws/examples/multiple-regions ] Permalink: https://app.pulumi.com/moolumi/p-it-travis-job-multiple-r-f90333b3/updates/4
[ -aws/examples/multiple-regions ] error: error: no changes were expected but changes occurred

I am guessing this refresh issue is not something that we care about and should be ignoring it.

It looks like this is a bug in govendor-override, which is not pinning exact versions.

@jen20 Can we update the CHANGELOG for this?


Any possibility of seeing a java-6/7 port for v 2.4.0

Hi Brett,
$subject. I only see 2.4.0 for java 8.

Ramith

The 2.4.0 JAR supports both Java 7 and Java 8.  Java 6 has entered maintenance mode and will continue to receive bug fixes and releases from the 2.3.x branch.

Excellent ! thanks for the info.


Merges 674

Merges #674.

Current coverage is 86.08% (diff: 100%)

Merging #735 into master will not change coverage

@@             master       #735   diff @@
==========================================
  Files            29         29
  Lines          2609       2609
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
  Hits           2246       2246
  Misses          363        363
  Partials          0          0

Powered by Codecov. Last update 5f6fcae...4c9e59a


Problem with "ignite generate screen PizzaLocationList"

Hey,
I think there is an inconsistency when we generate screens by the command that ignite provides.
$ ignite generate screen PizzaLocationList

After run this command we will have 2 autogenerated files:
✔ App/Containers/PizzaLocationListScreen.js
✔ App/Containers/Styles/PizzaLocationListScreenStyle.js
Inside PizzaLocationListScreen.js file we see the following autogenerated line.
import styles from './Styles/PizzaLocationListStyle'

As we can see there is an inconsistency with file names. That autogenerated file should have been as follows:
import styles from './Styles/PizzaLocationListScreenStyle'

Note: I use ignite version 2.0.0-beta.9.
Thank you for your considerations.

@tokgozmusa Thanks for reporting this. Did you use the default Ignite boilerplate or ignite-ir-next?

@jamonholmgren I installed ignite with npm install -g ignite-cli command.
I created my first app using ignite new PizzaApp command.
And apparently this command uses ignite-ir-boilerplate-2016 boilerplate. I can open this bug on that boilerplate.
But I would like to ask how can I use ignite-ir-next boilerplate with ignite-cli?
Kind regars,
Musa

@tokgozmusa Thanks for the other issue. I have pull requests in both boilerplates now for this:
https://github.com/infinitered/ignite-ir-next/pull/37
https://github.com/infinitered/ignite-ir-boilerplate-2016/pull/38
To use ir-next, pass in -b ir-next:
ignite new PizzaApp -b ir-next

When Ignite CLI 2.0.0 is released, we'll default to our first post-2016 boilerplate release, code named Andross.

@jamonholmgren Thank you so much for your interest.


Smart search

Pull Request Description

Closes https://github.com/enso-org/cloud-v2/issues/782

owner: to do exact matches against owner username
label: (already existed)
name: to do exact matches against asset name (exact match; case insensitive; the glob character * matches 0 or more characters)
negated searches (-owner:, -label:, -: for negating keywords)

Related changes that were not explicitly requested:

add no:, -no:, has: and -has: to filter for assets that lack a specific field. Currently this only works for labels, because most (if not all) other fields cannot be empty.
label:a,b,c to OR results together. label:a label:b ANDs results as usual.
shift-click labels to add/remove it to the last search term as an OR
clicking labels now cycles them from: absent -> present (positive) -> present (negative)

Unrelated changes

Switches unit tests to use vitest

Important Notes
Some other suggestions have been added in the original issue, but currently intentionally left out to avoid prematurely over-engineering this feature.
Checklist
Please ensure that the following checklist has been satisfied before submitting the PR:

[x] The documentation has been updated, if necessary.
[x] Screenshots/screencasts have been attached, if there are any visual changes. For interactive or animated visual changes, a screencast is preferred.
[x] All code follows the
Scala,
Java,
and
Rust
style guides. In case you are using a language not listed above, follow the Rust style guide.
All code has been tested:

[x] Unit tests have been written where possible.
[x] If GUI codebase was changed, the GUI was tested when built using ./run ide build.

If there is a match in file name but the file is in folder we need to show the whole catalog tree

For matching empty directory the "This folder is empty" is not working

Is it hard for owner and label to support * as well?

Add suggestion for owner, label, name and autocomplete on tab. Maybe something like gmail has?

label:a,b,c to OR results together. label:a label:b ANDs results as usual. for me the intuitive flow is other wat around? but lets leave it for now as it is

good point, this will take a while though as i think it'll need quite a bit of minor refactoring to display things that weren't directly matched (the parent folders)
yeah, the "this folder is empty" is a special placeholder value that currently doesn't actually have a title, the component itself is responsible for rendering that text. i could make it work but idk whether that's intuitive, maybe we could do something else instead, but i don't think it's too important of a feature right now. like maybe no:children would work, but it doesn't really fit in with the rest of the no:* which refer to other tags.
no! it's just regex replacement so it should be very simple
this may be true, i don't really have an opinion - i just copied how GitHub does it. FWIW, if you think it should be changed, it really isn't a big deal to change the order - the only change required would be to change array.some(subArray => subrray.every()) into .every(.some()) instead.

but, it's pretty late in the day so i suppose i'll address these tomorrow.

this PR has scope creeped quite a bit...
... but do note that some things are still considered out of scope for this PR:

dedicated elements for individual search terms:

one issue is that we need to figure out how text input should interact with search terms
another issue is that the input will need to be completely refactored. rendering would be trivial using the backing data structure, however making sure all editing functionality (ctrl+a, ctrl+left, home, end) etc works would probably be quite difficult.

being able to click to toggle off a search suggestion:

implementation-wise, it's not trivial to implement this. i don't think it'd be particularly difficult, but it's still difficult enough to take a step back and consider whether this is a good idea first
actually, i think it's potentially counter-intuitive until there is some indication as to which items are currently selected.

... on second thought i just went ahead and implemented toggling off of suggestions

@jdunkerley ping for visibility. no rush of course, just figured i'd ping before i forget


Protobuf include emission support

Currently we emit .proto files independently, but because they typically include references to other proto files, we need to ensure that these transitive references can be maintained with the asset emission process.

The nature of protobuf so far has been that these files have always worked out due to the root emission. So parsing the imports has not yet been necessary. Closing until this is shown to actually be a needed feature for protobuf support in any real world usage.


Make stream buffer capacity configurable

Fixes: https://github.com/aspnet/AspNetCore/issues/4399
Adds StreamBufferCapacity hub option.

Are you doing

Add similar option to the client config (also global to the connection) which is used to configure the channel bounds for StreamAsChannelAsync

In this PR?

Are you doing <...> In this PR?

Wasn't planning on it. I'll file a follow up for it.

Are you doing

Add similar option to the client config (also global to the connection) which is used to configure the channel bounds for StreamAsChannelAsync

This would require protocol changes to support some sort of flow control like I mentioned in https://github.com/aspnet/AspNetCore/pull/9877#discussion_r280188387 to be truly useful. Otherwise, the client will always empty the channel immediately unless the entire TCP connection is facing backpressure.

😄


Payment visibilty on cart with virutal products

Payment option should not be showed when cart contains only virutal products,

Questions
Answers

Description?
Payment option should not be showed when cart contains only virutal products,

Type?
bug fix

BC breaks?
no

Deprecations?
no

Fixed ticket?

How to test?
Add virtual products to cart and go go checkout

Possible impacts?

It seems this PR is now stale.
I close this PR, following our review guidelines "Pull Requests may be closed after 30 days of inactivity".
@iqit-commerce if you are still interested in it, please ping us 😉  we can reopen ! 😄
Thank you for your contribution.


Add Funk Logdata shipper

Hi @fasibio, thanks for the suggestion.
This is an Awesome List, which means it's a curated list of things. I'd like to wait for your tool to have some more users, stars, and maturity before we consider it for an Awesome List.
From the Awesome List guidelines:

Only has awesome items. Awesome lists are curations of the best, not everything.

Also, I'm not sure why I'd need it. Docker/Swarm already ships with logging drivers to push logs to something like ELK. One suggestion is to detail the benefits in the README over traditional ELK solutions for Docker. I'd also recommend running the README text through https://grammarly.com to improve the English.
I'll close this PR for now. If, in the future, the tool gets wider use, then feel free to re-open it.


Investigate using OpenSSL (again)

By popular request, will reconsider removal of OpenSSL in CUPS 1.x for CUPS 3.0.

Mozilla’s NSS is another alternative, BTW.

@DemiMarie Perhaps, however if the point is to minimize the number of TLS libraries needed for a minimal install, using OpenSSL is a no-brainer...

Amazon’s s2n would actually be the one I prefer for server-side use, BTW.

@DemiMarie Hadn't heard of s2n, but looking at it I see a few issues:

They don't want people to look at this as a replacement for OpenSSL.
It seems to still depend on OpenSSL's libcrypto for the low-level crypto functions (it just implements the TLS layer).
It doesn't implement TLS 1.3.

@DemiMarie Hadn't heard of s2n, but looking at it I see a few issues:

They don't want people to look at this as a replacement for OpenSSL.
It seems to still depend on OpenSSL's libcrypto for the low-level crypto functions (it just implements the TLS layer).

The reason I suggested s2n is that it has less attack surface in the TLS layer.  OpenSSL’s crypto code has a good track record.

It doesn't implement TLS 1.3.

Git master does at least.  Not sure if that has made it into a release.

Are there any reasons why currently used gnutls is worse than OpenSSL?

As a distro package maintainer, I would ask to avoid using libraries with potentially instable API and ABI, it will be a pain to maintain (do not know how stable s2n is, but usually such libraries do not maintain backwards compatibility properly and may even forget to bump sonmae when breaking ABI).

As a distro package maintainer, I would ask to avoid using libraries with potentially unstable API and ABI, it will be a pain to maintain (do not know how stable s2n is, but usually such libraries do not maintain backwards compatibility properly and may even forget to bump sonmae when breaking ABI).

Why do you state this?

would be nice, I would very much like to include CUPS in the main repository of my distribution (since a variety of things needs it or at very least libcups) but I would like to avoid having to pull gnutls in there (it's in another section of the repository); I already have openssl in main, so having this would be great

This is now implemented (PR #362) for CUPS 2.4.2!

So is this specifically now OpenSSL1 and not OpenSSL3?

@nanonyme You can use OpenSSL 1.x or 3.x.


Add "Copy everything to clipboard" function in Order Details

Now only "Order #" and "Ship to" have a "Copy to clipboard" function.
Some vendors have another person sending items for them, therefore they must send an email each time a sale is made. I'd like to have a function "Copy everything to clipboard".
"Delivery Time" information is also missing on "Order Details" page, and this is something importantto give to the person which will send the items.
So, all the following information should be copied to clipboard:
"
Order #:
Accepted:
Payment Details:
Order Details:

Listing:
Ship to:
Coupons:
Quantity:
Moderator:
Total:
Shipping Option:
Shipping Service:
Delivery Time:
Memo:
Additional Contact Information:"

We will not be implementing this at this time. If more users request it, we may revisit.


Gateway should validate handle before adding the transaction to the queue and returning 201

Needs to go in gateway repo.


Catch BadZipFile errors

Summary
Closes #268
Testing
import requests

filename = "sample-docs/fake-text.txt"
resp = requests.post("http://localhost:8000/general/v0/general",
    files={"files": (filename, open(filename, "rb"), "application/vnd.openxmlformats-officedocument.wordprocessingml.document")},
    )
resp.text

PR also organizes imports into standard library, external packages, project imports.

Awesome! Can you add a test case like the PdfStream one? Just to send some sample doc with the wrong content type.

Ah, sure thing!


feat: build projects on the target machine

Build Projects on the Target Machine
Description
In this PR, a significant refactor is made to how the workspace creation process is run. The process is now ran on the target machine (local or remote) and directly on the host (before the process was ran inside a container). With this change, the creation process will see a significant performance improvement, especially on subsequent builds, because it will be cached on the hosts docker registry. Additionally, devcontainer Docker speficic capabilities (e.g. gpu support) are now supported alongside full support for devcontainers that leverage docker compose projects.
Breaking change
The provider interface was changed so they will need to be updated by the user

[x] This change requires a documentation update
[ ] I have made corresponding changes to the documentation

Related Issue(s)
Closes #697
Notes

The most significant portion of the changes are made in the docker pkg which means that all providers will benefit from this change.
The building process we have implemented beforehand is now ignore during the creation process. The builders will be using during the prebuilding process which will be tackled in a separate issue.

In draft while I fix a couple of issues (initialize command, socat container removal)


Can't join foreign table with a local table in a multi-node cluster

CrateDB version
5.7.2
CrateDB setup information
local / docker

[!WARNING]
Only multi-node clusters are affected

Problem description
Comment from @BaurzhanSakhariev:
"StreamerVisitor is missing visitForeignCollect implemenetation"
Steps to Reproduce
PostgreSQL (or remote CrateDB)
CREATE TABLE IF NOT EXISTS pg_t01 (
   txt TEXT,
   val INT
);
INSERT INTO pg_t01 VALUES ('Hello', 1);

CrateDB
CREATE FOREIGN TABLE IF NOT EXISTS pg_t01 (
   txt TEXT,
   val INT
)
SERVER neon OPTIONS (schema_name 'public');

CREATE TABLE IF NOT EXISTS local_t01 (
   txt TEXT,
   val INT
);

INSERT INTO local_t01 VALUES ('Hello', 1);

SELECT * FROM pg_t01 JOIN local_t01 ON pg_t01.val = local_t01.val;

Actual Result
SQLParseException[Couldn't create executionContexts from
NodeOperations: [0=>NodeOp{ phase{id=0/doc.pg_t01, nodes=[H3qWdFB7Qki-ZzlplC56sQ], dist=MODULO}, downstreamNodes=[H3qWdFB7Qki-ZzlplC56sQ], downstreamPhase=4, downstreamInputId=0}]
Leafs: [JoinPhase{executionNodes=[H3qWdFB7Qki-ZzlplC56sQ], executionPhaseId=4, name=hash-join, joinType=INNER, joinCondition=(INPUT(0) = INPUT(3)), outputTypes=[integer, text, text, integer], jobId=49c30e3f-f57a-48fa-0e43-9b8c6b957e51, executionNodes=[H3qWdFB7Qki-ZzlplC56sQ]}]
target-sources: [4=>[0]]
original-error: UnsupportedOperationException: Got unsupported ExecutionNode io.crate.execution.dsl.phases.ForeignCollectPhase]

Expected Result
+-------+-----+-------+-----+
| txt   | val | txt   | val |
+-------+-----+-------+-----+
| Hello |   1 | Hello |   1 |
+-------+-----+-------+-----+
SELECT 1 row in set (1.573 sec)

Thanks for the report, this will be fixed with https://github.com/crate/crate/pull/16190 / 5.7.3


Which library recognizes operators and numbers？

Which library recognizes operators and numbers？
Identify the results of the calculations

Please use the Tesseract user forum for questions.


[C++] GeometryObject parent is a frame of type BODY

Refers to https://github.com/humanoid-path-planner/hpp-pinocchio/issues/34

I do not know how to update the Python bindings myself. Can someone give me a hint ?
I can as well remove the reference to Model in GeometryModel (and pass it by arguments when needed).

I think we need to really think twice before adding a const ref on the model inside GeomModel. But I think it is mandatory for your case.
So to sum up the situation, the idea is we have two trees inside the mode:

a minimal one which contains only joints with at least one DoF and the tree can be retrieve through model.parents vector
a large one, which contains all the frames that a robot can contain: SENSORS, BODIES, JOINTS, FIXED JOINTS, OPERATIONAL_FRAMES, etc. The minimal one can be retrieve from the large one. This large tree is accessible trough model.framesvector.

The idea is then to allows frames to index for example body frames as parent and not only joints as it is currently done.
Can we discuss that tomorrow with @florent-lamiraux?

I can then explain you how to update the Python interface. It is not difficult, but maybe hard to explain through the Github interface.

I think we need to really think twice before adding a const ref on the model inside GeomModel. But I think it is mandatory for your case.

I can just add an argument to every function requiring the Model and not add a const ref to the Model, cannot I ?

I have updated the code to remove the const reference to Model in GeometryModel.
Following #299, I close this pull-request and push to branch hpp-devel.

Note that some bugs in the URDF parser have been fixed.


Unclear readme: how to start a service of a rust process?

I have a rust application that i normally run with 'cargo run' so how can i use servicer with it ?

this is weird hm
thread 'main' panicked at 'No interpeter found for extension rs. Please provide a custom interpeter and try again.', /home/andy/.cargo/registry/src/index.crates.io-6f17d22bba15001f/servicer-0.1.12/src/handlers/handle_create_service.rs:129:20
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
why cant it assume i am using cargo ?

You should first build the binary, then run servicer. Do you have a hard requirement to use cargo run?  ser edit might better help if that's the case.
cargo build
sudo ser create ./target/release/my-binary


How to use the esutil bulk indexer?

I have a worker which is a kafka consumer which listens to a message queue. When a message is received, it syncs the data in the message to elastic search. I am reading how to use esutil bulk indexer in this article: https://www.elastic.co/blog/the-go-client-for-elasticsearch-working-with-data
I'm wondering, should I call this code once or everytime I want to sync (kafka message is received)?
bi, err := esutil.NewBulkIndexer(esutil.BulkIndexerConfig{
  Index:         indexName,        // The default index name
  Client:        es,               // The Elasticsearch client
  NumWorkers:    numWorkers,       // The number of worker goroutines
  FlushBytes:    int(flushBytes),  // The flush threshold in bytes
  FlushInterval: 30 * time.Second, // The periodic flush interval
})

Because if I call it once only (when my worker is initialized), I will never close the bulk indexer:
if err := bi.Close(context.Background()); err != nil {
  log.Fatalf("Unexpected error: %s", err)
}

because listening to queue is non-stop. I'm wondering if its OK to not call this function?
Thanks

While it is OK to never call this function, I would still call it before quitting your program to ensure the BulkIndexer queue has been flushed.
If you don't, you may lose some documents that did not have enough time to be indexed in Elasticsearch.
The idea is to initialize the indexer once and then use the Add method to add documents in the queue to be indexed. Indexation is triggered either by time interval, size of the queue or by calling Close which flushes the entire queue before closing.

thank you, if you have time please take a look at my new issue @Anaethelion https://github.com/elastic/go-elasticsearch/issues/285


Fix checks for key existance in objects

Closes #181 and closes #182

[x] I have updated CHANGELOG.md
[x] I have added unit tests to demonstrate the contribution is correctly implemented
[x] I have run all the tests locally and no test failure was reported
[x] I did not introduce unrelated formatting/whitespace changes in the code base
[x] I did not do unrelated and/or undiscussed refactorings

It was a bug with how existance of object keys was checked.
Accessing key by doing myObject[key] also includes built-in keys like toString and valueOf and results problems afterwards. In such cases .hasOwnProperty method should be used.
UPD: Object.prototype.hasOwnProperty.call has to be used instead of .hasOwnProperty, as explained in https://stackoverflow.com/a/39283005. This also was reported by ESLint before fix.

@anton-trunov done!


State full XMLSERVICE calls

Reading up online how to make use of state full XMLSERVICE calls for faster response and simulate an open recurring job (as if I was using the 5250 terminal window), this is the route I would like to go. Especially since I would be able to gather data based on an "elapsed time" as well. For example, I could run "wrkdsksts" and get read/writes/reqs about disks since the time the job was first run, or maybe "dspsyssts" and get CPU over a period of time. Without this, I would only get data at the time the job was executed and it would be a snapshot at that time, not over a period of time.
My question is, using ipc and ctl parameters during the transport definition of idb2call, is this possible? Does the call to the stored procedure end the job or will it leave it open to re-use if I were to call it again?
I'm trying to follow the examples here (It's PHP, but at least gives me an idea of what's capable):
http://yips.idevcloud.com/wiki/index.php/XMLService/XMLSERVICEConnect
http://yips.idevcloud.com/wiki/index.php/XMLService/XMLSERVICEConfig
Thanks!

Yes, using the ipc and ctl keywords, it is possible to run stateful. This is the same regardless of the transport. Running stateful is quite an advanced technique and requires quite a deal of understanding about how XMLSERVICE works. When you use ctl='*here', XMLSERVICE runs your requested actions from the job in which you called XMLSERVICE (for idb2call, this is a QSQSRVR job). When you use ctl='*sbmjob', XMLSERVICE submits a new job and associates that with the ipc key specified. That job stays running until explicitly shut down (I think you can set a timeout somehow).
Running stateful and calling CL commands won't be of any use however, due to the way XMLSERVICE calls CL commands. It does so by calling through the PASE system command, which means a new job is spawned to call the CL command, regardless of statefulness.

Ohh i'm learning :-). Thanks for the prompt input, much appreciated!

Also, as long as you keep the database connection open, you can make repeated calls to XMLSERVICE through it using ctl='*here' and everything should be stateful. You only need stateful XMLSERVICE when you use the REST transport or when you're using it from a web application and need statefulness across multiple requests.