Open design questions: unsafe qualifier

Add unsafe as a type qualifier, just like const and volatile.
Calling unsafe functions on unsafe-qualified expressions is not unsafe. This will allow us to adapt safe containers with legacy types, eg:
std2::vector<unsafe std::string>
unsafe is part of the template argument type. It becomes part of the data members inside the vector impl. Calling unsafe dtors, copy ctors or move ctors is safe when done through an unsafe-qualified expression.
Unfortunately can't be treated exactly the same as const/volatile. Those types permit implicit addition, but not removal. How do we want to deal with standard conversions involving change of unsafe-qualifier? What about ranking for OR?

I'm starting work on this feature. But we need a new way to indicate unsafe contexts. unsafe T x = 1; will now declare an object with the unsafe type qualifier, rather than a normal declaration in an unsafe context.
Options:

unsafe { } where { } establishes lexical scope. (i.e. it's a compound-statement).
unsafe { } where { } does not establish lexical scope. I think this is better. There is precedence with extern "C" { }.
unsafe: stmt; where the context is unsafe until the end of the statement.
unsafe: stmts where the context is unsafe until the end of the lexical scope. This is more like public/private specifiers.

We could also have both:

unsafe: stmt for statement-level unsafe
unsafe { expr } for expression-level unsafe

Tentative rules for unsafe qualifier:

you can call unsafe functions from an unsafe object.
special member functions will return unsafe return types if the object is unsafe.
direct initializers have the unsafe qualifier on the destination.
subobject access transfers unsafe qualifier, as it does const/volatile.
pointer offset transfer unsafe as they keep the cv-quals of the pointer operand.
the return type of a function is unsafe if it has an unsafe argument with the same type or a derived type of the return type.
unsafe qualifiers do not matter when determining if types are ref-related.

using-unsafe-declaration

This is a using-declaration but it adds unsafe to the object, function or overload set that is pulled in scope.
int main() safe {
  using unsafe std::cout;

  // cout is found by unqualified name lookup. It references the using-unsafe-declaration.
  // Naming that object produces an unsafe lvalue.
  // You can use the object outside of an unsafe block.
  cout<< "Hello world\n";
}

A big problem I ran into with unsafe qualifier.
#feature on safety

template<typename T>
struct Vec {
  void push_back(T obj) safe;
  const T^ operator[](const self^, unsigned long i) noexcept safe;
        T^ operator[](      self^, unsigned long i) noexcept safe;
};

struct String {
  // Unsafe ctor.
  String(const char*);

  // Unsafe dtor.
  ~String();

  // Unsafe append.
  void append(const String&);
};

int main() safe {
  Vec<unsafe String> vec { };
  mut vec.push_back("A string");

  // unsafe type qualifier issue with append!
  // vec[0] yields lvalue unsafe String. Good.
  // vec[0]& yields prvalue unsafe String&. Good.
  // Since the object of append() is unsafe, it's safe to call append. Good.
  // Now perform copy initialization of a String into append's function
  // parameter. It calls the String(const char*) ctor. That's unsafe!

  // How does the initialization of append's argument get the unsafe
  // qualifier? Does it have to? In this case, not really, since the
  // user can put the statement in an unsafe-block. But if a similar thing
  // were done inside the Vec template, it would have to carry the unsafe
  // qualifier over.

  mut vec[0]&.append(" goes on...");
}

unsafe-qualified arguments in function calls should treat the immediate function call implicitly as unsafe.
For example, the following code:
#feature on safety

void not_safe(int* p){}
void not_safe(int* p, int a){}

struct rawr {
  int x_;

  rawr() safe
    : x_{1337}
  {
  }

  int not_safe_memfn(self const^){return self->x_ + 7331;}
};

int main() safe
{
  int x = 3;
  auto* unsafe p = addr x;

  p[0]; // already compiles

  // should compile, `not_safe()` here should be `unsafe { not_safe(...); }`
  not_safe(p);

  // should not compile, `rawr{}` is not `unsafe`-qualified
  not_safe(p, rawr{}.not_safe_memfn());
}
}

In the case of nested calls, we should defer to the unsafe-qualification of the return type and then continue up the callstack, repeating the same rules: only the immediate function call is implicitly unsafe.

// should not compile, rawr{} is not unsafe-qualified
not_safe(p, rawr{}.not_safe_memfn());

Why should this not compile? The p argument is unsafe qualified.

Who is to blame when undefined behavior is detected–the caller or the callee? Standard C++ does not address this. But Rust’s safety model does: whoever typed out the unsafe token is to blame. Safe C++ adopts the same principle. Code is divided into unsafe and safe contexts. Unsafe operations may only occur in unsafe contexts. Dropping from a safe context to an unsafe context requires use of the unsafe keyword. This leaves an artifact that makes for easy audits: reviewers search for the unsafe keyword and focus their attention there first. Developers checking code into Rust’s standard library are even required to write safety comments[safety-comments] before every unsafe block, indicating proper usage and explaining why it’s sound.

This paragraph is thrown out of the window by the unsafe type qualifier. Here is why :

Code being divided into unsafe and safe contexts isn't the whole picture anymore. Unsafe operations can happen inside a safe context without explicit usage of the unsafe keyword because of unsafe type qualifiers.
Unsafe operations may only occur in unsafe contexts is not true anymore. It should be Unsafe operations may only occur in unsafe contexts, or variables of an unsafe qualified type.
Dropping from a safe context to an unsafe context requires use of the unsafe keyword must become Dropping from a safe context to an unsafe context requires use of the unsafe keyword or the unsafe type qualifier
This leaves an artifact that makes for easy audits: reviewers search for the unsafe keyword and focus their attention there first. needs to be supplemented with a second step: Second, reviewers search for the unsafe type qualifier and review all the references to variables of unsafe-qualified types
Safety comments are also required where there are references to variables of unsafe-qualified types

Here’s the memory safety value proposition: language and library vendors make an extra effort to provide a robust environment so that users don’t have to read the docs.

I believe unsafe type qualifiers, while making incremental adoption potentially easier by making it easier to mix safe and unsafe code diminishes the value proposition with regards to auditability, and insulation of unsafe operations in explicit contexts.

The goal of this proposal is to advance a superset of C++ with a rigorously safe subset. Begin a new project, or take an existing one, and start writing safe code in C++. Code in the safe context exhibits the same strong safety guarantees as code written in Rust.

Emphasis is mine and shows another piece of the draft that needs to be updated if the unsafe type qualifier is kept the way it currently is.


Feat: Improving the responsiveness of the cards in mobile view and adding rounding edges so that the ui of the card looks enhanced.

Describe the feature
The cards ui in mobile form is very small so the text are  not visible correctly. So this can be changed and the card's breakpoints can be changes so that it looks good in mobile view.
Add ScreenShots
Screenshot 2024-05-24 173309
Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

Please add labels to this issue.


[Clip] Update media device

Fixes media device name and fw params for media's external reader

Closing as it was integrated into #1163.


Initialisation issues

The dps-meter is always going up when launching a sim
If things were perfectly randomized, we could see it sometimes go up, go down, be stable...
This is a hint that there are probably some initialization issues in the sim.

Is this just a "simple" DPS meter behavior thing? The detailed damage graph shows spikes and dips just fine.
https://i.imgur.com/ldhTyGk.png

I'm not talking about dps vs Time like in your graphs, but dps (average) vs Iterations. Hence why you can't see it in your runs.

More on this issue (my discord comment):
Ok so I was meddling with the rollover function, trying to figure out how to properly rollover the spell haste modifier. I added some increasing crit bonus when rollover was called to see whether at least the crit that was already taken into account. And after quite a few tests, trying to make sense of what was displayed, it dawned on me. Some spell variables (and at least the dot crit) is not properly re-initialized. Why do I think that? Well check these couple of screenshots:
On one hand, I have a near 100% crit rating on corruption, which is what I would expect with my hacked in crit rating.
However, on the second screen, you can see the results of 1 iteration. And I'm sure it's always the first iteration. You can see it doesn't crit nearly that frequently!
I pretty sure we have some variables in the dot program that need to be reinitialized properly.
https://cdn.discordapp.com/attachments/891807022973087825/1001856162808991864/unknown.png
https://cdn.discordapp.com/attachments/891807022973087825/1001856163186483330/unknown.png

This should now be fixed by #1335


Danger: bulk strategies can fail to index if called from within a transaction with use_after_commit_callbacks

class Chewy::Strategy::Demo < Chewy::Strategy::Atomic
  def update(type, objects, options = {})
    puts "called update for #{type} with #{objects.size} objects}"
    super
  end

  def leave
    puts "leaving #{self.class} strategy, indexing<EMAIL_ADDRESS>    super
  end
end

# Make sure only root strategy is something which doesn't index, to demonstrate:
Chewy.strategy.instance_variable_set(:@stack, [Chewy::Strategy::Base.new])

ActiveRecord::Base.transaction do
  Chewy.strategy(:demo) do
    ActiveRecord::Base.silence { Record.first.save! }
  end
end

This will output:
   (0.1ms)  BEGIN [sql_query]
DEBUG: Chewy strategies stack: [2] <- demo @ (pry):58
DEBUG: Chewy strategies stack: [2] -> demo @ (pry):58
leaving Chewy::Strategy::Demo strategy, indexing 0
   (1.7ms)  COMMIT [sql_query]
#<Chewy::UndefinedUpdateStrategy: Index update strategy is undefined in current context.
Please wrap your code with `Chewy.strategy(:strategy_name) block.`

This is highly problematic because for any given method which specifies a strategy, its caller may have started a transaction, which can break the indexing semantics or worse fail to index something altogether.

Hm, this is actually something worth a discussion. First of all, why can't you wrap a transaction with a strategy?

First of all, thank you for Chewy. It's for the most part been a delight.

First of all, why can't you wrap a transaction with a strategy?

Because different strategies have different semantics and the best place for deciding the semantics that should apply to a given component is probably in or close to that component, not its callers. Otherwise, what's the point of having nestable strategies at all?
In our case, we have a large background import which uses staged transactions. I just added Chewy and ES to our codebase and indexed ~70mil records quite happily. However, I was surprised to find that new data was not being imported to ES and nor were there any exceptions. Chewy ate the exceptions and dropped the index request, because the code looked like:
transaction do
  Chewy.strategy(:a_custom_async_strategy_which_I_hope_to_opensource) do
    # ...
  end
end

I then audited the code and realised a few places where this was problematic
Perhaps the current behaviour is correct or at least from an ergonomics/API perspective, you may wish to keep things as they are. Nonetheless, I found this behaviour surprising (and worse, kinda dangerous because of the exceptions being eaten) and thought others may also run into this so I thought I should open an issue for awareness.
Perhaps there needs to be some kind of documentation about this caveat and a note that errors in after_commit hooks are buried in Rails (depending on your version of Rails, and how it is configured).
In my particular case, the best thing to do is actually (surprisingly) to set Chewy.use_after_commit_callbacks = false, because my custom strategies[1] queue via the DB and will: a) be rolled back with any parent transaction anyway; b) only be committed if the changed data is committed[2]; and c) tolerate enqueued items that were not committed or were subsequently deleted (e.g. if the block leaves the strategy outside of any transactions).
If I were to explore this more deeply, I would probably separate "atomic" strategies from "incremental" strategies. Specifically:

If current strategy is incremental, update the strategy after_commit
If current strategy is atomic, store the current strategy as-of the after_save event, and update strategy that was active at the time the record was saved. Alternatively, for atomic strategies, update it during after_save but delay leave until the transaction is committing (tricky, but not impossible).

FWIW, I think part of the issue here is the name :atomic. I'd consider naming this :batch which is clearer in purpose. A :transactional or :atomic strategy might have some stronger guarantees about DB transaction interop.
Sorry, this turned into a bit of a ramble. Let me know your thoughts or if you have any questions.
[1]: One strategy enqueues items to database as soon as update is called, the other enqueues the batch to the DB when leave is called (but this will be before receiving the items, when use_after_commit_callbacks == true
[2]: Incidentally, this is why I prefer to have my strategy inside the transaction and wrote it like that only to discover this implication.

I'd consider naming this :batch which is clearer in purpose
Great suggestion.

Overall, I need to dig into this. Going to investigate it further and see what can I do with it. Right now I so busy with other stuff. Anyway, thanks for the report and your suggestions, I'll definitely consider it.

No worries. No rush on my end, as I have a custom strategy which actually works better with after_commit callbacks off. I just wanted others to know. Nonetheless, I'd be happy to help in any way if you need it.

Any update on this?


Game Crash

The only mods in my folder are:
flywheel 1.16-0.2.4
create-mc1.16.5_v0.3.2d
Im running them on the regular minecraft launcher with 1.16.5 forge
So when I launch the game, it instantly crashes and puts this msg:
The game crashed whilst rendering overlay
Error: net.minecraftforge.fml.config.ConfigFileTypeHandler$ConfigLoadingException: Failed loading config file flywheel-client.toml of type CLIENT for modid flywheel
Exit Code: -1
idk how to fix this

Have you tried deleting the config file from the config folder? by deleting the config file, it should recreate it automatically the next time you run the game and have default values.

Deleting the config file will fix this.


Publish To Environment: InternalServerError : Internal Server Error Object reference not set to an instance of an object

I have 3 environments setup.

QA works fine, and i can push the 'latest' build to PROD too. But whenever i try and publish to the MyFAT (FAT) environment i always get, Internal Server Error Object reference not set to an instance of an object.
Both MyFAT and QA were a copy of the production sandbox, so were identical.
I've run out of ideas to try and fix it, any ideas?
Screenshots below

I've been looking a bit more into this.
Finally, I have created a copy of my FAT environment, called it UAT, then created an Environment in Github called UAT (FAT) with an environment name of uat.
That then works (this was a direct copy of the sandbox uat), so no idea why it works now.
Any combination of the below names seems to fail where the environment in BC is called FAT
FAT (FAT)
MyFAT (FAT)
Completely puzzled by this.
Are there certain names we should avoid?

I will try to create some environments and test this - it should just work.

Could you share your repo settings file (from .github)?

From the image - the AL-Go environment name is MyFAT - but it tries to publish to a BC environment name called FAT.
If your environments are defined in GitHub environments, you might have an Environment secret under the environment which sets the BC environment name.
You could also have a setting called "MyFAT_EnvironmentName": "FAT" which would accomplish the same.

Removed my former comment.
The error is an internal server error - and seems to be happening on the server side
I would assume you get the same if you tried to publish the app from VS Code to FAT - maybe that would reveal more information?

Removed my former comment. The error is an internal server error - and seems to be happening on the server side I would assume you get the same if you tried to publish the app from VS Code to FAT - maybe that would reveal more information?

Ah ok, i removed the environment, but i will try it again and see what happens

Tested it again, had to create another environment, typical worked fine this time, so i think we can close this for now.


Attached PDF Renders very slowly

https://pdf.yt/d/StMRgGUi1Ms8-95v
This PDF renders very slowly compared to others.  It is not a large file (~600kb) and has 6 pages.
Interestingly, I notice that the problem does not occur with the viewer on the pdf.yt site.
Using Chrome 39.0.2171.95 and Windows 8.1 x64

This document contains sevaral 32Mpixel JBIG2 images. JBIG2 decoding has speed has improved a little in #5114 which was merged last August, so if you have an older version of pdf.js, try to use a newer one.

@sethnk can you add a version number to the report?

It was the latest version under 'example' on the pdf.js homepage.  Is there a simple way to to identify the version for this?

It was the latest version under 'example' on the pdf.js homepage. Is there a simple way to to identify the version for this?

Version for the PDF.js viewer is usually printed in the console. If you have a custom solution, please provide a public URL or exact steps in details, so contributors could reproduce the problem you have.

Yury - I am using the version on the pdf.js website (version 1.0.1086).  There are no customisations.

Oh, I see it's something specific to Chrome. Rendering times on Firefox:

On Chrome 39 it takes 3-4 seconds per page

5b83e0b9a3b2fbed8cc92397b8f874ab4375fc6d is the first bad commit
Introduced by #5114

@fkaelberer looks like Chrome does not like this optimization. (The question: why?)

At jbig2.js#L185 the array indices are sometimes out of range. Looks like Chrome and IE11 don't like it, so I guess we should check the array index boundaries ourselves.

Very interesting. So it is somewhat corrupted PDF.

This was intended, but I didn't know it would slow things down. I'll provide a fix soon.


User Credentials remembered from session to session

I think that I have found a bug. When I close the application and relaunch it the user creds are remembered from the previous session.
This is without selecting the "remember login" box.

Fixed with ea51f234


Issue with "MITRE ATT&CK - Courses of Action" in @site/docs/reference/articles/courses-of-action.md

Describe the problem
Wrong copy / paste in the Unit 42 Feed chapter

Page: MITRE ATT&CK - Courses of Action
Source: https://github.com/demisto/content-docs/blob/master/content-repo/extra-docs/articles/CoA-premium-pack-readme.md

Suggested fix
In Unit 42 Feed chapter, sentence shall be "The Courses of Action pack and playbooks work with both versions of the Unit 42 Feed pack." instead of "The Courses of Action pack and playbooks work with both versions of the MITRE ATT&CK pack."

this is not updated any more.
the page is now in https://xsoar.pan.dev/docs/reference/packs/courses-of-action
and in the repo it was moved to: https://github.com/demisto/content-repo/extra-docs/packs/CoA-premium-pack-readme.md.


"Tips" page shows only "Kijk verder:"

Hey Frank,
Like mentioned in #24 and #25, the NPO has deprecated the API and is not updating the tips anymore. So unfortunately there is not a whole lot to do about it...
Cheers


Can't remove items from media library

It looks like the draft that contains the interface for removing items blocks the removal action:
"The mutation(s) failed: Document "b430b968-9037-44ab-9878-28376fb86d08" cannot be deleted as there are references to it from "drafts.b32572a0-8c4f-43f5-acc1-edf0f6ac9b64""

Thanks for reporting @patrickdaley! This is probably a “not a bug, but a feature”. The backend doesn't let you remove documents with incoming (strong) references. This is to prevent you from unintentionally breaking something in another document that might rely on the asset existing.
Did you remove the video reference from the drafts.b32572a0-8c4f-43f5-acc1-edf0f6ac9b64 document?

Hi @kmelve! That reference is the active draft. Because you need to be in the interface for adding the asset to remove it, it's counted as referenced by the same draft you're trying to use to remove it. Hope that makes sense!

+1 that currently open draft is preventing the item from being removed

I can also confim that this makes it kind of impossible to remove a video from the studio interface.

I would also like to bump this. Is there a timeframe for when we can expect this to be fixed or does anyone have a workaround?

I would also like to bump this. Is there a timeframe for when we can expect this to be fixed or does anyone have a workaround?

I needed this for a client asap so I published a fork to npm with some updates. Rush-job so not 100% tested yet but seems to work so far. I added the new sanity ui components and fixed some visual bugs too (checkbox labels not visible etc).
If you're reading this and are in the same situation - please feel free to try it out by running sanity install @kimpaow/sanity-plugin-mux-video-input. Hopefully the official version gets an update soon though!

I needed this for a client asap so I published a fork to npm with some updates. Rush-job so not 100% tested yet but seems to work so far. I added the new sanity ui components and fixed some visual bugs too (checkbox labels not visible etc).
If you're reading this and are in the same situation - please feel free to try it out by running sanity install @kimpaow/sanity-plugin-mux-video-input. Hopefully the official version gets an update soon though!

@KimPaow Thanks for the fork! Working great so far! 😃

Struggling with this today, client can't remove a video from their page so I had to go into the frontend codebase and disable it manually, very frustrating.

@gristleism what version of the plugin are you on? I believe this was added back since v1.0.0

My package.json file says it should be loading ^1.0.1.

Yep, this was added since v1.0.0.
@gristleism could you also share what version of the studio you have?

@KimPaow happy to! Would you mind letting me know where I can find which version of the studio I'm using?

@gristleism you can run sanity versions in the terminal and share the output or check your @sanity deps from package.json

Results from initial run:
@sanity/cli             2.18.0 (latest: 2.20.0)
@sanity/base            2.18.0 (latest: 2.20.0)
@sanity/cli             2.18.0 (latest: 2.20.0)
@sanity/components      2.14.0 (up to date)
@sanity/core            2.18.0 (latest: 2.20.0)
@sanity/default-layout  2.18.0 (latest: 2.20.0)
@sanity/default-login   2.18.0 (latest: 2.19.0)
@sanity/desk-tool       2.18.0 (latest: 2.20.0)
@sanity/vision          2.17.2 (latest: 2.20.0)

I then ran sanity upgrade and afterwards here are the versions:
@sanity/cli             2.18.0 (latest: 2.20.0)
@sanity/base            2.20.0 (up to date)
@sanity/cli             2.20.0 (up to date)
@sanity/components      2.14.0 (up to date)
@sanity/core            2.20.0 (up to date)
@sanity/default-layout  2.20.0 (up to date)
@sanity/default-login   2.19.0 (up to date)
@sanity/desk-tool       2.20.0 (up to date)
@sanity/vision          2.20.0 (up to date)

Running again locally, I attempted to delete a video file again and received the following error:
the mutation(s) failed: Document "d1a72d5b-e897-4ad9-9cb3-7df4c8709afc" cannot be deleted as there are references to it from "frequentlyAskedQuestions"

@gristleism I just tried running this with the same dependencies you listed here and am unable to reproduce it. Maybe try deleting node-modules and lock files and then reinstall all packages.
If the asset was created using an older version of the plugin (pre v1) this could also be causing problems. To fix that you could delete the old documents and recreate it using v1+ of the plugin. Or you could write a patch to update the asset field on the mux.video doc to have the key weak: true.

Same "not a bug" issue, but the problem is that i don't anymore have access to the drafts.xxxxxxxxxx, as i made mutations that deleted the schema of  this object/document that refers to the other document i want to delete. So it doesn't exist anymore in my Sanity Studio.  Is there anyway to access it via url and its id drafts.xxxxx ?

@JulienDesmettre You can access the document directly by running sanity documents create --watch --replace --id 'drafts.xxxxx' from the terminal.

@JulienDesmettre You can access the document directly by running sanity documents create --watch --replace --id 'drafts.xxxxx' from the terminal.

Thanks for your answer geball. Yes i found it on Sanity CLI doc just few minutes after my question here.

Ran into this issue as well. The issue happens when the mux.video is used inside of a schema with the type array which includes Portable Text for site building which does not seem optimal.
I made a quick hacked solution by adding mux.video to a document solely to delete my videos.
Here is a small screen recording for reference:
https://user-images.githubusercontent.com/15996198/174471561-bdd56ce4-755c-4795-ba74-e4be88d1dc99.mp4

I have the same issue. As described by @jimmi-joensson it is impossible to delete videos inside the studio, because they have to be added to a document to access the delete button (and therefore can't be deleted as there are references to it).
Maybe it would be a good idea to have a delete button inside the browse/select grid (just like we have inside the image select grid)

Did anyone manage to find a way through this? Multiple versions later and this is still a problem, totally unable to remove a video from a field. Seems to be a very core functionality that bugs out and certainly getting in the way of using Mux in this project.

Has anyone figured out a solution to this?
I am in the same boat as others above... The plugin still does not allow a video to be removed and when I create a separate module/page section containing a mux component, the plugin breaks with a Mutation(s) failed with 1 error(s) Client Error:
POST https://*studio-code*.api.sanity.io/v2021-05-17/data/mutate/production?tag=sanity.studio&returnIds=true&returnDocuments=true&visibility=sync 409

On top of this, if I delete a video within the Mux dashboard the videos are not updated within the Select Videos tab. Unfortunately this means that instead of using Mux for a client, I am going with Vimeo most likely (which isn't bad, but not ideal).
@kmelve is there any timeframe on this fix? This is the only breaking change that is really holding many developers back from using this plugin/going with Mux over another video service.

Facing the exact same issue on my end as well. Unable to remove a video. Looking for a fix.

It does seem to work in the new Sanity Studio v3 using version 2.0.2 of the plugin. The videos in the select grid have a delete option where you can delete the video from the dataset and also from Mux.

Closing this issue as deleting assets is now fully supported and respects strong references being used.
Thank you all for showing up, keeping track of updates and helping debug 🙏


Debug should allow uri in DebugProtocol.Source

In the Java language server, we can open classfiles with source attachment in the editor. It uses virtual documents with our own uri scheme. (e.g. jdt://contents/rt.jar/java.io/PrintStream.class).
We would like the debugger's stackframe location to return that uri too.
However, current debug protocol always expects file path. See the debugSource.ts https://github.com/Microsoft/vscode/blob/master/src/vs/workbench/parts/debug/common/debugSource.ts
Ideally DebugProtocol.Source would also support a field uri that could be used instead of a path

The debug protocol has support for this: https://github.com/Microsoft/vscode-debugadapter-node/blob/master/protocol/src/debugProtocol.ts#L244
But VS Code uses native paths when communicating with the DA.
Flipping this switch in VS Code would probably break many debug extensions that are not prepared for passing URIs instead of native paths.
So we have to find a way around this...

Currently language server protocol textDocument/definition supports uri in its response Location, and supports user customized TextDocumentContentProvider to parse the special uri scheme (e.g. jdt://contents/rt.jar/java.io/PrintStream.class).
But vscode debugger doesn't support setBreakpoints in the user customized editor. See the code debugConfigurationManager.ts#canSetBreakpointsIn. It's a problem that the user is able to view the third party source via go to definition feature, but cannot set breakpoints there.
If the debug protocol has some configuration items to allow the specific uri schemes to be set breakpoints in, it would be better.
@weinand @isidorn

@weinand What about adding a new field 'uri' to DebugProtocol.Source and deprecated 'path'.
When 'uri' is set, it will be used instead of the path. In messages to the debug adapter, both should be set for backward compatibility.

@aeschli before we discussing solutions to the problem, let's try to understand the problem first.
@testforstephen's comment from above says:

vscode debugger doesn't support setBreakpoints in the user customized editor.

VS Code supports setting breakpoints for "languages" that are registered for breakpoints (this is the "breakpoints" contributions point).
Has this been done for the "class" files?
If not, you can easily enable setting breakpoints everywhere via the "debug.allowBreakpointsEverywhere" setting.
If breakpoints have been registered for "class" and you still cannot set breakpoints, then this is a bug.

The gif shows the problem.

when going to a declaration that is inside a JAR or has a source attachment, the Java language server returns a custom URI for a virtual resource:  jdt://contents/rt.jar/java.io/PrintStream.class.
the Java extension registers a resource provider for these URIs. The content is provided on demand by the language server
.class files are registered to the java language mode. The debug adapter enables breakpoint for java, but setting breakpoint doesn't seem to work
when debugging and stepping inside a class file, the debug adapter would like to return the same URI: jdt://contents/rt.jar/java.io/PrintStream.class so that the debug UI can open that resource. But source.path is limited to file paths, so that doesn't work.
Currently, the debug adapter returns class file contents through the inMemory mechanism, which leads to two different editors for the same content. Also, setting breakpoints isn't possible ahead of debugging.

@aeschli if setting breakpoints still doesn't work, then this is a bug. @isidorn could you please investigate.
The other issue is already being solved as we speak. The DA will be able to pass jdt://contents/rt.jar/java.io/PrintStream.class as a Source.path and VS Code will not assume that it is an native path but treat it as a uri.

Thanks for pointing this out.
There was an issue in the debug code that it allowed breakpoints only for certain schemas which was bogus. This was orignally introduced to prevetn setting breakpoints in settings files, but now I simple disable setting brekapoints in .json always. Or at least until a json debugger comes along hehe

This is now nicely supported aslo via 22d7830048
Anyways you should be able to send a Source.path as a string uri. And it will get properly handled here https://github.com/Microsoft/vscode/blob/master/src/vs/workbench/parts/debug/common/debugSource.ts#L33

@isidorn @weinand @aeschli thank you for the discussions. The solution looks perfect for me, i can't wait to verify it in the coming insider version. I'll update the status after i verified it in the next insider version.

@isidorn it seems the latest insider version is published at 2017-09-20. Can you share how often the insider build is taken?

@testforstephen normally, Insiders is created in the morning of every working day. However, due to the hurricane activities the signing service for Windows and macOS builds is currently failing and a backup service is overloaded. You can try the linux builds.

@aeschli It would be great, if we could verify this with the Java extension.

@weinand @aeschli @isidorn Today i have tried the latest insider in ubuntu 14.04 LTS, the feature works like a charm. Great thanks for the work.
Now for third-party class, the java debugger just need send the uri in Source.path, VSCode will use jdt content provider (registered by vscode-java extension) to render the class source automatically. Besides, i can also add breakpoint in jdt content provider before launching debugger. The debugger could work with java language server extension seamlessly.
Below is the insider version i used.

@weinand @isidorn
Log one more little thing here.
When launching a new debug session, the uri paths of the previously persisted breakpoints in the jdt content provider seem not consistent with the original uri that the java debugger returns.
The original uri in the StackTrace response body returned by the java debugger is jdt://contents/rt.jar/java.io/PrintStream.class?=1.helloworld/%5C/usr%5C/lib%5C/jvm%5C/java-8-oracle%5C/jre%5C/lib%5C/rt.jar%3Cjava.io(PrintStream.class.
The uri in the breakpoint request is changed to jdt://contents/rt.jar/java.io/PrintStream.class?%3D1.helloworld%2F%5C%2Fusr%5C%2Flib%5C%2Fjvm%5C%2Fjava-8-oracle%5C%2Fjre%5C%2Flib%5C%2Frt.jar%3Cjava.io%28PrintStream.class.
It's query segment is encoded by VSCode (reference uri.toString() https://github.com/Microsoft/vscode/blob/master/src/vs/base/common/uri.ts)
Although the encoded uri can be parsed by our java debugger very well, no feature break yet.
But if VSCode can keep the uri consistent, that would be better.


How can I apply this technology to my own business scenario?

We have our own database, should we also build a data structure similar to the Spider dataset? Our business scenario includes 40 tables, do we need to expand all these 40 tables into a data structure similar to Spider?

You can check the LICENSE attached to this repo.


Gemm, Gemv: Support symbolic expressions

I don't think this is the right solution. Instead, we should warn on those indeterminate cases rather than error out.
In SymPy:

M == M - 3 returns False
M == M returns True
M == N returns M == N aka indeterminate

if we check (size1 == size2) == False (note the == False rather than is False because SymPy is SymPy), that would be preferable.

@lukastruemper @tbennun @tbennun
I made #1321 as an alternative. If you agree, we can merge it, and Lukas can continue adding tests and amending the validation methods.

@lukastruemper @tbennun @phschaad
I made #1321 as an alternative. If you agree, we can merge it, and Lukas can continue adding tests and amending the validation methods.

Very nice, I'll be glad to review too once ready


infrastructure/kubernetes: pin latest version compatible with helm v2

nri-bundle versions older than 3.2.11 are not compatible with Helm v2, as they use a v3-introduced function get: https://github.com/newrelic/helm-charts/issues/671
This PR pins version 3.2.11 if Helm v2 is detected during installation.

Closing as we have decided the CLI should not support Helm v2 (internal thread).


Built-in VS Code Simple Browser Limitations and other Load/Close App Page and webview UI customization options

We researched this yesterday and shared preliminary info in slack:

looking at the built-in simple browser implementation I don't see a quick and easy way for us to open it on the side, or change that webview title and icon. So, if we want to see more stylized Evidence built-in browser and have more open/close/reload page options, we'd need to invest time into implementing our own. ... https://github.com/microsoft/vscode/blob/main/extensions/simple-browser/src/extension.ts

Also, that Simple Browser webview API doesn't provide close view option, and VS Code itself still doesn't have an API to iterate over all open webviews, only visibleTextEditors list is available. Therefore, it is not possible for us to close Simple Browser, without implementing our own or using another extension like the popular, but not maintained Browser Lite.
The only other quick workaround without investing 2-3 days on implementing your own custom Evidence App webview, is:

Have your app dev team add Start Dev Server "greeting" page to your app ./static folder and standard app template
We can set global state for when we open one of the Evidence pages in a Simple Browser extension, and what page it was on
On VS Code reload and Evidence extension activation, we can show new Start Dev Server "greeting" page in a Simple Browser, so that it doesn't show up blank.
When a user starts a server, we can take them to the last viewed page saved in the global state from the previous Preview session.
and lastly, your app still needs proper Loading ... home page while the server starts and generates pages. I've requested this multiple times in slack and tickets here to address long blank app page view in a browser.

That's a few hours, most likely a day of setup, dev, testing and coordination with the app dev team, if you'd like us to open a UX enhancement ticket to address it this or next week.
On the related note,  we can also open new active editor group on the side, and maybe add Preview on the Side for the Simple Browser to show up there. However, changing that built-in browser title and icon is not possible per my read of that built-in vscode extension API referenced above.
Originally posted by @RandomFractals in https://github.com/evidence-dev/evidence-vscode/issues/24#issuecomment-1569921632

@RandomFractals FYI added an issue in our main project about the page loading: https://github.com/evidence-dev/evidence/issues/922

@hughess thank you! even a starting static app page with some fancy animation could make it much better while we SSR those pages and modules, and get data.


pythonPackages.lmdb: enable tests

Motivation for this change
I noticed that the pythonPackages.lmdb tests were unnecessarily disabled for Python 3. I confirmed that the tests for python37Packages.lmdb were failing with the error described, and that they no longer fail (on Linux, at least.)
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[x] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

@GrahamcOfBorg build python2.pkgs.lmdb python3.pkgs.lmdb


Allow setting source (fields) from options

In case you want to limit returned fields.
Use case: search index has added custom fields that are searched but should not be returned in results

Sorry for late answer. could you pull master to resolve failed checks

done, checks pass.

cool. could you update README and CHANGELOG as well please?

I propose 7.10.0 next release

What do you think about checking the options['source'] is array or not? I am sure that it will throw an error. Do we need to check it or just need to assume. Ar least we can put a test in test cases with note 🧐

setSource takes array|string|bool so the logic is more complicated

Released. Thanks for contribution @hulkur


Too many newlines

This is not meant as a PR, but a bug report.

Thanks for writing these testcases to identify the problem. i ran into this while working on plone.recipe.codeanalysis. we get a lot of extra lines reported there. Issue:

https://travis-ci.org/timothycrosley/isort/jobs/78731166#L155
https://travis-ci.org/plone/plone.recipe.codeanalysis/jobs/78652480#L376

@timothycrosley do you have an idea about this?

@do3cc Thanks for reporting this issue and providing test cases! @saily thanks for the additional data point, I'm sorry to hear you are also running into this issue, ill add these test cases and fix before the next release Thanks!

This is already fixed (thanks @timothycrosley !) would it be possible to get a release? :smile:

@gforcada Welcome! Just released 4.2.0, which should fix this issue
Thanks!
~Timothy

Thanks!

Thanks!


bug fix for Get-WingetOutdatedApps when handling Chinese characters

Proposed Changes

(Describe the changes and rationale behind them)

Related Issues
https://github.com/Romanitho/Winget-AutoUpdate/issues/363#issuecomment-1646523644

Resolves #363

https://github.com/Romanitho/Winget-AutoUpdate/issues/363#issuecomment-1649125892

@Romanitho  I noticed that the Handle Chinese chars commit is not merged into the main branch. Also, for lines 61 to 62, the length of id and version should not subtract the $nameDeclination. Here’s the updated code:
            $software.Id = $line.Substring($idStart - $nameDeclination, $versionStart - $idStart).TrimEnd()
            $software.Version = $line.Substring($versionStart - $nameDeclination, $availableStart - $versionStart).TrimEnd()

Hi. Thanks for your feedback. Is it possible for you to do all the changes to make it to work correctly as I'm not on my computer at the moment. This way I just merge your new PR.

Hi,
After some investigations, this PR causing the issue #372
I guess that Russian characters are similar issue than Chinese one.
the GetByteCount part report wrong value.
probably we need to keep our first approach with your change  -replace '[\u4e00-\u9fa5]', '**'

maybe something like this ?
    #Get header titles
    $index = $lines[$fl] -split '\s+'

    # Line $fl has the header, we can find char where we find ID and Version
    $idStart = $lines[$fl].IndexOf($index[1] -replace '[\u4e00-\u9fa5]', '**'))
    $versionStart = $lines[$fl].IndexOf($index[2] -replace '[\u4e00-\u9fa5]', '**'))
    $availableStart = $lines[$fl].IndexOf($index[3] -replace '[\u4e00-\u9fa5]', '**'))

Hi, After some investigations, this PR causing the issue #372 I guess that Russian characters are similar issue than Chinese one. the GetByteCount part report wrong value.
probably we need to keep our first approach with your change  -replace '[\u4e00-\u9fa5]', '**'

I think I have figured out why this happen. Some characters like Russian characters have 2 bytes, but they have the same width with Latin characters. I think change all System.Text.Encoding]::UTF8.GetByteCount(*) to *.Length will fix this.

Do you make the change and submit a PR, or do you want me to change it?

Do you make the change and submit a PR, or do you want me to change it?

https://github.com/Romanitho/Winget-AutoUpdate/pull/374


Synchronising a folder on Dropbox / Google Drive

Currently synchronisation is only available on a file-by-file basis.
While it is possible to manually add multiple documents to the same folder on Dropbox or Google Drive, it isn't optimal as it takes extra time and is an error prone process - files may end up in a different Dropbox folder in case of typos.
It would be very useful if it was possible to synchronise a whole folder, and then any document that is in that folder would also get synchronised automatically.

Related: Issue #875
When opening a file from Dropbox, in the file picker list, you can select multiple files at once, so you at least don't have to do that multiple times. But they're all imported into the root folder in StackEdit, so may need reorganization once there.

Another use case: I use both Chrome and Firefox at work and at home. I'd like to have the same folder structure synced to StackEdit (from Google Drive, in my case) in all of those instances. Otherwise, it doesn't make sense for me to use folders in StackEdit, since I'd have to redo that work 4 times.
Yet another use case: It would allow me to import all of my StackEdit4 documents into StackEdit5 in one fell swoop, rather than individually. Btw, Thanks @benweet for bringing us v5!

You can now use Google Drive workspaces in v5.


Add Open Global Infrastructure Risk/Resilience Analysis

Insert URLs to the project here:
https://github.com/nismod/open-gira

[x] The projects is active, documented, open source licensed, shows usage from external parties and is directly targeting environmental sustainability. Find more details in the Contribution Guide.

All issues labeled as 'Good First Issue' of the project listed on OpenSustain.tech will be visible on ClimateTriage.com. This is a great way to welcome new community members to your project.

@thomas-fred @tomalrussell @tlestang


Simplify override.json and split config.json into two files

I confirm (by marking "x" in the [ ] below):

[X] This is not a troubleshooting question. Troubleshooting questions go here: http://www.mattermost.org/troubleshoot/.
[X] This doesn't reproduce on web browsers (such as in Chrome). If it does, issue reports go to the Mattermost Server repository.
[X] I have read contributing guidelines.

Summary
Simplify override.json and split config.json into two files. This is a follow-up to #586
1 - Having both default and 1 lists in the /config/base.json file might be confusing to admins/developers building the apps from the source. It's not immediately obvious why there are two sets of default settings present and what their purpose is.
Propose splitting this file into base.json (containing "default") and pastDefault.json (containing "1", and other versions in the future)?
2 - Split the base.json file into defaultPreferences.json (user config) and buildConfig.json (system config)?

Closed via #633


plot in a web page

I can't find any example for show plot into a web application
so is it possible to show an object in a web page ?
If yes, how can do it (a simple example can help me!)

what you can do is to generate html code to be embedded into web apps.
Examples are:
https://vedo.embl.es/examples/geo_scene.html
https://fedeclaudi.github.io/br-mouse.html
The above are using k3d for the actual renderings.
Exporting to X3D is also possible:
https://vedo.embl.es/examples/fenics_elasticity.html
To generate these pages check out python examples:
examples/other/export_x3d.py
examples/advanced/geological_model.py
basically you just need to do:
exportWindow('page.x3d')
or
exportWindow('page.html')

Thanks alot.
This method works.

I use export x3d. And my file is very large. So I can say that "this method is not a good way to explore data into web"
I used cavity.vtk example  and export to x3d, the file size was about 400 MB.
I change some parameter and reach a 140 MB file.

how big is your data file on disk, before exporting?

The in put file : cavity.vtk (43.8 kb)
Output  : .x3d (150 MB)
i used streamlines3.py code to generate x3d file

that's because there are too many vertices:
   nr. of actors: 4 (829711 vertices)
you can try removing the arrows and decimate the lines with:
stream.clean(0.005) # impose a min point distance of .5% of the bounding box size
but... the k3d is buggy for displaying lines so it might not work.. and x3d i'm not sure if it supports it ( in my browser it just doesnt work)
So it could be that streamlines3.py is a case where we cannot do much..

I have some question ,
I need to export without showing anything . Just calculate then export.

Use offscreen keyword:
from vedo import *

pan = load(datadir+'panther.stl')

show(pan, offscreen=True)

exportWindow('panther.html')

Traceback (most recent call last):   File "/media/ssmns/DriveD/Work/CFD_Project/cfdproject/workkpost.py", line 28, in <module>     show(stream,offscreen=True,axes=0)   File "/mnt/4ef0f442-5eb4-401e-83df-eaab2f011520/Programming/Python/miniconda/envs/pyWeb/lib/python3.8/site-packages/vedo/plotter.py", line 297, in show     _plt_to_return = plt.show(   File "/mnt/4ef0f442-5eb4-401e-83df-eaab2f011520/Programming/Python/miniconda/envs/pyWeb/lib/python3.8/site-packages/vedo/plotter.py", line 1774, in show     self.interactor.SetInteractorStyle(vtk.vtkInteractorStyleTrackballCamera()) AttributeError: 'NoneType' object has no attribute 'SetInteractorStyle'
when use offscreen =True, found this error

by beloow i export data .
`
vp = vedo.Plotter(axes=5)
vp.load('motorBike.obj')
vp += stream
vp += stream1
vp.show(interactive=0,offscreen=1)
vedo.exportWindow(target)
`
but I see a bug in exportWindows()
If I have a one stream, every thing is fine but when I used two streams in show, exportWindows to  x3d,  miss a </X3D> at end of file  .

AttributeError: 'NoneType' object has no attribute 'SetInteractorStyle'

Uhm are you running it from terminal or in a jupyter/ipython session?

If I have a one stream, every thing is fine but when I used two streams in show, exportWindows to x3d, miss a  at end of file .

indeed i never tested it for multiple objects, but the html output is only meant as an illustrative example, what matters is the x3d file. You have to create your own html to embed it the way you like.

i used in terminal not into jupyter .

indeed i never tested it for multiple objects, but the html output is only meant as an illustrative example, what matters is the x3d file. You have to create your own html to embed it the way you like.

The exported file.x3d missed </X3D> in last line, I add a </X3D> with a simple code, but  I can't find watts happen in your code to solve it.

for sever used (without x server) the
vp.show(interactive=0,offscreen=1) dosent work .
error:

[141CC74D] vtkOpenGLRenderWindows:1183 Error | vtkOpenGLRenderWindows (0x56 ..) bad x server conection . Display...

Hi all, not sure if anyone else is having this issue, but the exportWindow(.x3d) uses self closing html tags that don't always work in x3dom. Not a biggie, I edit them manually, but something to note if anyone else has the problem trying to embed to a website.
Specifically the components:  <Background /> <Viewpoint /> <NavigationInfo /><DirectionalLight />  that always start the <scene> . It may be safer to make these not-self closing?
I haven't fleshed this out fully yet, but there appears to be some jank with the axes when exported to .x3d. I have not been able to debug specifically yet, but will figure it out eventually and update. (ATM I think its a bad line transform, tick marks show out of place and grid lines look wacky).


vscode-languageclient 6.0.0-next.10

The main change is that it removes progress from the proposed section.

bors r+

bors r+


Add gl node weight expansions

The default algorithm for low n is now forward recurrence because the weights in Golub-Welsh become exactly zero long before they should underflow #52 . The corresponding functions laguerreRec and laguerreRecDer can be called with BigFloats and this can be extended to gausslaguerre.jl as well if for example besselroots.jl can handle BigFloats  #22 .
method="RH"  is removed such that the code is a lot shorter and the function polyAsyRHgen can still be called if expansions of corresponding orthogonal polynomials are needed: I can change the normalisation to something standard if required.
The timings mentioned in README.md should be repeated on the reference machine, as the explicit expansions are extremely fast: for example, the six Newton iterations required to compute t seem to take more than half the total execution time of gausslaguerre. Higher-order asymptotics were not implemented in the Airy region (currently only O(n^{-4}) for the nodes and O(n^{-2/3}) for the weights) as the latter easily underflow, like the subsampled quadrature rule in reference [7]. I plan to do something similar for Jacobi (, Legendre) and Hermite.

Codecov Report

Merging #53 into master will decrease coverage by 2.94%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master      #53      +/-   ##
==========================================
- Coverage   97.72%   94.78%   -2.95%
==========================================
  Files           8        8
  Lines        1849     1074     -775
==========================================
- Hits         1807     1018     -789
- Misses         42       56      +14

Impacted Files
Coverage Δ

src/gausslaguerre.jl
92.64% <ø> (-5.84%)
:arrow_down:

src/gausslegendre.jl
97% <0%> (-0.98%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c3db719...3e73d9d. Read the comment docs.

There has been some progress on this pull request. A lot of memory allocations were removed, some still remain in the evaluation of special functions (these could be removed by implementing an alternative evaluation method). Even without those, this is a lot faster than the previous implementation without explicit expansions.
The code for more generalized weight functions was moved to gaussfreud.jl, since the generalizations correspond to Freud-type weights (exp(-x^m) for m greater than 1).

Some more changes to this pull request are forthcoming, and it is currently quite a bit behind master. Stay tuned.

@daanhb, do you want to update this PR so that I can take a look and merge?

Please see #4 for an update to this pull request.

Is this PR still active?

No, it was superseded by #69 which is now merged.


Adding title and cve's to openscap_rule_result creation

New pull request for https://github.com/ManageIQ/manageiq/pull/17219#issuecomment-401064220 with a rebase.
In conjunction with PR OpenSCAP/ruby-openscap#12
Gives the ability to store OpenSCAP Rule title and linked CVE's. This gives usefulness to the OpenSCAP reporting options in cloudforms. Currently only only a non human friendly id is present and there are no links to the actual CVE references.
This change also requires a schema change to the database:
class AddCVEToOpenScapRuleResults < ActiveRecord::Migration[5.0] def change add_column :openscap_rule_results, :title, :string add_column :openscap_rule_results, :cves, :string end end
@hsong-rh please review.

@ianwatsonrh 👍
@roliveri Please review, thanks.


feat: create product image type

This PR defines types for the ProductImage data the browser extension will be reading and
writing to.

Current dependencies on/for this PR:

l/replace-button-component

PR #218   (this PR)

PR #219

This comment was auto-generated by Graphite.


Question: is interaction telemetry ready to be used?

While browsing this repo I've noticed this section on interaction telemetry:
https://github.com/aws-observability/aws-rum-web/blob/main/docs/configuration.md#interaction
As far as I can tell this is not documented anywhere else (https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-RUM.html) and it's unclear to me whether I can try to use it to record DOM events.
Thank you in advance 🙏

The interaction telemetry is safe to use for recording DOM events. For example, if I wanted to record clicks on elements with attribute data-rum="record_this or data-rum="record_that", then I might configure the web client (the interaction telemetry specifically) like so:
  const config: AwsRumConfig = {
    sessionSampleRate: 1,
    endpoint: "https://dataplane.rum.us-west-2.amazonaws.com",
    telemetries: [
        "performance",
        "errors",
        "http",
        [ "interaction", {
                interactionId: (e) => event.target.getAttribute("data-rum"),
                enableMutationObserver: true,
                events: [
                    { event: 'click', cssLocator: '[data-rum="record_this"]' },
                    { event: 'click', cssLocator: '[data-rum="record_that"]' },
                ]
            }
        ],
    allowCookies: true
  };

@qhanam thanks for the answer! Once I set up an interaction telemetry, how can I see the recorded clicks?

@tlakomy the CloudWatch RUM console doesn't show click metrics yet. A workaround is to export the data to CloudWatch Logs. From there you can create metrics and dashboards for DOM events, and search the raw events using CloudWatch Insights queries.


Azure Virtual Machine Scale Set is not returning the publicIPPrefixID after a SKU update

To reproduce:

Create a VMSS with a publicIPPrefixID assigned by using the following tf config

resource "azurerm_linux_virtual_machine_scale_set" "linux_nvavss" {
  name                            = "test"
  location                        = azurerm_resource_group.main.location
  resource_group_name             = azurerm_resource_group.main.name
  computer_name_prefix            = "dapzhang"
  admin_username                  = "dapzhang"
  admin_password                  = "Passw0rd123!"
  disable_password_authentication = false
  instances                       = 2

  # automatic rolling upgrade
  upgrade_mode = "Manual"
  sku          = "Standard_F2"

  zones        = []

  source_image_reference {
    publisher = "Canonical"
    offer     = "UbuntuServer"
    sku       = "16.04-LTS"
    version   = "latest"
  }

  os_disk {
    caching              = "ReadWrite"
    storage_account_type = "Standard_LRS"
  }

  network_interface {
    name                          = "eth0"
    primary                       = true
    enable_accelerated_networking = true
    enable_ip_forwarding          = false
    ip_configuration {
      name                                   = "ipconfig1"
      primary                                = true
      subnet_id                              = azurerm_subnet.front.id
      load_balancer_backend_address_pool_ids = [azurerm_lb_backend_address_pool.test.id]
      public_ip_address {
        name                    = "instancePublicIP"
        idle_timeout_in_minutes = 30
        domain_name_label       = "testdomain"
        public_ip_prefix_id     = azurerm_public_ip_prefix.test.id
      }
    }
  }
}

Update the VMSS sku with the following request body:

   "properties": {
        "upgradePolicy": {
            "mode": "Manual"
        },
        "virtualMachineProfile": {
            "networkProfile": {
                "networkInterfaceConfigurations": [
                    {
                        "name": "eth0",
                        "properties": {
                            "primary": true,
                            "enableAcceleratedNetworking": true,
                            "dnsSettings": {
                                "dnsServers": []
                            },
                            "ipConfigurations": [
                                {
                                    "name": "ipconfig1",
                                    "properties": {
                                        "subnet": {
                                            "id": "/subscriptions/85b3dbca-0000-0000-0000-67a141095a76/resourceGroups/test/providers/Microsoft.Network/virtualNetworks/network/subnets/front"
                                        },
                                        "primary": true,
                                        "publicIPAddressConfiguration": {
                                            "name": "instancePublicIP",
                                            "properties": {
                                                "idleTimeoutInMinutes": 30,
                                                "dnsSettings": {
                                                    "domainNameLabel": "testdomain"
                                                }
                                            }
                                        },
                                        "privateIPAddressVersion": "IPv4",
                                        "applicationGatewayBackendAddressPools": [],
                                        "applicationSecurityGroups": [],
                                        "loadBalancerBackendAddressPools": [
                                            {
                                                "id": "/subscriptions/85b3dbca-0000-0000-0000-67a141095a76/resourceGroups/test/providers/Microsoft.Network/loadBalancers/acctestlb-001/backendAddressPools/test"
                                            }
                                        ],
                                        "loadBalancerInboundNatPools": []
                                    }
                                }
                            ],
                            "enableIPForwarding": false
                        }
                    }
                ]
            }
        }
    },
    "sku": {
        "name": "Standard_D3_v2",
        "tier": "Standard",
        "capacity": 2
    }
}

Get the updated VMSS and found publicIPPrefixID is empty.

Here are two assumptions:

The PATCH request erased the public ip prefix
The PATCH request did not change the public ip prefix, but it just did not returned after the PATCH is done

@avirishuv, could you please take a look at this.

@ArcturusZhang In the second step where you are updating the VMSS, the new "publicIPAddressConfiguration" does not provide the "PublicIpPrefix" as a property. So this is getting updated with an empty value and the original value is getting removed. This seems like expected behavior - please help clarify why you think this is an issue.

Hi @avirishuv I think I found the root cause of this.
Here the update version of the PublicIPAddressConfigurationProperties, it only has two properties (without publicIPPrefix). But the create version of the PublicIPAddressConfigurationProperties has more properties (with publicIPPrefix).
This is the reason in the update body, we do not have a publicIPPrefix assigned, and therefore the public ip prefix is erased.
So I assume maybe this is a swagger issue? Is the publicIPPrefix is updatable?

Thanks for the inputs, let me check the property publicIPPrefix and whether it should be added to the update version also.

Reviewing this internally with NRP team.  May be bug on NRP side preventing this from being updated.
cc @NitinGoelMsft

@brianlehr any update on this?

This should be in NRP queue as a bugfix.  Apologies as I'm not sure how to sort it there.

Assigning back to Avi to look into the issue and route to appropriate queue.

@brianlehr any updates on this ?

Please assign to @NitinGoelMsft .  This is a known bug his team is addressing.

@brianlehr @NitinGoelMsft Any progress or any place to follow progress?  I believe this issue is also impacting the ability to change the instance count via terraform when a vmss is used in conjunction with a public ip prefix.

We were able to reproduce the issue. The exact behavior is explained below.

Suppose customer creates a VMSS with public ip prefix using a terraform template, all the VMs are assigned an IP address according to the public Ip prefix
Now when you scale the VMSS (say change the instance count from 3 to 4) and redeploy the template, this will trigger an update on VMSS. Terraform does not send “publicIp prefix” in the update request to the CRP, because of which the publicIp prefix is lost and the new VM will be assigned a random Ip address.
Now remember that there are 2 update groups in VMSS (one update group which has public Ip prefix in the network profile with which the old instances were created and the other update group without the public Ip prefix with which the new instance was created)
Whenever Terraform encounters more than one update group during scaling of VMSS, it just initiates a DELETE on the existing VMSS and recreates a new one. So during the next scaling operation (say you change the instance count from 4 to 5), the existing VMSS will be deleted and a new VMSS will be created. This behavior is again specific to Terraform. However when the request is sent to create a VMSS, Terraform does send publicIp prefix in the request and in this case, there is no issue and all the VMs will be assigned an Ip address according to the public Ip prefix (also, in this case there is only one update group in VMSS since all the VMs are created using one configuration)
The above cycle continues. So every alternate scale up operation will have the issue where the new VMs are assigned a random Ip address. Every other alternate scale up operation will delete the existing VMSS and recreate it with the new instance count, in which case there is no issue with the Ip address assigned to the VMs.
Because of this weird behavior which is specific to Terraform, a temporary solution to the issue (when the new VMs are assigned a random Ip address) is to just redeploy the template (works without making any changes too), which will destroy the existing VMSS and recreate a new one without the above mentioned issue.

Coming to the long term fix, there are 2 issues here -

Terraform does not send the publicIp prefix in the request to Compute when it wants to update the VMSS. At this point, we are not sure how it generates this request. Should this issue be assigned to Terraform to fix their request?
It is weird that it tries to delete the VMSS during every alternate scale up operation (when it encounters 2 update groups). We are not sure if this is intended behavior because when it tries to delete and recreate the VMSS, all the VMs will be assigned a new IP address which might be different from the old one (even if the new IP address is within the range of public Ip prefix). Since this behavior is specific to Terraform, should this issue be assigned to Terraform as well?

Note that Terraform sends the request to Compute and then Compute forwards the request to NRP. NRP does not own any logic to delete and recreate the VMSS if there are 2 update groups. NRP just performs the necessary operation according to the request.

@kkakarla4820 Please note we see similar behavior for non-Terraform usecases also, e.g. when we want to modify a Public IP Prefix that is associated to a VMSS Uniform instance.

Any updates here? This is still breaking Terraform uses of VMSS with public IP prefix.

@NitinGoelMsft FYI

Any recent update on this issue? my Cx is facing a blocker due to VMSS recreate every time the run terraforms without any changes to the code.

I checked in this fix long back - https://github.com/karthikka4820/azure-rest-api-specs/commit/d3e9a5099d46e15fb50493e3bc92bd1b536ec57b
But I do not see this change anymore in the repo. I am not sure what happened there. Will follow-up with swagger team

@karthikka4820 could you please help understand how to test this. Also does this issue also cause VMSS being recreated every time the Terraform is run even without any changes to the code?

It looks like the API bug is still open and owned by the Terraform AzureRM changes associated with that issue. There is a workaround for it in the provider that is expected to be merged in the latest release. Can someone please share the workaround with a sample code example please.
Thanks in advance!

Not sure if the solution is still implemented but if anyone looking for a workaround the network configuration do include below in Terraform to get ahead of the VMSS recreation. Hope this helps broader community.
lifecycle {
ignore_changes = [
instances,
network_interface,
custom_data
]
}


Feature Request: depopulate labels from the plist when successfully installed

With the goal of inconveniencing stressed employees as little as possible with app updates, we consider running AppAutoPatch twice.
In the first run, we update all apps that are not open with the options interactiveMode="0" (completely silent), discover="true" and BLOCKING_PROCESS_ACTION="silent_fail". During the discover phase, the labels are written to the plist that Installomator runs.
The second pass is to update only the apps where Installomator aborted the installation because there would have been user interaction. This time, so that interactive patching can be started at the push of a button, we choose interactiveMode="1", discover="false", and BLOCKING_PROCESS_ACTION="prompt_user". But the plist still contains all labels, even the ones for which Installomator has applied the update.
Hence my suggestion to include an option that removes labels with "All done!" from the AppAutoPatch.plist. This way the second pass would only bother the user to install opened applications.

Can you check out if the new deferrals and AAP-Activator script may help you out?

@Apfelpom , with the latest version 2.0.3 there will be an additional check for the installed app version. For your workflow may I suggest to re-run discovery each time, you can set the interactiveMode to 1 so the user will only be presented with the apps that indeed need updates at the time of discovery. You can keep prompt_user as the blocking process since you can now pass in your own Installomator labels into a Jamf Pro Script parameter.

The deferrals and the APP-Activator-script, as well as all additions since version 2 are great. Thank you all for your work! (and sorry to be late)
If, as you recommend, we do not perform discovery interactively and installation interactively, users will have to wait until the closed applications that do not require interaction are updated. Hence the idea of running AAP silently and a second time interactively when the user has really to take a decision and click a dialog.
Moreover we want the discovery phase to run as little as possible, as it takes up time and energy. During our first run of APP, all applications that are closed are updated without interaction. Open applications still need their update.
If we start AAP a second time without discovery and with interactiveMode 1, all already installed updates of closed applications are run through. This is fast, because Installomator determines the last version beforehand, but it takes more effort than necessary and on screen time.
Hence the desire to fully evaluate the results of the first run and obtain a list of applications that really need updating, which updates can be run interactively in a second run of AAP.
The https://github.com/robjschroeder/App-Auto-Patch/issues/48 issue shows that I'm not the only one who wants to run discovery and silent patching. I think it would also be an improvement for APP-Activator to know that after a silent run, the list of applications to be updated has been cleaned up, so that a new discovery or an interactive run is not necessary.
BTW: To speed up APP (significantly), I have been toying with the idea of reducing the list of Installomator labels to the actual list of our software. This could also be an option of APP to ignore Installomator labels of never installed software, provided APP has already run once.


Internet Explorer restricted this website from running scripts or activex control

ie 11
windows 7
running Norton Security

is this normal behavior of remark to require user authorization to be executed upon page load?

curiosity is there about the functionality of remark, because I am looking for a slideshow solution that does not require user permissions to be able view the slideshow..   scripts from other websites work just fine on this same browser/os combination.

admittedly, this windows installation that I have access to right now is very restrictive.  also, admittedly, I know very little about how remark works.  maybe I am overlooking something.

You are the only one I can recall ever having brought this issue up. I'm guessing your system settings may be causing this.

that is probably true.  i  was running remark on one of the most locked down machines windows machines i've ever seen.  it is good to get your feedback.  the browser simply asked for permission to run remark, which may not be a very acceptable side-effect considering that this issue is likely to be extremely rare.


Collecting Human Trajectories Using Playwright

Hello,
I am currently working on a project that involves the collection of Human Trajectories, and I noticed your project utilizes Playwright for this purpose. I have a few questions regarding how you achieved this:
From my understanding, Playwright’s official tools typically create a new Chromium interface where the user can perform actions, and the tool records those actions. However, it seems that while it captures user operations, it doesn’t record the page information itself in real-time. The page details can be obtained upon replaying the actions, but I am curious if there is a method to record both the user actions and the page information simultaneously during the interaction.
Could you please share some insights or suggestions on how to approach this? Any guidance would be greatly appreciated!
Thank you for your time and this wonderful project!

Hi,
Have you look into the zip file you obtain after recording the trace? It should record the raw HTMLs which you can inspect with playwright viewer. Just to note that playwright can be quite noisy though.


GoodSignal type binding

I was wondering if it would be possible to get a type binding for GoodSignal that you would consider adding. As of now when using GoodSignal and importing it through wally I have to add my own type binding but when I use the wally install command it removes that binding I'm able to use these types by using wally-package-types
export type GoodSignal = typeof(Signal.new());
I would also ask if possible you update it to the wally registry so I can then use this.

I will let this fade


Implement vector length definition at init time in PGVector for indexing

Feature request
Allow manual definition of embedding column length in PGVector
Motivation
Right now embeddings stores in PGVector are initialised with:
embedding: Vector = sqlalchemy.Column(Vector(None))
This won't allow indexing of the DB (https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/howto-optimize-performance-pgvector)
Your contribution
I can open a PR shortly to fix it

Please get this into the next release - langchain is a ways behind llamaindex when it comes to pgvector support.  They already support defining your own table names, correct column init,  both index based retrieval methods, as well as hybrid search.  Langchain support is not production ready without just using SQL.


Make error types non_exhaustive

We cannot be sure that these error types are exhaustive. Thus, do mark them as non_exhaustive for now.

bors merge


Lineage best practice #863

Related Issue(s): #863
Proposed Changes:

Moved the derived_from note to best practices, makes the item spec a bit slimmer and bundles details at a single place.
Added idea from #863 for lineage information to BP.

PR Checklist:

[x] This PR is made against the dev branch (all proposed changes except releases should be against dev, not master).
[x] This PR has no breaking changes.
[ ] I have added my changes to the CHANGELOG or a CHANGELOG entry is not required.
[ ] This PR affects the STAC API spec, and I have opened issue/PR #XXX to track the change.

We'll have a discussion on that tomorrow, hope to get that in afterwrds.

I'll try to make a small extension for lineage fields in the provider arrays....

I think we should merge this with #907!

+1 on merging with #907

+1 closing in favor of #907

Closed in favor of #907.


A couple of questions while using Cisco runtime libacvp

Hello,
Currently, I am able to talk with ACVP demo server by using Cisco Runtime libacvp app and successfully perform AES, TDES, CMAC, SHA and HMAC tests (the Crypto Module is OpenSSL 1.1.1). How ever, I ran into a couple questions about how to find out the vendorID and how to create a new testSession.
Question 1: Regarding how to find out the newly created VendorID, below is the details of what I did:

Went to /libacvp/metadata# path and modified original vendor.json file to match the new vendor information and saved the edits.
Ran command './app/acvp_app --post ./metadata/vendor.json /acvp/v1/vendors' to ask for creating a new vendor at ACVP demo server.
Obtained the response as below.
"acvVersion" : "1.0"
}, {
"url" : "/acvp/v1/requests/1484",
"status" : "initial"
} ]

(My understanding is that the number 1484 is the new Vendor Request ID. Please correct me if i am wrong.)
4. Issued the command './app/acvp_app --get /acvp/v1/vendors/1484' to check if the new vendor information that was edited in the vendor.json file has been accepted by ACVP server.
5. However, I got the following response:
***ACVP [STATUS][log_network_status:1362]--> GET...
Status: 200
Url: https://demo.acvts.nist.gov:443/acvp/v1/vendors/1484
Resp:
[ {
"acvVersion" : "1.0"
}, {
"url" : "/acvp/v1/vendors/1484",
"name" : "INTEGRITY Security Services",
"website" : "www.ghs.com",
"emails" : [ ],
"phoneNumbers" : [ ],
"contactsUrl" : "/acvp/v1/vendors/1484/contacts",
"addresses" : [ {
"url" : "/acvp/v1/vendors/1484/addresses/1484",
"street1" : "7585 Irvine Center Dr.",
"street2" : "Suite 250",
"street3" : null,
"locality" : "Irvine",
"region" : "CA",
"country" : "USA",
"postalCode" : "92618"
} ]
} ]
It looks like the vendor information displayed after command '--get /acvp/v1/vendors/1484' is not the one that i edited in my vendor.json. Any suggestions?
Question 2: To create a New testSession intended for a demo server certificate, can you please provide me with an example command? I used command '--post testSessions', but it reported an error. Please see below.
root@acvp1:~/richard-libacvp/libacvp# ./app/acvp_app --post testSessions
Using the following parameters:
ACV_SERVER:     demo.acvts.nist.gov
ACV_PORT:       443
ACV_URI_PREFIX: /acvp/v1/
ACV_CA_FILE:    /root/libacvp-master/certs/acvp.nist.gov.crt
ACV_CERT_FILE:  /root/libacvp-master/GSS-ACVP.cer
ACV_KEY_FILE:   /root/libacvp-master/GSS-ACVP.pem

***ACVP [STATUS][log_network_status:1397]--> POST Login...
Status: 200
Url: https://demo.acvts.nist.gov:443/acvp/v1/login
Resp:
[ {
"acvVersion" : "1.0"
}, {
"accessToken" : "eyJhbGciOiJIUzI1NiJ9.eyJqdGkiOiJjNDI5NWYwMC1jZGJjLTRkYTYtYWJjNy0xMjJmNTlmYzYyZDMiLCJpYXQiOjE1NzU3NTE2MDQsInN1YiI6IjEuMi44NDAuMTEzNTQ5LjEuOS4xPSMxNjE4NDU2NDRkNmY3MjcyNjk3MzQwNDc2ZjczNzM2MTZkNjU3MjUzNjU2MzJlNjM2ZjZkLENOPU5WTEFQIExhYiBDb2RlIDIwMDk5Ny0wLE9VPUNTVEwsTz1Hb3NzYW1lcixMPUNhdG9uc3ZpbGxlLFNUPU1hcnlsYW5kLEM9VVMiLCJpc3MiOiJOSVNUIEFDVlRTIiwiZXhwIjoxNTc1NzUzNDA0fQ.BVgMfjhDPov8DXU2KfPMbYVLeoH4YNYdgmfFNc8Mzcc",
"largeEndpointRequired" : true,
"sizeConstraint" : 4194304
} ]
***ACVP [STATUS][acvp_parse_login:1713]--> JWT: eyJhbGciOiJIUzI1NiJ9.eyJqdGkiOiJjNDI5NWYwMC1jZGJjLTRkYTYtYWJjNy0xMjJmNTlmYzYyZDMiLCJpYXQiOjE1NzU3NTE2MDQsInN1YiI6IjEuMi44NDAuMTEzNTQ5LjEuOS4xPSMxNjE4NDU2NDRkNmY3MjcyNjk3MzQwNDc2ZjczNzM2MTZkNjU3MjUzNjU2MzJlNjM2ZjZkLENOPU5WTEFQIExhYiBDb2RlIDIwMDk5Ny0wLE9VPUNTVEwsTz1Hb3NzYW1lcixMPUNhdG9uc3ZpbGxlLFNUPU1hcnlsYW5kLEM9VVMiLCJpc3MiOiJOSVNUIEFDVlRTIiwiZXhwIjoxNTc1NzUzNDA0fQ.BVgMfjhDPov8DXU2KfPMbYVLeoH4YNYdgmfFNc8Mzcc
***ACVP [ERR][acvp_post_data:2480]--> JSON val parse error
root@acvp1:~/richard-libacvp/libacvp#
It would be very appreciated for any suggestions.
Thanks!

(My understanding is that the number 1484 is the new Vendor Request ID. Please correct me if i am wrong.)
That is the ID associated with the POST request.  You can get the vendorID by issuing a GET on that
request ID like this:
./acvp_app --get /acvp/v1/requests/1484
You'll get something like this back:
}, {
"url" : "/acvp/v1/requests/10",
"status" : "approved",
"approvedUrl" : "/acvp/v1/vendors/11444"  <--- your vendorID will be here.
} ]
So #4 and #5 should give you the proper vendor info if you use the correct vendorID
There are 2 ways to request a certificate.  First is to submit your vendor and person metadata and then run your test with --fips_validation metadata/<file with vendor, person, module and oe info>
See metadata/validation.json for an example. After the test completes a PUT is made to request the cert.  There will be a similar request ID you'll need to do a GET on to see when the cert is generated.
See #328 for a good description of these steps with example commands.
The other way is to POST vendor, person, module and oe at some point and then run the test. Once the test is complete there is a file generated called testSession_.json. You can add your module and oe info to that file and then perform a PUT.  The command line will look like this:
app/acvp_app --put testSession_24254.json
And you have to add your module and oe to the testSession file so it looks something like this:
[ {
"url": "/acvp/v1/testSessions/24254",
"jwt": "eyJhbGciOiJIUzI1NiJ9.eyJqdGkiOiI3ZDEyMWFkYS04OTlmLTRlYTEtOWJjMi04NDY5OTgyM2U0NTMiLCJpYXQiOjE1NjgxNTAxNzYsInN1YiI6IjEuMi44NDAuMTEzNTQ5LjEuOS4xPSMxNjEyNjI2Njc1NzM3MzY1NmM2YzQwNjM2OTczNjM2ZjJlNjM2ZjZkLENOPUFDVlBfQ2lzY28sT1U9U1RPLE89Q2lzY28sTD1SVFAsU1Q9TkMsQz1VUyIsImlzcyI6Ik5JU1QgQUNWVFMiLCJleHAiOjE1NjgxNTE5NzYsInZzSWQiOiJbNDg1NjEsIDQ4NTYyLCA0ODU2MywgNDg1NjQsIDQ4NTY1LCA0ODU2NiwgNDg1NjcsIDQ4NTY4LCA0ODU2OSwgNDg1NzAsIDQ4NTcxLCA0ODU3MiwgNDg1NzMsIDQ4NTc0LCA0ODU3NSwgNDg1NzYsIDQ4NTc3LCA0ODU3OCwgNDg1NzksIDQ4NTgwLCA0ODU4MSwgNDg1ODIsIDQ4NTgzLCA0ODU4NCwgNDg1ODUsIDQ4NTg2LCA0ODU4NywgNDg1ODgsIDQ4NTg5LCA0ODU5MCwgNDg1OTEsIDQ4NTkyLCA0ODU5MywgNDg1OTQsIDQ4NTk1LCA0ODU5NiwgNDg1OTcsIDQ4NTk4LCA0ODU5OSwgNDg2MDAsIDQ4NjAxLCA0ODYwMiwgNDg2MDMsIDQ4NjA0LCA0ODYwNSwgNDg2MDYsIDQ4NjA3LCA0ODYwOCwgNDg2MDksIDQ4NjEwLCA0ODYxMSwgNDg2MTIsIDQ4NjEzLCA0ODYxNCwgNDg2MTVdIiwidHNJZCI6IjI0MjU0In0.-wJlSRuC5xBaAmtZU4SKT6SKGHUAMoB83HAwzLJFZHs"
},     {
"oeUrl": "/acvp/v1/oes/21274",  <--- add this and
"moduleUrl": "/acvp/v1/modules/11230" <--- add this
}
]

Thanks much @bfussell for the quick turnaround, even on the Saturday evening time !
After reading your message, I used the first method and successfully received the "validationId" : "A10079" for my HMAC test on the ACVP demo server. Really appreciate your help !!!
In addition, the testSession ID I received is 42812. By using the first method, if i gave the command './app/acvp_app --put testSession_42812.json', the system would report  "error" : "Test Session has already been published.". Please see the output below.
root@acvp1:~/richard-libacvp-demo/libacvp# ./app/acvp_app --put testSession_42812.json
Using the following parameters:
ACV_SERVER:     demo.acvts.nist.gov
ACV_PORT:       443
ACV_URI_PREFIX: /acvp/v1/
ACV_CA_FILE:    /root/libacvp-master/certs/acvp.nist.gov.crt
ACV_CERT_FILE:  /root/libacvp-master/GSS-ACVP.cer
ACV_KEY_FILE:   /root/libacvp-master/GSS-ACVP.pem

***ACVP [STATUS][log_network_status:1413]--> PUT testSession Validation...
Status: 400
Url: https://demo.acvts.nist.gov:443/acvp/v1/testSessions/42812
Resp: {
"acvVersion" : "1.0",
"error" : "Test Session has already been published."
}
***ACVP [STATUS][acvp_put_data_from_file:2772]--> Failed to perform PUT
root@acvp1:~/richard-libacvp-demo/libacvp#
Thus, my understanding is that by using the first method to request the algo cert., the tester doesn't need to issue a PUT command. It was automatically conducted by the libacvp. Is my understanding correct?
Last question: Before stepping up to take the algo testing on ACVP Production Server, ACVP team sent us the following requirement:
"We need the JSON that you sent when you performed the “certify” step specifically, as outlined in section <IP_ADDRESS>."
Does it mean we need to send the generated testSessions_42812 file to ACVP for review?
Thanks,
-Richard

Correct, you either use --fips_validation or --put not both.
You probably need to send the json file you used with --fips_validation, but they may also want the testSession number.  You should ask them what they want to be certain.

Again, thanks much @bfussell .
One more question:
While reviewing the HMAC algo testing result, I noticed that the  parameter of prerequisites is empty. Below is the example from HMAC-SHA2-512 testing.
"algorithmCapabilities" : [ {
"algorithm" : {
"name" : "HMAC-SHA2-512",
"mode" : null,
"revision" : "1.0"
},
"capabilities" : {
"macLen" : [ {
"min" : 32,
"max" : 512,
"increment" : 8
} ],
"keyLen" : [ {
"min" : 256,
"max" : 448,
"increment" : 8
} ]
},
"prerequisites" : [ ]
},
As the prerequisite algorithm for HMAC-SHA-512 is SHA-512, what file shall i go to fill in the prerequisite information?
Thanks,
-Richard

You'd have to ask NIST, mine does not show a prerequisite either.   You can ask questions or open issues here:
https://github.com/usnistgov/ACVP

Hi Berry,
Thanks for the clarification. I looked back into the draft ACVP (draft-fussell-acvp-spec-00) and found out that section <IP_ADDRESS> has the following requirements to the algo prerequisites:
"algorithmPrerequisites - array of algorithm prerequiste objects, optional, for any algorithm that has a prerequisite that was not included in testing, the prerequisite MUST be provided by adding an element to this array
algorithm - string, name of the algorithm
mode - string, mode of the algorithm, optional, not all algorithms have a mode
prerequisites - [CREF8] string, array of prerequiste objects
algorithm - string, required
validationId - string, required
[
{"acvVersion": },
{
"moduleUrl": "/acvp/v1/modules/20",
"oeUrl": "/acvp/v1/oes/60",
"algorithmPrerequisites": [{
"algorithm": "TEST_ALGO_1",
"prerequisites": [
{
"algorithm": "TEST_ALGO_0",
"validationId": "123456"
},
{
"algorithm": "TEST_ALGO_0.1",
"validationId": "123456"
}
]
}]
}
]
Question: what file does a libacvp User need to go so as to fill in the prerequisites information?
Thanks a lot,
-Richard

app/app_main.c in the registration code.
They are all presently set to "same" which means the prerequisite algorithm is part of the same module that is under test.  For the openssl FOM thaqt is an accurate registration. The only time you need something other than "same" is if your prerequisite (SHA in this case) implementation is not part of the module under test.  I see no issue other than how the NIST server presents that information when you GET the validation information.

Berry,
For the 2nd case mentioned in your message, if the prerequisite algo was implemented by the other module (not the same algo implementation), the libacvp User needs to go to app/app_main.c to make the updates to fill in the prerequisite information and then to recompile the ./app/acvp_app. Is it the correct process?
Thanks,
-Richard

yes

Thanks @bfussell
-Richard

Hi Berry,
A follow up question regarding the output from command './app/acvp_app --get /acvp/v1/validations/41256', detailed as below.
My module supports multiple algorithms, (AES, TDES, HMAC, SHA, etc.). After having HMAC tested first, i ran the tests for AES by using the command './app/acvp_app --aes --fips_validation metadata/validation.json', then followed up the similar steps as i did on testing HMAC to request the algo cert for AES. During the tests, I received the different requestID, but got the same validationID. Eventually, I received same algo cert as I received from HMAC testing. It kind of made sense to me because both HMAC and AES were implemented by the same module, leading the same algo certificate to cover both algorithms. However, after giving the command './app/acvp_app --get /acvp/v1/validations/41256', the output just showed the detailed HMAC algorithm, no AES algorithm. Please see the details below.
Step 1: root@acvp1:~/richard-libacvp-demo/libacvp# ./app/acvp_app --aes --fips_validation metadata/validation.json
Response: "url" : "/acvp/v1/requests/1505",
Step 2: ./app/acvp_app --get /acvp/v1/requests/1505
Response:  "approvedUrl" : "/acvp/v1/validations/41256"
Step 3: ./app/acvp_app --get /acvp/v1/validations/41256
Response:
"acvVersion" : "1.0"
}, {
"url" : "/acvp/v1/validations/41256",
"validationId" : "A10079",
"productUrl" : "/acvp/v1/modules/11523",
"scenarios" : [ {
"operatingEnvironments" : [ {
"url" : "/acvp/v1/oes/21348",
"name" : "Ubuntu Linux 4.15.0-60-generic x86_64",
"dependencies" : [ {
"url" : "/acvp/v1/dependencies/23021",
"name" : "Linux 4.15.0",
"type" : "software",
"description" : "Ubuntu Linux Distribution 3.1"
} ]
} ],
"algorithmCapabilities" : [ {
"algorithm" : {
"name" : "HMAC-SHA2-512",
"mode" : null,
"revision" : "1.0"
},
"capabilities" : {
"keyLen" : [ {
"min" : 256,
"max" : 448,
"increment" : 8
} ],
"macLen" : [ {
"min" : 32,
"max" : 512,
"increment" : 8
} ]
},
"prerequisites" : [ ]
}, {
"algorithm" : {
"name" : "HMAC-SHA2-384",
"mode" : null,
"revision" : "1.0"
},
"capabilities" : {
"keyLen" : [ {
"min" : 256,
"max" : 448,
"increment" : 8
} ],
"macLen" : [ {
"min" : 32,
"max" : 384,
"increment" : 8
} ]
},
"prerequisites" : [ ]
}, {
"algorithm" : {
"name" : "HMAC-SHA2-256",
"mode" : null,
"revision" : "1.0"
},
"capabilities" : {
"keyLen" : [ {
"min" : 256,
"max" : 448,
"increment" : 8
} ],
"macLen" : [ {
"min" : 32,
"max" : 256,
"increment" : 8
} ]
},
"prerequisites" : [ ]
}, {
"algorithm" : {
"name" : "HMAC-SHA2-224",
"mode" : null,
Step 4, I confirmed that I received a new testSession_42818.json file after running AES test.
Question: Why does the output from the command './app/acvp_app --get /acvp/v1/validations/41256 just show the algorithm tested at the first place? In my case, the output just showed the HMAC, but no AES. How can I check all tested algorithm under the same validationID?
Thanks,
-Richard
Any thoughts?
Thanks,
-Richard

No command responses dump all the data unless you use --verbose.

It works.
Thanks Berry !
-Richard

Hello Berry,
One more question:
I submitted the testSession_42812.json file generated by libacvp to CAVP team and received the following feedback:
"The JSON we were asking for was what was sent to our API, not the libacvp client files.  In the future, for any troubleshooting to take place, we will need the actual JSON submitted to the API."
Question: Does libacvp Runtime application create such a JSON file used to communicate with ACVP server API?
Thanks,
-Richard

I don't have enough context to go on.
Did you have an issue you wanted them to help you with ?
If so, please describe it.
Every ACVP message sent from the libacvp client to the NIST server contains json.

Berry,
First of all, I don't have an issue that I wanted ACVP team to help with. The reason made a submission to AVCP was because I wanted to obtain the credentials for logging on their ACVP Prod Server.
Second, my understanding to ACVP's question is to find out a JSON file containing the following information:
***ACVP [STATUS][acvp_parse_login:1713]--> JWT: eyJhbGciOiJIUzI1NiJ9.eyJqdGkiOiJlMDM3ODE1MC1lOWQ2LTQwMmEtYTc5ZS0zNGY4ZWZhMjQ3NTgiLCJpYXQiOjE1NzU3Nzk5OTgsInN1YiI6IjEuMi44NDAuMTEzNTQ5LjEuOS4xPSMxNjE4NDU2NDRkNmY3MjcyNjk3MzQwNDc2ZjczNzM2MTZkNjU3MjUzNjU2MzJlNjM2ZjZkLENOPU5WTEFQIExhYiBDb2RlIDIwMDk5Ny0wLE9VPUNTVEwsTz1Hb3NzYW1lcixMPUNhdG9uc3ZpbGxlLFNUPU1hcnlsYW5kLEM9VVMiLCJpc3MiOiJOSVNUIEFDVlRTIiwiZXhwIjoxNTc1NzgxNzk4fQ.bHMiY8xjxqIpdzy5Ih-1Ocj_aEzY-bBIr1lGV01Jocw
***ACVP [STATUS][log_network_status:1362]--> GET...
Status: 200
Url: https://demo.acvts.nist.gov:443/acvp/v1/validations/41256
Resp:
[ {
"acvVersion" : "1.0"
}, {
"url" : "/acvp/v1/validations/41256",
"validationId" : "A10079",
"productUrl" : "/acvp/v1/modules/11523",
"scenarios" : [ {
"operatingEnvironments" : [ {
"url" : "/acvp/v1/oes/21348",
"name" : "Ubuntu Linux 4.15.0-60-generic x86_64",
"dependencies" : [ {
"url" : "/acvp/v1/dependencies/23021",
"name" : "Linux 4.15.0",
"type" : "software",
"description" : "Ubuntu Linux Distribution 3.1"
} ]
} ],
"algorithmCapabilities" : [ {
"algorithm" : {
"name" : "HMAC-SHA2-512",
"mode" : null,
"revision" : "1.0"
},
"capabilities" : {
"macLen" : [ {
"min" : 32,
"max" : 512,
"increment" : 8
} ],
"keyLen" : [ {
"min" : 256,
"max" : 448,
"increment" : 8
} ]
},
"prerequisites" : [ ]
}, {
"algorithm" : {
"name" : "HMAC-SHA2-384",
"mode" : null,
"revision" : "1.0"
},
"capabilities" : {
"macLen" : [ {
"min" : 32,
"max" : 384,
"increment" : 8
} ],
"keyLen" : [ {
"min" : 256,
"max" : 448,
"increment" : 8
} ]
},
"prerequisites" : [ ]
}, {
"algorithm" : {
"name" : "HMAC-SHA2-256",
"mode" : null,
"revision" : "1.0"
},
"capabilities" : {
"keyLen" : [ {
"min" : 256,
"max" : 448,
"increment" : 8
} ],
"macLen" : [ {
"min" : 32,
"max" : 256,
"increment" : 8
} ]
},
"prerequisites" : [ ]
}, {
"algorithm" : {
"name" : "HMAC-SHA2-224",
"mode" : null,
"revision
But, due to the use of libacvp runtime application, the test vectors request and response validation was done automatically. There might not a separate JSON file that was created during the testing.
If i am using libacvp non-runtime application, there could be existing such a JSON file used to talk with ACVP server API.
Do you agree?
Thanks,
-Richard

There is no difference between runtime and non-runtime with respect to any capabilities json file, we don't create one.  That's what you're showing in your validation output - the capabilities and registration.  If that's what they want you can put that output in a file and send it to CAVP.

Thanks Berry for the clarification.
-Richard

Hi Berry,
A quick question: There is a file named "creation.json" under "/libacvp/metadata". I reviewed the contents of this file and found out that it included the Vendor and Person information. During the algo testing, it looks like the user doesn't need to manipulate this creation.json file.
Question: When and where this creation.json file was created and used during the algo testing?
Thanks,
-Richard

I don't use it but it is part of the repo. It was not created as part of any operation.  There is a description in the metadata/README.md, however I'm not the person to help you with that part of the metadata generation.

That's fair enough.
Thanks Berry !
-Richard


Kubeshark worker running error prompt "no BTF found for kernel version 5.4.210-1.el7.elrepo.x86_64"

Describe the bug
Kubeshark workder pod can be runing, The front UI prompt error is "TypeError: Failed to fetch" after input filter parameter click apply
Provide more information
Running on  Rancher 3master 3 worker
To Reproduce
Steps to reproduce the behavior:

download kubeshark release binary
kubeshark tap --proxy-host <IP_ADDRESS>

Expected behavior
Front UI show data and don't have promp error
Logs
Upload logs:
ERR tracer/tracer.go:264 > stack="*fmt.wrapError field GoCryptoTlsAbi0Read: program go_crypto_tls_abi0_read: apply CO-RE relocations: no BTF found for kernel version 5.4.210-1.el7.elrepo.x86_64: not supported\n/app/tracer/tracer.go:66 (0x1ecec85)\n/app/worker.go:307 (0x1ef3528)\n/app/worker.go:53 (0x1ef15f2)\n/app/main.go:87 (0x1ef0f05)\n/app/main.go:62 (0x1ef0bf7)\n/usr/local/go/src/runtime/proc.go:255 (0x439027)\n/usr/local/go/src/runtime/asm_amd64.s:1581 (0x4683e1)\n"
Screenshots
Desktop (please complete the following information):

OS: Window10
Web Browser: Google Chrome

Additional context
Add any other context about the problem here.

@prometheus-tao the error log no BTF found for kernel version 5.4.210-1.el7.elrepo.x86_64 you see, is I think not related to the issue that you're experiencing. It's related to eBPF (TLS capture). It means that with your Linux kernel version, the TLS capture for Go language does not work. So, try to upgrade your kernel.
I guess you see a toast message TypeError: Failed to fetch when you apply a certain filter. What is the exact filter, could you please share it? What are the steps to reproduce this error? Can you share the browser console output? It can simply be a network error between your browser and the Hub.

@mertyildiran thanks, I will be try to upgrade linux kernel, The chrome browser to access kubeshark-front occured  console error prompt as follow, Kubeshark-front and kubeshark-hub two service port mapping by nodeport, Can't foud 8898 port on kuberenetes service.
[error log]
Failed to fetch
at EntriesList.tsx:78:5
at p (regeneratorRuntime.js:44:17)
at Generator. (regeneratorRuntime.js:125:22)
at Generator.next (regeneratorRuntime.js:69:21)
at r (asyncToGenerator.js:3:20)
at s (asyncToGenerator.js:22:9)
at asyncToGenerator.js:27:7
at new Promise ()
at Object. (asyncToGenerator.js:19:12)
at i (interval.ts:19:11)
(
TafficViewer.tsx:120 WebSocket connection to 'ws://xxx.xx.xxx.xx:8898/ws?session-token=&refresh-token=' failed:
e
triesList.tsx:78     GET http://xxx.xx.xxx.xx:8898/pcaps/total-size?refresh-token= net::ERR_CONNECTION_REFUSED
(anony

@mertyildiran I haven't changed anything in yaml, Nothing deploy kubeshark by helm, Starting command as follow.
kubeshark tap --proxy-host <IP_ADDRESS>
[kubeshark starting log]
2023-06-21T17:29:46+08:00 INF tapRunner.go:145 > Targeted pod: tcpdump-6d945cdfcc-7vzht
2023-06-21T17:29:46+08:00 INF tapRunner.go:145 > Targeted pod: web3-69f4995d9d-7mxhj
2023-06-21T17:29:46+08:00 INF tapRunner.go:145 > Targeted pod: web4-8588669475-phrbk
2023-06-21T17:29:46+08:00 INF tapRunner.go:92 > Waiting for the creation of Kubeshark resources...
2023-06-21T17:29:46+08:00 INF createResources.go:103 > Successfully created a pod. pod=kubeshark-hub
2023-06-21T17:29:46+08:00 INF createResources.go:115 > Successfully created a pod. pod=kubeshark-front
2023-06-21T17:29:46+08:00 INF createResources.go:65 > Successfully created a service. service=kubeshark-hub
2023-06-21T17:29:46+08:00 INF createResources.go:71 > Successfully created a service. service=kubeshark-front
2023-06-21T17:29:46+08:00 INF tapRunner.go:176 > Added: pod=kubeshark-hub
2023-06-21T17:29:46+08:00 INF tapRunner.go:267 > Added: pod=kubeshark-front
2023-06-21T17:30:25+08:00 INF proxy.go:29 > Starting proxy... namespace=kubeshark service=kubeshark-hub src-port=8898
2023-06-21T17:30:30+08:00 INF proxy.go:29 > Starting proxy... namespace=kubeshark service=kubeshark-front src-port=8899
2023-06-21T17:30:41+08:00 WRN common.go:34 > Couldn't connect using proxy, stopping proxy and trying to create port-forward... service=kubeshark-hub
2023-06-21T17:30:41+08:00 INF proxy.go:117 > Starting proxy using port-forward method... dst-port=80 namespace=kubeshark pod=kubeshark-hub src-port=8898
2023-06-21T17:30:45+08:00 WRN common.go:34 > Couldn't connect using proxy, stopping proxy and trying to create port-forward... service=kubeshark-front
2023-06-21T17:30:46+08:00 INF proxy.go:117 > Starting proxy using port-forward method... dst-port=80 namespace=kubeshark pod=kubeshark-front src-port=8899
2023-06-21T17:30:46+08:00 INF workers.go:49 > Creating the worker DaemonSet...
2023-06-21T17:30:46+08:00 INF workers.go:68 > Successfully created the worker DaemonSet.
2023-06-21T17:30:46+08:00 INF tapRunner.go:466 > Hub is available at: url=http://localhost:8898
2023-06-21T17:30:51+08:00 INF tapRunner.go:492 > Kubeshark is available at: url=http://localhost:8899