[BUG] Handle hallucinated tool name

Describe the bug

Sometimes I get NPE because toolExecutor is NULL.
Log and Stack trace

java.lang.NullPointerException: Cannot invoke "dev.langchain4j.agent.tool.ToolExecutor.execute(dev.langchain4j.agent.tool.ToolExecutionRequest, Object)" because "toolExecutor" is null
	at dev.langchain4j.service.DefaultAiServices$1.invoke(DefaultAiServices.java:155)

To Reproduce

I asked my chatbot directly like: "Calculate document link using the documentId".
I see (in the debugger) that it knows documentId value from the context window.
Expected behavior

The corresponding function gets called
Current behaviour
NPE in the DefaultAIServices class.
Please complete the following information:

LangChain4j version: 0.30.0
LLM(s) used: Azure OpenAI (with gpt-3.5-turbo)
Java version: 21
Spring Boot version (if applicable): 3.2.5

Additional context

This is where it breaks:

Please NOTE that the toolExecutionRequest contains a strange name "python". At the same time, it knows the documentId.

My tool is defined as follows:
    @Component
    static class Tools {

        @Tool("Returns document link")
        String getDocumentLink(@P("documentId") String documentId) {
            return "https://<private-address>/%s".formatted(documentId);
        }
    }

@vprudnikov thanks a lot for reporting!
It seems that LLM hallucinated the tool name (python), do you have a tool with such name? It seems that it provided a code to interpret?
In any case, we should handle such cases better. What would be the desired behavior in such case?

Fail-fast (similar to what is happening now, bad UX)
Retry the same request (might not help if temperature is low)
Reply to LLM with a message telling that there is no such tool?
Something else?
Should the strategy be configurable?

@langchain4j , it's a Spring Boot app with only one tool defined (please find it in the original message).

the temperature is 0.3  Retrying the same request doesn't give me the link
Reply to LLM with a message telling that there is no such tool? - this can be hard to achieve since end users are not aware of such specifics :)

Reply to LLM with a message telling that there is no such tool? - this can be hard to achieve since end users are not aware of such specifics :)

I mean an automatic handling, users should not know about this.
In the case when LLM wants to execute a tool which does not exist, we could automatically (in DefaultAiServices) reply to the LLM that there is no such tool, optionally adding a list of available tools (if this will improve the results). And see if it corrects itself or not. If yes, AiMessage with wrong tool execution request and automatic retry message can be then automatically removed from the ChatMemory to save tokens.

Another option would be to retry using a better LLM (e.g. gpt-4-turbo instead of gpt-3.5-turbo), if current model fails to provide a valid tool execution request

@langchain4j , I think it would be a very good idea to handle this issue internally in the DefaultAiServices since nobody expects to get NPE thrown by the library.
I'm actually using the gpt-3.5-turbo (I'll fix it in the report).
Btw, I tried to force it to execute that function, and it answers like "to get the document link, please execute functions.getDocumentLink({documentId})" :)

@vprudnikov did you tell it that there is no function/tool with such name?

@langchain4j , I did, but we go in circles. It apologies and says that it found a documented with a certain value. When I ask it directly "use the "getDocumentLink" tool" it fails again with NPE.
When I say "use the "getDocumentLink" tool, not "python"", it says "sorry, but I don't have access to the tool "getDocumentLink""...

@vprudnikov hmm, do you see getDocumentLink tool in the request to the LLM? You can enable detailed logs.
Are you using demo key?

@langchain4j the getDocumentLink tool perfectly works in other use cases. I just found one original question that leads to hallucinations. I assume the context confuses the LLM. However, I don't understand why it doesn't pick the tool when I explicitly say to do that.
So usually, I get the document link calculated correctly.
However, sometimes, I get the following response, and the tool is not used at all:
"Yes, you must register hours in Tempo in Jira. This is evident from "Instructions for timekeeping regarding IT tasks" with documentId 1010 and document link as follows: functions.getDocumentLink({documentId: "1010"})."

@langchain4j it looks like I managed to get a more consistent behaviour.
I removed ALL links from the "contents", and now the tool is never used :)
This is my question:
Answer based on this context:
                                 {{contents}}
                                 My question: {{userMessage}}
                                 Provide answer, documentName and document link.

Now. the tool is never called

@vprudnikov if the tool is always present in the request to the LLM, but LLM hallucinates, I guess we can't do much here at this point.
Either use a better model or try to add few-shot examples (currently manually, will be automated once this is implemented).

@langchain4j , OK, I see, thank you. Do you have any recommendations regarding the prompt? Maybe some adjustments may help?

Few-shot examples should help, but you will have to use lower-level APIs cause AI Services don't support it out of the box yet

@algotechnology what is your use case? What LLM are you using?

I'm hitting similar issue with Ollama (0.5.1) and llama3.2:3b which hallucinate a tool name.
@langchain4j what would be the correct behavior in this case ? returning nothing to the LLM ?

Hi @ehsavoie, I would argue that this should be handled on the LLM API side (in this case by Ollama/llamacpp). After all, we provide the exact schema for the tool. How hard would it be to validate the generated tool name in the response against tool names in the requests before sending the response?
But since we are not living in the ideal world, I guess we could/should implement a configurable retry/reprompt/custom strategy to make it more robust. We discussed it with @ddobrin some time ago, but I did not have time yet to address it.
Would you like to take a lead here and implement something? 🙏

Hi I've got the same hallucination with Ollama and the llama3.3 model. See https://github.com/spring-petclinic/spring-petclinic-langchain4j/issues/10
Do you know if an issue has been created for the Ollama team?
Waiting for a more advanced improvement, instead of a NullPointerException, LangChain4j can return a more friendly error like The function name %s does not exist.
Providing a way to customise error handling might also be a good idea.


Committing to a branch or creating a branch from a commit doesn't update the dangling references.

When calling:
public TCommit commitToBranch(TArea contentAreaToCommit, String branchName, String message, TRepo repo, ByteArrayIndex byteArrayIndex, Clock<? extends Timestamp> clock)
or
public void createBranchAtCommit(TCommit commit, String branchName, TRepo repo)
we don't remove the commit from dangling branches.
There might be other edge cases that don't do the same.
Also picked up an issue with MemoryRepoAPI Java docs that don't describe the dangling commits correctly.
    /**      * The references to the commits that have been tagged.      * The key is the tag name.      * The value is the commit that has been tagged.      * @return The references to the commits that have been tagged.      */     LinkedHashSet<TCommit> getDanglingCommits();

Fixed the logic to correctly update the dangling references when creating commits and tags.
Added an API call to remove a branch.
Now we correctly update the dangling references when removing a branch and tag.


WIP: Event based webhooks

This is a work-in-progress PR to add support for arbitrary third-party integrations via webhooks, modeled after Github Webhooks.
Users will be able to specify URL that OneFuzz will call and a set of events that will hit said URL.
Currently, the event types supported are:

task_created (sends a WebhookEventTaskCreated)
task_stopped (sends a WebhookEventTaskStopped)
task_failed (sends a WebhookEventTaskFailed)

The webhooks will atempt up to 5 times, then be marked as failed.
TODO:

[ ] Add support for onefuzz webhooks get to include the WebhookMessageLog entries for said webhook
[ ] Add support to clean out old events from the WebhookMessageLog
[x] Add support for a 'ping' endpoint, to verify the webhooks are setup correctly
[x] Add a tool that automatically builds documentation for how the webhooks are laid out.

Until this is merged, this link should work to view the generated webhook documentation.
https://github.com/bmc-msft/onefuzz/blob/event-based-webhooks/docs/webhook_events.md


Update restore-project.md

To restore a project, Project Collection Administrator permission

@AlexandraRamirezMS : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@ktoliver : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Update Kotlin and AGP versions

Update Kotlin lang version to 1.8 - https://kotlinlang.org/docs/whatsnew18.html
Update AGP version to 7.4.0 - https://developer.android.com/studio/releases/gradle-plugin

implemented and merged


Added Cursor change in simulator

Fixes #1088
Describe the changes you have made in this PR -
Added Cursor changes to a pointer when choosing circuit elements
Cursor changes to the pointer on the canvas when hovered on an element
Cursor changes to grabbing when the mouse button is clicked
Screenshots of the changes (If any) -

Coverage remained the same at 84.266% when pulling 77c3d844b5ee8b43e2e42217e90672a3ac5900da on darkshredder:patch-8 into 610bd90d88f18dc48f05402bf6f3bd84700335cd on CircuitVerse:master.

@tachyons  @nitin10s  @aayush-05  @satu0king, please review

Fixed in #966


Creating dataset from large numpy arrays via from_tensor_slices crashes without any error message or warning

I use tf 2.6 and when I try to create datasets from larger numpy arrays (>10GB) via from_tensor_slices the code breaks when I try to train via "fit" or even just attempt to iterate over the dataset. The code just breaks and exists, no warning, no error message, nothing. I have not found any similar issue mentioned anywhere else. What is the actual limitation here?
I have over 128GB ram and due to the code breaking already on the dataset iteration part it is surely unrelated to my GPU and its memory (24GB). The numpy arrays load without issues into memory but once the iterator causes the execution of "from_tensor_slices", the code breaks shortly after.
What are workable solutions here? DataGenerators? Creating TFRecord files? I try to avoid going the route of TFRecords because it appears very poorly documented how to create such binary files.
As no warning or error message is output there is no log to show at all.
The problem can be reproduced with a large numpy array (random data works) and the following code:
#obtain training data
print("loading training data...")
train_data = np.load(os.path.join(os.getcwd(), "source_datasets", f"{train_data_id}_features.npy"))
train_targets = np.load(os.path.join(os.getcwd(), "source_datasets", f"{train_data_id}_targets.npy"))
print("training dataset construction...")
train_ds = tf.data.Dataset.from_tensor_slices((train_data, train_targets))

for x,y in train_ds:
    do_something

Hi, Try iterating your dataset using this :
for x,y in train_ds.take(1):
    do_something

and check if it is able to print out elements from the first list of train_ds
Also, try defining a batch size for your train dataset here, using this:
train_ds = train_ds.batch(BATCH_SIZE)
I recommend using a smaller BATCH SIZE first.
The above suggestions might not be enough to fix your problem.Even with a a large RAM, loading a data this big directly into the memory is not very efficient.It would be better to load just a chunk of the file directly into memory rather loading all of them at once.

Thanks for your suggestions, unfortunately those ideas did not solve the issue of crashing due to large size of numpy arrays even though there are no internal memory limitations. I decided to ditch from_tensor_slices and went with generators which seems to work just fine.


Missing pdfs or other info about the papers

Many times authors might not provide a camera ready paper and this can raise some issues, e.g. this error: "pdf_file": submission.content["pdf"].split("/")[-1], KeyError: 'pdf.
As I had to solve this quickly, please find attached the following,  it might be useful:
or2papers.py

openreview.py

Thank you @MirunaClinciu!


Add server security alerts list by server

MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Changelog
Please ensure to add changelog with this PR by answering the following questions.

What's the purpose of the update?

[ ] new service onboarding
[ ] new API version
[ ] update existing version for new feature
[x] update existing version to fix swagger quality issue in s360
[ ] Other, please clarify

When you are targeting to deploy new service/feature to public regions? Please provide date, or month to public if date is not available yet.
When you expect to publish swagger? Please provide date, or month to public if date is not available yet.
If it's an update to existing version,  please select SDKs of specific language and CLIs that require refresh after swagger is published.

[ ] SDK of .NET (need service team to ensure code readiness)
[ ] SDK of Python
[ ] SDK of Java
[ ] SDK of Js
[ ] SDK of Go
[ ] PowerShell
[ ] CLI
[ ] Terraform
[ ] No, no need to refresh for updates in this PR

Contribution checklist:

[x] I commit to follow the Breaking Change Policy of “no breaking changes
[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

[ ] Ensure to check this box if one of the following scenarios meet updates in the PR, so that label “WaitForARMFeedback” will be added automatically to involve ARM API Review. Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs, all “removals” and “adding a new property” no more require ARM API review.

Adding new API(s)
Adding a new API version
Adding a new service

[ ] Please ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If there are following updates in the PR, ensure to request an approval from API Review Board as defined in the Breaking Change Policy.

[ ] Removing API(s) in stable version
[ ] Removing properties in stable version
[ ] Removing API version(s) in stable version
[ ] Updating API in stable version with Breaking Change Validation errors
[ ] Updating API(s) in preview over 1 year

Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.
Please follow the link to find more details on PR review process.

/azp run

/azp run

/azp run

/azp run

Approved breaking changes since it is fixing a S360 issue.

Approved breaking changes since it is fixing a S360 issue.

/azp run

/azp run

@ayeletshpigelman Can you confirm that this is a swagger bug fix for an existing API rather than introducing a new API?
@ArcturusZhang What's the breaking change status. I see you added the approved labels and then removed them.

@ayeletshpigelman Can you confirm that this is a swagger bug fix for an existing API rather than introducing a new API?
@ArcturusZhang What's the breaking change status. I see you added the approved labels and then removed them.

@ayeletshpigelman Can you confirm that this is a swagger bug fix for an existing API rather than introducing a new API?
@ArcturusZhang What's the breaking change status? I see you added the approved labels and then removed them.

Sorry about the Approved-BreakingChange because initially I misunderstood the policy and it turns out even for S360 fixings we still need the breaking change review board's approval. Therefore please ignore my approval on the breaking change.

@ayeletshpigelman Can you confirm that this is a swagger bug fix for an existing API rather than introducing a new API?
@ArcturusZhang What's the breaking change status? I see you added the approved labels and then removed them.

Sorry about the Approved-BreakingChange because initially I misunderstood the policy and it turns out even for S360 fixings we still need the breaking change review board's approval. Therefore please ignore my approval on the breaking change.

Thanks for explaining. @ayeletshpigelman please follow the breaking change instructions in one of the openapi bot comments above and obtain breaking change approval.

Thanks for explaining. @ayeletshpigelman please follow the breaking change instructions in one of the openapi bot comments above and obtain breaking change approval.

@ArcturusZhang and @majastrz this is a swagger bug fix for an existing API

@ArcturusZhang and @majastrz this is a swagger bug fix for an existing API

I created this request to approve the breaking change: https://msazure.visualstudio.com/One/_workitems/edit/9095017

I created this request to approve the breaking change: https://msazure.visualstudio.com/One/_workitems/edit/9095017

LGTM one comment I have and for future swaggers is that for singletons, include /default as part of the URI instead of modeling it as an enum with one value ("default") in it.

LGTM one comment I have and for future swaggers is that for singletons, include /default as part of the URI instead of modeling it as an enum with one value ("default") in it.

/azp run

/azp run


Replace renderOneFrame for per-workspace update calls

Current behavior
Currently Gazebo is doing, for every camera (of any type):
this->dataPtr->ogreCompositorWorkspace->setEnabled(true);
auto engine = Ogre2RenderEngine::Instance();
engine->OgreRoot()->renderOneFrame();
this->dataPtr->ogreCompositorWorkspace->setEnabled(false);

This is inefficient. Because:

Ogre::SceneManager::updateSceneGraph can be called once before all cameras (unless objects are moved or added/deleted in between)
We force a GPU flush per camera, instead of flushing after all cameras (some exceptions apply)

Desired behavior
The code can be replaced with the following:
this->dataPtr->ogreCompositorWorkspace->_validateFinalTarget();
this->dataPtr->ogreCompositorWorkspace->_beginUpdate(false);
this->dataPtr->ogreCompositorWorkspace->_update();
this->dataPtr->ogreCompositorWorkspace->_endUpdate(false);

Ogre::vector<Ogre::RenderTarget*>::type swappedTargets;
swappedTargets.reserve( 2u );
this->dataPtr->ogreCompositorWorkspace->_swapFinalTarget( swappedTargets );

While Ogre::SceneManager::updateSceneGraph gets moved to Scene::PreRender and a GPU flush is moved to Scene::PostRenderGpuFlush (PostRenderGpuFlush currently doesn't exist)
Exceptions
Scene::PostRenderGpuFlush Flushes all buffered frame data and starts 'a new frame'.

If we flush too often, the CPU will often have to wait for the GPU doing nothing.
If we flush too infrequently, RAM consumption will rise due to queueing up unsubmitted work.
This function should be called after updating all cameras / once per frame, but it may be called more often if you're updating too many cameras and running out of memory.
Example:
Cubemap rendering w/ 3 probes and 5 shadowmaps can cause
a blow up of passes:

(5 shadow maps per face + 1 regular render) x 6 faces x 3 probes =
108 render_scene passes.

108 is way too much, causing out of memory situations;
so flushing once per probe may make better sense in this case.
Note however that Gazebo is currently flushing once per cubemap face (i.e. flushing 6x3 = 18 times). While with my proposal (once per cubemap) we would flush 3 times.
Reason
This is merely a performance improvement.
This paves the way for an additional improvement. For example currently Depth Sensors are doing the following:
for every camera
  camera->Render();
  camera->PostRender(); // Downloads depth data

This negates our optimization proposed in this ticket because PostRender will force the CPU to wait for the GPU. The render loop can be improved to the following:
while( cameras.empty() )
  // Queue up some commands to GPU (N is arbitrary and user controlled)
  for N cameras
    camera[i]->Render();
    cameras.pop();
  // Dispatch the commands to GPU
  scene->PostRenderGpuFlush();

// Collect the depth data to CPU. By now some data should already be on
// CPU memory so the first PostRender calls will be quicker than the last
// ones
for every camera
  camera->PostRender();

This allows the CPU to do more work while the GPU is rendering the previous cameras in parallel
N is an arbitrary number, where it depends on available memory and scene type (large values of N queue up too much, low values cause too much stall).
Impact
This is an ABI and API breaking change.
Other components need to change so they also call PostRenderGpuFlush (usually looking for calls to Scene::PreRender is a good start).
e.g. ign-sensors needs to add a call to PostRenderGpuFlush
Users must call PostRenderGpuFlush at some point or else errors or out of memory conditions may occur.
Because of this, a boolean will be added for migrating users. This boolean will control whether the old behavior or the new behavior is used.
In the old behavior, Gazebo will issue one updateSceneGraph and one PostRenderGpuFlush per Camera (default) which is what Gazebo is currently doing.
In the new behavior, Gazebo assumes the user properly calls PreRender and PostRenderGpuFlush when appropriate
Samples need to be updated to reflect this change
Risks

As with the Sensors example, the road to optimum performance is one where no stalls occur. Ideally CPU and GPU are always busy. However a single sync operation that forces a stall (usually GPU -> CPU communication) can heavily negate the benefits of this ticket. This needs to be constantly monitored and profiled. Nonetheless Gazebo's behavior can be improved.
Failing to call updateSceneGraph after scene has changed between cameras will result in incorrect rendering (e.g. missing lights, objects in the wrong location) or a crash (e.g. if lights are deleted). Ogre has asserts in place to detect this problem; which is why I also proposed Gazebo includes a 'validation' where Ogre is run using a Debug build. Note that currently Gazebo is sometimes failing these asserts (that's for another ticket) but don't seem high risk (it seems to be related to ray queries used by mouse picking). Please also note if a call to updateSceneGraph is required, then it is also extremely likely that a call to Scene::PreRender is also needed (to perform internal Gazebo maintenance). Which would mean Ogre's assert would just be detecting incorrect gazebo usage

Implementation suggestion
I'm the one implementing this feature. This ticket is for tracking progress.
Progress

[x] Replace most renderOneFrame calls for per workspace calls
[x] Add PostRenderGpuFlush
[x] Add boolean setting to toggle behavior
[x] Document that Camera::Update is a utility function that shouldn't be called if you have many cameras
[x] Replace remaining renderOneFrame calls for per workspace calls
[x] Test in every Render() call that we're inside PreRender / PostRender
[x] Document SetCameraPassCountPerGpuFlush is an upper bound
[x] Rename functions for consistency
[ ] Modify samples to use new behavior
[x] Document scene should not be altered (e.g. creating/destroying lights) between Scene Pre/PostRender calls
[ ] Try to call PostRender as a fallback mechanism, but severely warn the user. Should we use caps locks? Warn every time (not just once)?

Progress can be tracked at:

https://github.com/darksylinc/ign-rendering/tree/matias-PostFrame5
https://github.com/darksylinc/ign-gazebo/tree/matias-PostFrame5
https://github.com/darksylinc/ign-sensors/tree/matias-PostFrame5

Blocking problems:
With cameraPassCountPerGpuFlush = 6

[x]  69 - INTEGRATION_depth_camera (Failed)
[x]  71 - INTEGRATION_camera (SEGFAULT)
[x]  72 - check_INTEGRATION_camera (Failed)
[x]  73 - INTEGRATION_render_pass (Failed)

With cameraPassCountPerGpuFlush = 0

[x]  69 - INTEGRATION_depth_camera (Failed) - It's #332
[x]  73 - INTEGRATION_render_pass (Failed) - It's just #315

@iche033 Quick question. Camera::Update says:
/// \brief Renders a new frame.
/// This is a convenience function for single-camera scenes. It wraps the
/// pre-render, render, and post-render into a single
/// function. This should be used in applications with multiple cameras
/// or multiple consumers of a single camera's images.
public: virtual void Update() = 0;

I believe it should say "This shouldn't be used in applications with multiple cameras" ?

Figured out a more user friendly solution to efficient loops of cameras i.e. the while( cameras.empty() ) thing:
All Cameras should call FlushSomething() (name pending) inside Camera::PostRender for every camera. This happens inside Gazebo so the user doesn't have to change its update loops.
However the Flush function will only flush if it gets called every N times.
The user would still have to call Scene::PostRenderGpuFlush at the end of everything, but aside from that, the user can  call:
Scene::SetNumCameraPassesPerFlush( uint value ) where value must be >= 0
A value of 0 means we flush every camera (old behavior) and no need to call Scene::PostRenderGpuFlush at the end of everything.
A value > 1 is new behavior, and user must call Scene::PostRenderGpuFlush at the end of everything

I believe it should say "This shouldn't be used in applications with multiple cameras" ?

yes :)

Figured out a more user friendly solution to efficient loops of cameras i.e. the while( cameras.empty() ) thing:

sounds good! I'm assuming the idea is to keep the value at 0 as default behavior and let advanced users call SetNumCameraPassesPerFlush with value > 0?

sounds good! I'm assuming the idea is to keep the value at 0 as default behavior and let advanced users call SetNumCameraPassesPerFlush with value > 0?

The idea is that default is 0 yes.
Though also my idea is that most samples show this value not being set to 0 to show to how to use, and new users (rather than just advanced once) set this to non-0; since it's not really that advanced.
This value being 0 is to ease porting of existing users, where failing to call PostRenderGpuFlush results in leaks or other weird behavior (and the complexity of existing codebases is unknown but presumably large).

Update
With my recent changes in my branch, GpuRays/GpuRaysTest.RaysParticles test was failing so I took a deeper look to fix it.
Turns out this change will fix a subtle bug caused by Ogre2GpuRays; therefore this fix is no longer merely a "performance improvement" and now it actually fixes a (subtle) bug:

Every cube face is rendered with renderOneFrame
As per Ogre, calling renderOneFrame means 'this is a new frame'
Particle FXs will therefore advance their simulation forward in time with each call
As a result, each face will render the particles in a different time state! This is hard to notice in simple scenes if the scene took very little time to render

Technically the problem is global e.g. rendering to a two regular cameras and then to all 6 faces of a Ogre2GpuRays means the particles can be in up to 8 different momentum instead of 1.

I have a question for you @iche033 about the default value of SetNumCameraPassesPerGpuFlush
I made a couple changes to the proposed ones, which is less intrusive than what was originally proposed.
The current changes are like the following:

Added Scene::SetNumCameraPassesPerGpuFlush. A value of 0 is the legacy (i.e. current) behavior. A value >= 1 controls memory vs performance tradeoffs
When SetNumCameraPassesPerGpuFlush > 0, Users must call the new Scene::PostRender function.
If Scene::PreRender is called twice in a row without calling Scene::PostRender, or PostRender gets called twice in a row, Ignition will assert, which immediately notifies the user there's something wrong that needs their attention.

Given that we can warn the user what's wrong (i.e. they didn't update their code to call Scene::PostRender), it may be better to set a default value for SetNumCameraPassesPerGpuFlush somewhere between [1; 6] rather than defaulting to legacy value of 0
What do you think?

The changes sound good to me. We should probably target these changes to main since it's a behavior / API usage change, and also update the Migration guide.

If Scene::PreRender is called twice in a row without calling Scene::PostRender, or PostRender gets called twice in a row without PreRender, Ignition will assert,

I am thinking of a way to ease transition for users. Instead of an assert, we could print out an error msg like you're already doing,  and in addition, we either call PostRender for them (if PreRender is called twice) or ignore the duplicate PostRender call (if PostRender is called twice)? For the Camera::Update function, we could just call PostRender for them - I think you're already doing that in your branch.

Given that we can warn the user what's wrong (i.e. they didn't update their code to call Scene::PostRender), it may be better to set a default value for SetNumCameraPassesPerGpuFlush somewhere between [1; 6] rather than defaulting to legacy value of 0

yeah I think that's fine. How about a default value to 1?  For users wanting to use more cameras, they'll then need to change this value. We'll likely need a small tutorial teaching users how to use these APIs later
Some other minor comments on APIs:

SetNumCameraPassesPerGpuFlush -> SetCameraPassCountPerGpuFlush. To be consistent with Ignition naming convention, we use Count in place of Num, e.g. here
GetLegacyAutoGpuFlush -> LegacyAutoGpuFlush. Ignition prefers leaving out the Get prefix in function name

I am thinking of a way to ease transition for users. Instead of an assert, we could print out an error msg like you're already doing, and in addition, we either call PostRender for them (if PreRender is called twice) or ignore the duplicate PostRender call (if PostRender is called twice)? For the Camera::Update function, we could just call PostRender for them - I think you're already doing that in your branch.

We can do that, but issuing some Ogre commands before PostRender (e.g. creating/destroying a light between PreRender/PostRender is a no-go*) might cause trouble, hence the message must put emphasis in the importance of doing it.

I should probably mention that in the doc.

yeah I think that's fine. How about a default value to 1? For users wanting to use more cameras, they'll then need to change this value. We'll likely need a small tutorial teaching users how to use these APIs later

I should clarify this better in the docs: If you set a value of e.g. 6, but you have a single camera and call PostRender, this will force a flush and hence having a value of 1 or 6 won't matter as the result will be exactly the same (in every term: performance, memory consumption).
A value of 6 is like an upper bound. We may queue up to 6 render passes or less; but never more.
But that does not mean a value of 6 unnecessarily wastes 6x more RAM than a value of 1 if you only have one regular camera.

Some other minor comments on APIs:
* `SetNumCameraPassesPerGpuFlush` -> `SetCameraPassCountPerGpuFlush`. To be consistent with Ignition naming convention, we use `Count` in place of `Num`, e.g. [here](https://github.com/ignitionrobotics/ign-rendering/blob/ign-rendering5/include/ignition/rendering/Scene.hh#L188)

* `GetLegacyAutoGpuFlush` -> `LegacyAutoGpuFlush`. Ignition prefers leaving out the `Get` prefix in function name

Got it.
TL;DR changes for me to implement

[ ] Document scene should not be altered (e.g. creating/destroying lights) between Scene Pre/PostRender calls
[ ] Document SetCameraPassCountPerGpuFlush is an upper bound
[ ] Try to call PostRender as a fallback mechanism, but severely warn the user. Should we use caps locks? Warn every time (not just once)?
[ ] Rename functions for consistency

I should clarify this better in the docs: If you set a value of e.g. 6, but you have a single camera and call PostRender, this will force a flush and hence having a value of 1 or 6 won't matter as the result will be exactly the same (in every term: performance, memory consumption).

I see, thanks for the clarification, that makes sense.

OK!
So I did extensive testing(*) and it looks stable and working. Which surprised me.
It looks ready or almost ready to start merging.
But I do have a couple of questions on the merge:

Right now it's based off ign-rendering5. But it obviously goes should be merged into main branch. Do I create the PR to be merged into main? Or do I first merge my branch into main and submit the PR? This obviously means that all changes done so far to ign-rendering5 (including those unrelated with this feature) will be merged into main.

The same question goes for ign-gazebo and ign-sensors

The current default is SetCameraPassCountPerGpuFlush = 6 instead of 0 (legacy). Which is not what was proposed. The rationale is that:

The final method ended up being less invasive than I thought. All samples that I could test (more on this later) did not need modifications at all. They 'just worked' because they were using Camera::Update or Camera::Capture which handles all these details
When the rules are violated (e.g. Camera::Render was called outside PreRender/PostRender calls, PreRender was called twice in a row, etc) we inform the user via asserts and the fix is extremely easy: Either add the missing calls, move them around, or SetCameraPassCountPerGpuFlush to 0. What goes wrong should be easy to identify and fix.
I tried looking into 'recovering', i.e. instead of asserting to do the cleanup ourselves; but it just won't work. Doing this can either work fine.... or mask serious bugs, from subtle ones like performance problems or hard-to-diagnose crashes (because it'd crash deep inside Ogre and the user will be clueless on what went wrong). So it's better to just stop the whole engine via assert and report what went wrong and how it needs to be fixed.

I tested all samples except these ones:

custom_shaders_uniforms (crashes in Ogre2, seems to be known issue)
gazebo_scene_viewer (needs 'gazebo'). There's a higher chance this one needs updating. I'd need help on how to set it up (because it couldn't find 'gazebo'. It looks like it depends on the old gazebo engine instead of ign-gazebo?)
heightmap (known issue: not implemented)
text_geom (known issue: not implemented)

Thus probably gazebo_scene_viewer demo is the only one I should be able to get working to see if it works, and then the code should be merged to main (repos to be merged are ign-rendering, -gazebo, -sensors)
(*)which of course means another user will encounter an obvious-unmissable problem the second they try it out

yes you can target main branch and then we don't need to worry about ABI changes. Let me merge ign-rendering5 forward to main first. Then you should be able to just merge your branch with main and create the PR.  That way, there won't be able unrelated changes in your PR. For other repos, I can also do the same but it can take some time depending on the changes that need to be merged forward. Alternatively, you can rebase your commits on top of main and the create the PR so you don't have to wait for the merge forward.

ok lets give SetCameraPassCountPerGpuFlush =  6 a try.
i. yes sounds about right. That makes me think that once the changes are merged in, we should probably have a small sample demo showing this new API usage.
ii. and iii. Ok I'll take a look at the code when the PR is up, and test it with ign-gazebo.

for the samples:
i. yes, ogre2 doesn't work yet
ii. there is an issue with the gazebo_scene_viewer sample on ign-rendering5: https://github.com/ignitionrobotics/ign-rendering/issues/290. I think that needs to be fixed first before we can test your changes. I would hold off on it for now. Maybe comment on the ticket and add a reminder to port the changes to the sample once its fixed.
iii. yes, ogre2 does not work yet
iii. yes, ogre2 does not work yet

Great!
I probably will have then to merge with SetCameraPassCountPerGpuFlush = 0 as default at least until ign-gazebo and ign-sensors get updated so that these modules are PostFrame-aware

Let me merge ign-rendering5 forward to main first

https://github.com/ignitionrobotics/ign-rendering/pull/350

@iche033 is this fixed?

yes we can consider this as complete.


hobgoblin: .html() output for  differs for shallow() vs. render()

Testing a minor component that renders only

.  I was attempting to evaluate testing with outputs from shallow() vs. render() when I found differences in the results from .html().
The shallow renderer's shallowWrapper.html() returns

The static renderer's  renderWrapper.html() returns

Is static rendering with cheerio causing the strange "how do we represent self-closing elements" differences?
Is there a mention in the docs that nothing is guaranteed about outputs from .html() and .text()?
Is there a stated/doced bias for which renderer to prefer?
Should I tell my consistency hobgoblin to go join the troll under the nearest bridge?

This is a strange difference. I suspect that shallow's .html() method will more directly represent reality, since that's just using React.renderToStaticMarkup internally.  The renderWrapper on the other hand is taking the result of renderToStaticMarkup, parsing it, then outputting a new "manufactured" string (is my guess, at least).

Has anybody given it a try? Maybe I could try it

Actually, render() and mount() both return

. Can I take a look, @dmnkgrc?

Can I give this a go? What should we fix first? Can anyone give me some insight on the problem? Thanks in advance!

I think the shallow renderer output more closely matches what people type in jsx - so i'd hope that all three would be able to use XHTML syntax.

Hello guys
I would love to help on this one. I was thinking of two solutions:

Regex the outerHTML return with the list of self closing tags to modify the result with  />
Use or create an equivalent of [XMLSerializer serializeToString](https://developer.mozilla.org/fr/docs/XMLSerializer method) to use in html() function.

I did some investigation on this and here's what I found:
The code path that is followed from ReactWrapper's html() function leads to effectively this to generate the content:
const domNode = ReactDOM.findDOMNode(node.instance);
return domNode.outerHTML;

(There is more than that but I am simplifying for explanation)
So the issue can actually be observed directly over in React (nothing to do with Enzyme).  And I believe the reason React renders in this fashion is because a
 tag is a "void element", which according to the spec does not require closing nor self-closing:
http://w3c.github.io/html/syntax.html#void-elements
So this may be an intentional thing from React's perspective.
@ljharb If my assessment above is accurate, I have questions around what you would expect to see here:

Would you still want Enzyme to return self-closing tags in this scenario?
Should it apply to all tags regardless of what kind they are? (void element, non-void element, etc)

@benlakey i think it would be reasonable for enzyme to normalize self-closing tags, and I also think it would be reasonable to leave it as-is.


Check for invalid digits when parsing octal constants

After introducing the pull request (#151 ), I use the following code to test the parser:
/* test.c */
int main() {
    int a = 03, b = 01118, c = 091;
    printf("%d %d %d\n", a, b, c);
    return 0;
}

$ qemu-arm out/shecc-stage2.elf -o test test.c
$ qemu-arm test
3 592 73

Obviously, the octal numbers (01118 and 091) are invalid but the parser doesn't check whether they are valid or not.
Thus, the pull request improves the parser to enhance the validation check.

Please also add your code to the testsuite for code validation.

The proposed changes let shecc generate a compilation error message when parsing invalid octal numbers.
However, originally, tests/driver.sh only compiles valid C programs and checks the exit code or standard output, and the test suite has never compiled any invalid C program before.
This is the first instance of adding an invalid C program, which includes some invalid octal constants, to  test/driver.sh to check whether (native) shecc generates the error message or not.
Thus, I will add a new function called try_compile_error() to the test suite, and the function will accept an invalid C program and then check if the exit code returned by shecc is a non-zero value. Eventually, the test suite will have the ability to test invalid C programs.
function try_compile_error() {
    local input=$(cat)

    local tmp_in="$(mktemp --suffix .c)"
    local tmp_exe="$(mktemp)"
    echo "$input" > "$tmp_in"
    "$SHECC" -o "$tmp_exe" "$tmp_in"
    local exit_code=$?

    if [ 0 == $exit_code ]; then
        echo "Error: compilation passed"
        exit 1
    fi
}

# ...

try_compile_error << EOF
int main() {
    int a = 03, b = 01118, c = 091;
    printf("%d %d %d\n", a, b, c);
    return 0;
}
EOF

Thank @DrXiao for contributing!


GEOMESA-2194 Implement export-leaflet command

This PR provides a new command export-leaflet that generates a leaftlet map from the query results and attempts to open it with the default browser. Generates both a feature and heat map.
Example maps generated from FSDS Quickstart

Signed-off-by: Austin Heyne<EMAIL_ADDRESS>

Note to self check default params printing.

could we just make the leaflet export a format in the normal export command? from a cursory look over, it seems like that would require a lot less code changes. the parameter differences appear to just be related to the default values, which we could override inside the execute method

I've merged it into the export command as format leaflet. I've also added handling so if the output file name is *.html it'll fill in the format. Additionally, I've worked out a way to handle slf4j not being configured by teeing the sub-shell's output to another I/O file that is redirected to the parents stdout.

i'll take a look soon!


sql: user account expiration (role LOGIN option and password expiry via VALID UNTIL option)

PR: https://github.com/cockroachdb/cockroach/pull/45541
From release notes:

User and role principals can now be prevented from logging in using the NOLOGIN attribute, which can be set using ALTER USER/ROLE. [#45541][#45541] {% comment %}doc{% endcomment %}
SHOW USERS now shows all roles/users that have LOGIN enabled. [#45541][#45541] {% comment %}doc{% endcomment %}
Added the VALID UNTIL option to indicate passwords are only valid until a given timestamp. VALID UNTIL only affects PASSWORDS - if passwords are disabled, VALID UNTIL has no effect. Also, you can login even with expired password if passwords are not required to login-- same as PostgreSQL). [#45541][#45541] {% comment %}doc{% endcomment %}
Added the LOGIN option to enable login for roles/users. When using CREATE USER syntax, LOGIN is enabled by default whereas when using CREATE ROLE, LOGIN is not enabled by default. [#45541][#45541] {% comment %}doc{% endcomment %}
Any role created prior to v20.1 will be able to log into clusters started with --insecure, unless/until they are given the NOLOGIN attribute. For secure clusters, roles cannot log in by virtue of not having a password nor a client certificate. [#45541][#45541] {% comment %}doc{% endcomment %}
The password field (used exclusively for password-based authentication) can now be configured to have an expiration date using the VALID UNTIL attribute, which can be set with ALTER USER/ROLE. Note that the attribute sets an expiration date for the password, not the user account. This is consistent with PostgreSQL. [#45541][#45541] {% comment %}doc{% endcomment %}

Addressed in https://github.com/cockroachdb/docs/pull/7061


Use scratch memory instead of perm buffer for the  RFFT kernel's work area.

BUG=333876877

https://github.com/Mergifyio refresh


Drag to scroll is missing mouseUpHandler function to remove event listeners.

This issue needs more explanation in order to understand it well !

It seems that is was fixed in this commit 4f29d7d7e477b828f974705ba4881408c8e1feba.


Cache key confusion when elements depend on an open workspace

See original issue on GitLab
In GitLab by [Gitlab user @samthursfield] on Jul 7, 2018, 15:56
Summary
If I open a workspace for an element, BuildStream messes up the build of subsequent elements in the pipeline.
Steps to reproduce
Using the example project from https://gitlab.com/valentindavid/org.gnu.Hello/:

bst workspace open hello.bst
bst build org.gnu.Hello.bst
bst checkout org.gnu.Hello.bst

What is the current bug behavior?
BuildStream fails to check out the artifact for org.gnu.Hello.bst with an error like this:
AssertionError: flatpak_image element at org.gnu.Hello.bst [line 1 column 0]: Missing artifact b6790049

If you rerun the bst build command, it will rebuild the artifacts that depend on hello.bst and now, the bst checkout will succeed.
What is the expected correct behavior?
None of the cache keys for hello.bst and beyond can be calculated until hello.bst has been built, but BuildStream currently displays (incorrect) cache keys for these.
Other relevant information

BuildStream version affected: 1.1.3+172.g9067e269

In GitLab by [Gitlab user @toscalix] on Jul 9, 2018, 12:34

[Gitlab user @valentindavid] explains this issue during the GUADEC BoF Monday session to core maintainers. They agree this should be considered a bug.
Added the corresponding metadata to consider this bug in the coming weeks.

In GitLab by [Gitlab user @valentindavid] on Aug 15, 2018, 16:35
mentioned in commit fafcca87583950d54a06a29d15c15276c475b2b9

In GitLab by [Gitlab user @valentindavid] on Aug 15, 2018, 16:36
mentioned in merge request !669

In GitLab by [Gitlab user @valentindavid] on Aug 16, 2018, 13:43
mentioned in commit 02f8d840a685f10e5855bf5c9bbc0367ebfc0703

In GitLab by [Gitlab user @valentindavid] on Aug 22, 2018, 15:09
Note that this issue can pollute the cache with artifacts with wrong keys. And since there are no tools for deleting artifacts manually, that means one would need to nuke the whole cache if this happened. And if this happens, there is no error from buildstream. Just wrong artifact. So it will take some time for users to see the issue.
We need to fix it for 1.2.

In GitLab by [Gitlab user @valentindavid] on Aug 22, 2018, 15:12
Open a workspace for the cache pollution:

Have b.bst depend on a.bst.
Make sure b.bst is not in cache.
Open workspace for a.bst and modify the source.
Build b.bst.
Close the workspace.

And now b.bst has an artifact with the key without workspace, even though it was build with the workspace.

In GitLab by [Gitlab user @tristanvb] on Aug 24, 2018, 10:16

And now b.bst has an artifact with the key without workspace, even though it was build with the workspace.

I'm not sure you've explained a problem here.
b.bst surely has an artifact in the local cache, but this artifact was "tainted" to never be shared, and is not reachable by any later resolved pipeline. In other words; closing the workspace for a.bst changes the cache key of b.bst.
What are the side effects which occur at this point, after you've closed the workspace ?
EDIT: Ok, I think I see what is happening here:

We are of course in non-strict mode
When we later perform any commands on b.bst, then a.bst will be in the cache with the same weak cache key
BuildStream in this case will read back the strong cache key from cached artifact, creating a new strong key for a.bst

I can imagine one possible side effect of this is that the resulting a.bst will not be properly "tainted", we might risk sharing it with peers even though it was originally built against a workspaced element which was later closed.
Further, after closing the workspace, we would expect b.bst to have changed it's strong cache key, but a side effect of reading back the strong cache key from the previously cached weak key of a.bst might be that b.bst's cache key does not change accordingly.
I have not verified any of this, it is all theory, I'm grasping at straws. Further, this does not appear to be what is happening in the original report from [Gitlab user @samthursfield], he did not include the closing of a workspace at all in his steps.
It is possible however, that we need to consider workspaced-ness in the calculation of weak cache keys, although this appears to be orthogonal to the reported issue - and we should track this possible breakage separately.
Back on track
To get back on track with this issue; it really appears to simply be an issue with prematurely calculated cache keys in a session where a given dependency is workspaced.
This needs to be addressed at load time.
When an element has a workspace:

It will perform an an initial calculation of it's cache key if possible

This is to determine if it is cached, avoiding rebuilds of every workspace in every build

If we were able to determine a cache key and it was cached, then that cache key need not be discarded, the workspace is seen as unchanged and no rebuild is necessary
If the workspace is changed, then this cache key should be discarded
Discarding this cache key should be done in exactly the right place, before any reverse dependency ever gets a chance to try to calculate their own key; the absence of a cache key on the workspaced element will prevent resolution of reverse dependency cache keys until the workspaced element is successfully built, this prevention happens in Element._update_state()
If a workspace has dependencies which have not yet resolved their keys, either because of tracking or because its dependencies are also workspaced

Then the whole operation needs to block until the dependencies have their cache keys resolved, again this should be a condition in Element._update_state()
In this case, we still need to tentatively calculate the workspaced element's cache key to determine if it is already cached.
Just because we tracked a dependency or such, we do not know whether we have a good workspaced element artifact cached until we check - also, the codepaths should just flow this way to ensure sanity (we don't do anything different at load time or in mid build; we just do the same thing "When it is time to resolve the workspaced element's cache key")

In GitLab by [Gitlab user @juergbi] on Aug 28, 2018, 08:50
This should already be in place in Element._update_state(). Cache keys are discarded early on for workspaced elements that are scheduled for assembly in the current session.
Re-reading the code a possible issue is that workspaced elements are not scheduled for assembly early on because they aren't marked as required yet (dynamic scheduling). If a workspaced element is not scheduled for assembly, its cache key is considered stable, but that's wrong in the case that the element is not yet scheduled. We need to make sure that cache keys are immediately discarded in that case as well.

In GitLab by [Gitlab user @juergbi] on Aug 28, 2018, 10:01
mentioned in commit ddbf7bc2381ae3b0955f64505b2515533b5c1a7a

In GitLab by [Gitlab user @valentindavid] on Aug 28, 2018, 10:01
mentioned in commit ffc556ff2e71c4fb3d06845a41eff0fd6dd5a88d

In GitLab by [Gitlab user @juergbi] on Aug 28, 2018, 10:03
mentioned in merge request !740

In GitLab by [Gitlab user @juergbi] on Aug 28, 2018, 10:05
I've opened an alternative merge request !740. It passes [Gitlab user @valentindavid]'s new test case, which I've included in the MR.

In GitLab by [Gitlab user @tristanvb] on Aug 28, 2018, 12:29
closed via merge request !740

In GitLab by [Gitlab user @tristanvb] on Aug 28, 2018, 12:29
mentioned in commit 6a5a8e7a9942c83581d3cb44e6509f773ca136f9

In GitLab by [Gitlab user @tristanvb] on Aug 28, 2018, 14:08
reopened

In GitLab by [Gitlab user @tristanvb] on Aug 28, 2018, 14:09
This was fixed in master with !740, and the redundant tests fixed by !741.
These need to be backported to bst-1.2

In GitLab by [Gitlab user @valentindavid] on Aug 28, 2018, 14:44
mentioned in commit 1a96b53792cc532298ff730c8b05c8c18b2231ee

In GitLab by [Gitlab user @juergbi] on Aug 28, 2018, 14:44
mentioned in commit 76a66abf99b701e8733703f3a512a6684a2734f3

In GitLab by [Gitlab user @juergbi] on Aug 28, 2018, 14:47
mentioned in merge request !744

In GitLab by [Gitlab user @juergbi] on Aug 28, 2018, 17:33
closed


fix: rule throws error for non eslint-disable comments

Fixes #6

@all-contributors please add @styleshit for code.

🤖 Beep boop! This comment was added automatically by all-contributors-auto-action.
Not all contributions can be detected from Git & GitHub alone. Please comment any missing contribution types this bot missed.
...and of course, thank you for contributing! 💙


add boot config param for lmdb

this pr fixed issue #19
add a new human readable storage size to boot lmdb. Its default value is 1 TB, but allows users to change the value in the config file.
Signed-off-by: mengyang02<EMAIL_ADDRESS>

Please rebase and resolve conflicts

Please rebase and resolve conflicts

done.


CRM457-1327: Spike queue

Simple task queue:

Now instead of calling subscriber webhooks directly, the python app saves a QueuedJob object which adds a row to the que_jobs table.
A task queue called Que polls that table, and its workers instantiate a Ruby NotifySubscriberJob with the saved arguments, and execute it, which triggers a request to the webhook
If the request returns anything other than a 200, an error is raised, causing the job to be retried (with exponential backoff)

Closing this as we're re-implementing the whole app in Ruby, not just the queue, and using Sidekiq rather than Que for consistency and better K8s integration.


Fix missing input background color

we had a small regression with default input bg colors (see /support) that probably stems from the theme refactor. DT has the proper bg but light theme had none.

@Jskobos thx, I'll go ahead and merge this one but will make sure to not go off rails for the next ones


Dependabot can silently pause with no obvious way to un-pause

Is there an existing issue for this?

 Merge a Dependabot pull request to resume automatically creating Dependabot updates or disable security and version updates.

I do actually want Dependabot to keep opening more PRs, but there are no open Dependabot PRs opened in this repository at the moment so it is not clear to me what I should do to un-pause it.
Also, I ended up on this notification by pure luck. It would be much better UX if dependabot somehow pinged me by e-mail when taking such important decisions, like Github Actions does when pausing scheduled runs.
Native package manager behavior
See "Updated dependency" section.
Images of the diff or a link to the PR, issue, or logs
N/A (private security tab)
Smallest manifest that reproduces the issue
N/A (not a manifest issue)

Hi @HadrienG2 ;
Thanks for the feedback. Yes, Dependabot does pause on inactivity, read more here.
You can unpause Dependabot by trying out one of these actions:

Merging a Dependabot pull request
Closing a Dependabot pull request
Making a change to the Dependabot config file
Manually triggering a security update
Manually triggering a version update
Enabling security updates
Using @dependabot commands on pull requests

For example, just editing your dependabot.yml (even a whitespace change) and committing it should re-enable Dependabot processing.
I hope this helps to resolve the pause.

Thanks for the tips regarding other ways to get the bot un-paused besides interacting with (nonexistent) PRs !
One thing that is not clear to me is how it got paused in the first place, since there are no active dependabot PRs in this repository, and either merging or closing any previous PRs that somehow got stale for a while (and 90 days is a lot !) should have gotten it out of the paused state. It is partially due to the absence of dependabot PRs with an associated dependabot-paused tag and status message that this state is so invisible that I missed it for quite a long while...

Dependabot will also pause after 30 job failures if no PRs are produced, which seems to be the case here.
Looking at some of the errors I see it's the same as https://github.com/dependabot/dependabot-core/issues/8672, so #8686 should fix this as well. After that is deployed I'll ask you to manually run a job and upon a successful run Dependabot should unpaused.

@HadrienG2 https://github.com/dependabot/dependabot-core/pull/8686 is deployed now if you want to kick off a run.

Thanks, seems to be working again!


SOLR-15826: ResourceLoader should better respect allowed paths

https://issues.apache.org/jira/browse/SOLR-15826

I think I have struggled in the past in Eclipse with this problem in running tests..  (I think I used to hardcode the lookup).
I sort of wish that setting SOLR_ALLOW_UNSAFE_RESOURCELOADING_PARAM was part of some sort of either annotation or super class, so you could do it on other tests that need it...

I think this might fix some issues I am running into in SOLR-15788.


AppData\Local\KeePass\PluginCache\ is created using KeePass portable

When using Keepass portable (zip archive extracted to a USB drive), this file is left on the local drive.
AppData\Local\KeePass\PluginCache\ random code \KeePassQRCodeView.dll
What exactly is in the cache?
This could be a big security & privacy risk when you use KeePass portable on an untrustworthy system.
KeePassQRCodeView <IP_ADDRESS>
KeePass 2.49 64bit
Windows 10 21H2

KeePass creates this folder for plugins.
https://sourceforge.net/p/keepass/discussion/329221/thread/ca50775e/


Incorrect use of

The error message looks like this:

This is how I used it in From:

Your <FormLabel></FormLabel> does'nt have an htmlFor attribute.

I add htmlFor="xyz" in my <FormLabel> but still sew this error.
The Form RadioGroup from https://ui.shadcn.com/docs/components/radio-group has such error! Please advise.

Somehow I fixed it... do not know why...
            {/* radio buttons */}
            <FormField
              control={form.control}
              name="enum1"
              render={({ field }) => (
                <FormItem className="space-y-1">
                  <FormControl>
                    <RadioGroup
                      onValueChange={field.onChange}
                      defaultValue={field.value}
                      className="flex flex-col space-y-1"
                    >
                      <div className='flex flex-wrap'>
                        <FormItem className="radio-item">
                          <FormControl>
                            <RadioGroupItem value="add" />
                          </FormControl>
                          <FormLabel>
                            Add
                          </FormLabel>
                        </FormItem>

Somehow I fixed it... do not know why...

because you removed the label for enum1

This issue has been automatically closed because it received no activity for a while. If you think it was closed by accident, please leave a comment. Thank you.


how can i integrate socketio-over-nodejs into canvas-designer application.

It works fine when using individually but when i try to integrate it into canvas application i am not able to connect both the user. I was following the video tutorial on this link http://www.rtcmulticonnection.org/docs/openSignalingChannel/ to create a signaling mechanism using socketio. I want to remove the firebaseio call and replace it with some local signalling mechanism.

@creedrahul  sir do you already use socketio-over-nodejs into canvas-designer application? I want to do this I also try to replace the connection.socketURL = 'https://rtcmulticonnection.herokuapp.com:443/'; by connection.socketURL = 'http://localhost:8888/'; but the socket.io say unhandled etc etc. Please help me :(


Yapf fix (only).

Pending tests.

@LeegleechN After this PR, the master branch will pass the CI test now.


Is there plan on handing over maintenance to someone else ?

Nobody pushed to this repository for the past 3 years. I checked @ostreifel profile and he seems to have fly away from GitHub some time ago....
Is there someone willing to take over the maintenance burden and release a new version of this super nice extention?

I just found it today as well and it is excellent but there are some bits that I would like to improve.
I don't have the development know how to be able to do this effectively though. One thing I noticed is that when repos are split across multiple projects it is harder to get the overall contribution picture for a user.


Add CONTRIBUTING.md guidelines

Seems useful at an external julia package: CONTRIBUTING.md - we could add something similar in mia and link from README. Can be helpful for less experienced users, and even for more experienced to harmonize conventions.

Added. Can be improved and extended over time.


With a long interval it could cause a crash and not loading any results when it doesn't crash

Steps to reproduce
First put on sankaku complex as source and put the interval for it on 8s by the page interval.
After that try to load nekopara for example.
After this most of the time you will first encounter that sankaku complex is offline (like in the image below).
Press now just a couple times OK and you will see that a crash occurs.
(sorry for making a second topic wanted to make it clearer since I thought that you didn't understand it)
Screenshots
https://i.imgur.com/MnBLy00.png
System information
Windows 10 with the latest nightly version

Indeed, I couldn't reproduce your other issue, but this one I can, thanks :+1:


Next-Pylint CI pipeline

Edit to CI pipeline for next version of pylint and docs explaining pylint and how it will work

/azp run python - eventgrid - ci

/azp run python - eventgrid


Bug: esc(..., 'raw')  fails on recursive data structures

PHP Version
8.2
CodeIgniter4 Version
4.4.6
CodeIgniter4 Installation Method
Composer (as dependency to an existing project)
Which operating systems have you tested for this bug?
Linux
Which server did you use?
apache
Database
No response
What happened?
In the process of upgrading from CI3 -> 4, I encountered a problem with a view() that uses a fairly complex $data structure including &references between elements. Even though the view() helper calls setData() with $context = 'raw', this still ends up in infinite recursion because esc() iterates through the fields of the array and calls esc(..., 'raw') on them. This whole operation is completely unnecessary because in the end it does not escape anything. However, because of the self-references, this walking of the array ends up in infinite recursion.
The esc() function should return $data immediately in the case of 'raw'. There's no need to walk the whole array tree just to build it anew.
Steps to Reproduce
A non-realistic example, just to illustrate the point:
$data = ['key' => []];
$data['key']['recur'] = &$data;
esc($data, 'raw');

Expected Output
The exact same $data array, without iterating its values.
Anything else?
No response

You have a valid point. Can you send a PR?
https://github.com/codeigniter4/CodeIgniter4/blob/develop/contributing/README.md


RailsAdmin::Config.reset does not fully reset config

In my app, I'm attempting to manage which models are excluded by customer, which I'm doing with a decorator (see below):
RailsAdmin::ApplicationController.class_eval do
	before_action :switch_tenant, :reload_rails_admin

	def switch_tenant
		Apartment::Tenant.switch!(params[:customer_id])
	end

	def reload_rails_admin
		Rails.application.eager_load!
		excluded_models = ApplicationRecord.descendants.map { |m| m.name unless m.base_class.table_exists? }.compact
		puts "Excluded models: #{excluded_models}"

		RailsAdmin::Config.reset
		RailsAdmin.config do |config|
			config.excluded_models = excluded_models

			config.actions do
				dashboard
				index
				new
				export
				bulk_delete
				show
				edit
				delete
				show_in_app
			end
		end
	end

end

What I've found is that the RailsAdmin config is sticky.  Depending on which customer I load first, that's the set of models that are used for the next customer.  At first, I thought maybe it was a caching thing (maybe it still is?), but I've attempted to insert Rails.cache.clear calls at various stages to no effect.
Any thoughts?

Yeah, you need to reload the server each time you change this config. It is the same with all initializers.

I have the same exact problem. No matter what I tried, I am not able to make the changes to included_models option work.
When I print the included_models option value, it seems it is changed but it doesn't affect the frontend.

Yeah this is not ideal, I'll take a look into this to see if it's fixable.


[BUG] Banner deletes item in hand

Plugin Version: FlagsH v4.0.1
Server Version:  Folia version 1.20.6, build 4
When a player holds a banner in their off-hand and holds an item in their right hand, the item in their right hand gets deleted and no banner is removed from the off-hand.
Video: https://youtu.be/xwAa9sqCxYg

Thanks for reporting it, I will fix it.


How to make Queue Event

What are some ways to create a queue event?

Queues aren't a fundamental building block in songbird. What you can probably make use of is TrackEvent::End: have a look at some of our examples to get a feel for how the event system works.


Target Capabilities / Board Directory Layout Capabilities

Starting this thread to discuss what features we want to support based on the layout of the boards directory.
Today we support/assume:

<BOARD> is expected to match a <BOARD>_defconfig in boards/<ARCH>/<BOARD_DIR_NAME>
Determining the ARCH of the board based on the directory path (its assumed that boards/<ARCH>/<BOARD_DIR_NAME> is the layout
<ARCH> is assumed as the direct parent of <BOARD_DIR_NAME>
[ I want the assumption of 3. to be that <ARCH> is the name right before "boards" - See PR #12377 ]

closing in favor of https://github.com/zephyrproject-rtos/zephyr/issues/51831


ExecutionPolicy under what circumstances, who can give an example to explain it?

Description

Environment

Pinpoint version:
OS, WAS, affected library, etc:
Additional details (e.g. how your collector/web is set up):

Additional Info

Hi @jacksu
ExecutionPolicy is closely related to interceptor scope.
You can check out the detail in one of our samples.

@Xylus  thanks, I haved understand.


Fix loading of unnecessary relations

While calling toArray or toJson on a model, subsequent call to attributesToArray may load extra relations that the following call relationsToArray will take into account. This can result in either including unnecessary relations to the final converted object or even stuck in infinite loop.
This can be avoided just by changing the order of these two functions.
Quick example that can lead to infinite loop:
User model has $with = ['profile']
Profile has a getter in $appends that eager loads a User by calling $this->user
On the other hand, if instead of calling $this->user, another relation was loaded, let's say $this->address, the later address object will also be unnecessarily included in the final converted array.

Yes, indeed some apps may start behave differently, but I don't think this is intended and consistent behavior. Personally, I run into this problem when I got a 500 because of the infinite loop scenario. It took me a long time to figure out the problem.

No plans to change on a point release. On another note, I wish people would stop using $with in this way. It only works "by accident" and was never an intended usage of $with property on the model.


Trying to add icon with no success

im using --icon=src/assets/icons/win/icon.ico with no success For some reason the icon is just not changing.
Am I doing something wrong?

Are you in the right repo? We don't even have an icon option 😆. electron-windows-store isn't responsible for the icon of your Electron app, you'd do that with electron-packager or any of the other "compilation" packages :wink:

Oh okay, didn't know lol. That makes sense. Yes im in the right repo just didn't realize I had to set that in electron-packager hahaha


Get workbook/data source by content URL?

In the REST API, is there a way to get a single workbook or data source via its content URL? Currently we are using the methods to list all workbooks just to find the ID which is pretty inefficient since we already have a site unique content URL.
Thanks.

How did you get the content URL? There are a set of "filters" you can apply to the get call to indicate which workbooks you want. We don't have a filter on contentUrl but we have filters on name, owner, and some other items. Perhaps you have those? So back to the original question ... how did you get the contentUrl?

One example would be from the Tableau Server log files which include the site and workbook/data source content URL in certain messages. The use case would include parsing those messages and then querying the REST API for details.
A second example might be when parsing the TWB/TDS file which sometimes includes the repositoryUrl. A use case may include parsing that from the file and then querying the REST API for details of the server's copy of that workbook or data source.

Maybe we should log the LUIDs so you just have them available.
contentURLs don't have a good contractual guarantee to stay a good identifier (I know they behave this way today, but there's no promise they will continue)

Should be coming in 2019.4

is there any update on this request? I want to retrieve datasources from the url
Thanks

Is this implemented yet?
When I access webpage_url attribute across datasources none of them has a value.  They all have a content_url though, but that's not what I'm after.
What I'd want to access is the <INTEGER>  from a datasource URL of this format:
https://<WEBSITE>/#/datasources/<INTEGER>

This would make data management a lot easier.  It's quite painful having to search by name each time in the server when you want to inspect a datasource. 😣

Get workbook by content url has been available for a while - https://help.tableau.com/current/api/rest_api/en-us/REST/rest_api_ref_workbooks_and_views.htm#query_workbook
Querying workbooks or datasources also allows you to filter based on the contentUrl - https://help.tableau.com/current/api/rest_api/en-us/REST/rest_api_concepts_filtering_and_sorting.htm#datasources

@jacalata as for tableauserverclient=0.26 and tableauhyperapi=0.0.16868 it returns null for datasource.webpage_url


System.MissingMethodException on call to .Report.ToElasticSearch()

Hi - I'm trying to use version 2.0.0-alpha in my project, and am stumbling at the first hurdle with an exception being thrown from the call to .Report.ToElasticSearch() on startup
I'm using the overload which takes the string url and elasticsearchIndex arguments
System.MissingMethodException
HResult=0x80131513
Message=Method not found: 'Void App.Metrics.GeneratedMetricNameMapping..ctor(System.Collections.Generic.IDictionary2<App.Metrics.HistogramValueDataKeys,System.String>, System.Collections.Generic.IDictionary2<App.Metrics.MeterValueDataKeys,System.String>, System.Collections.Generic.IDictionary2<App.Metrics.ApdexValueDataKeys,System.String>, System.Collections.Generic.IDictionary2<App.Metrics.CounterValueDataKeys,System.String>)'.
Source=App.Metrics.Formatters.Elasticsearch
StackTrace:
at App.Metrics.Formatters.Elasticsearch.MetricsElasticsearchDocumentFormattingOptions..ctor()
at App.Metrics.Formatters.Elasticsearch.MetricsElasticsearchOutputFormatter..ctor(String elasticsearchIndex)
at App.Metrics.Reporting.Elasticsearch.ElasticsearchReporter..ctor(MetricsReportingElasticsearchOptions options, IElasticsearchClient elasticsearchClient)
at App.Metrics.MetricsElasticsearchReporterBuilder.ToElasticsearch(IMetricsReportingBuilder metricReporterProviderBuilder, String url, String elasticsearchIndex)
..........
is this currently a known issue? Thanks

The elasticsearch reporter is somewhat out of date, an update is coming soon.
Are you using the same version of the App.Metrics package?

thanks for replying so quick
will check on the app.metrics version - watch this space 👍

Updated to all latest alpha packages, and I no longer get the MissingMethodException on startup, so that's progress
Unfortunately I now get this
System.TypeLoadException: 'Method 'get_MetricFields' in type 'App.Metrics.Formatters.Elasticsearch.MetricsElasticsearchOutputFormatter' from assembly 'App.Metrics.Formatters.Elasticsearch, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=0d5193a913d1b812' does not have an implementation.'
Package depdendency versions are described here:

Same Problem experienced here.
My package dependencies are:

Is this project abandoned? Nothing seems to work.

It's not abandoned I just haven't had capacity. I'll try getting to it this weekend.
I don't have plans in the short term though to update supporting the lasted version of Elasticsearch.

To the rest who are concerned, what I did is downloading the entire program, recompiling the source code, fixing wherever necessary and packaging it into a new nuget package for my own need. It worked really well.

@foresightyj Could you please push your changes to a GitHub fork?

@alhardy We're trying to decide whether to use InfluxDb or wait for Elastic Search fix. Could you please give a rough estimate on an Elastic Search fix?

@siberianguy trying to get to it, a lot in at the moment. I am assuming this is just a version dependency issue. Have you tried keeping all AppMetrics packages on 2.0.0-alpha?

To the rest who are concerned, what I did is downloading the entire program, recompiling the source code, fixing wherever necessary and packaging it into a new nuget package for my own need. It worked really well.

care to share?

FHT.App.Metrics.Elasticsearch.zip
FHT.App.Metrics.Elasticsearch.1.0.1.nupkg.zip
@mleyb  @siberianguy Here are the zipped code and nuget package.


Create new testing schema for each testing run

This PR reverts back to the old endpoints test, but now creates a new schema based on timestamp, and runs all endpoint tests within the created schema.
This PR closes issue #516 , in cooperation w/ PR 78 in brainscore core.

closing until new core update hits


Subdomain support for next-password-cookie

First of all, thanks for this package! It's really a handy one.
Describe the problem
I'm working on an app that supports custom subdomains.
Once logged in (with next-pass), I still need to repeat the same login process when navigating to each of the subdomains.
This happens due to the way the cookie is being set - domain attribute is omitted and thus it defaults to the current host (excluding subdomains).
Describe the solution
I would like to have an option to opt out of the default behavior and set the domain explicitly to allow subdomain support.
Additional
I'd be happy to offer help on this.

Hi @korac, thanks for the issue! Seems like a useful thing to support.
I think it should be possible to pass the domain to the cookie here. Would you be open to making a PR with this?

@BJvdA Thanks for the info! Will open a PR.


cpu_test

Ⅰ. Describe what this PR did
UT for ValidateCPUPeriod and ValidteCPUQuota
Ⅱ. Does this pull request fix one issue?

fixes #1637
fixes #1638
Ⅲ. Describe how you did it
offer multiple inputs to check different cases of if statement.
Ⅳ. Describe how to verify it
use map whose key is input, value if expected output. Compare result and expected output
Ⅴ. Special notes for reviews

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.root seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

Codecov Report

Merging #1697 into master will decrease coverage by 22.94%.
The diff coverage is n/a.


Remove biased annotations from prometheus.service.annotations

Why is this pull request needed and what does it do?
Makes the default prometheus.service.annotations blank {} map. Removing any bias.
Which issues (if any) are related?
Fixes https://github.com/coredns/helm/issues/157
Checklist:

[x] I have bumped the chart version according to versioning.
[x] I have updated the chart changelog with all the changes that come with this pull request according to changelog.
[x] Any new values are backwards compatible and/or have sensible default.
[x] I have signed off all my commits as required by DCO.

Changes are automatically published when merged to main. They are not published on branches.

Note on DCO
If the DCO action in the integration test fails, one or more of your commits are not signed off. Please click on the Details link next to the DCO action for instructions on how to resolve this.

This would potentially break exiting installation that rely on this values being set by default

@hagaibarel correct.
well, c'est la vie.
But this is easy to fix for anyone.
Currently I'm kinda fixing my setup for myself too, by providing null and I could ignore the "issues" completely.
I just wanted to make this chart for the community better.
Otherwise for how long are you going to haul this """not-that-smart""" non-standard piece in values.yaml?

c'est la vie

This isn't my preferred way of maintaing an OSS project where we have a community and users to support. If we wanted to introduce a change that would break people's exiting installation, this should have been a major version release, not a patch (or even a minor).

Otherwise for how long are you going to haul this """not-that-smart""" non-standard piece in values.yaml?

Given that there isn't a "standard" in values file, it's common to provide sane defaults with an option to override / change them.
Bottom line, I don't think this change has any meaningful value

added comment to the issue. I can bump chart version to 2.0.0

I don't think this change justifies bumping a major version, nor has any significant value.
There is an option to override the annotations if needed, and the defaults make sense and fit most use cases.

The override option is not functioning 100% as one would think by default and is not covering all cases.

Can you provide a real world use case from your experience where overriding didn't address the issue?

Can you provide a real world use case from your experience where overriding didn't address the issue?

I have provided exact example in the original issue linked to this PR.
Real world fact is that maps in Helm are not overridden but merged.


[Document] Adding UDF by impl ScalarUDFImpl

Which issue does this PR close?
Closes #9136.
Rationale for this change
See #9136
What changes are included in this PR?
Document update including adding a scalar udf by impl ScalarUDFImpl

Thanks for your review! @alamb and have applied the suggestions.
Yeah, I'm also thinking whether we should keep the old create_udf as a second way to adding udf or just remove that. :)
And after this PR, I think I can create a follow-up PR about Window UDF and Aggregate UDF.


Null values issue

I've just been debugging an issue for a while before realizing that it's submitting blog as null and erroring with:

"Missing required field: blog."

All of the values that are being submitted as null should be empty strings otherwise akismet won't process the request.

ping @nickurt this is pretty much breaking the whole package

Bump. Any news @nickurt ?

I can't repro it here, did what you told about it but I got an error then. Did you have a snipped/example what I can try?

What version of guzzle did you used @malinthe ? Can you show me your composer.json require/-dev, then I can look into it.

@nickurt It was Guzzle 6.2
My composer.json file:
"require": {
	"laravel/framework": "5.1.*",
	"facebook/php-sdk-v4": "^4.0",
	"guzzlehttp/guzzle": "^6.0",
	"barryvdh/laravel-ide-helper": "^2.0",
	"barryvdh/laravel-debugbar": "^2.0",
	"guzzlehttp/psr7": "^1.1",
	"intervention/image": "^2.3",
	"greggilbert/recaptcha": "dev-master",
	"sunra/php-simple-html-dom-parser": "v1.5.0",
	"nickurt/laravel-akismet": "dev-master"
},
"require-dev": {
	"phpunit/phpunit": "~4.0",
	"phpspec/phpspec": "~2.1",
	"mockery/mockery": "~0.9.2"
}

I tested it and it works here fine (nginx/php7) ...
Route::get('/', function () {
    $akismet = \Akismet::validateKey();

    dd($akismet);
});

Did you get an error or something else?

I've mentioned the error I receive below. The same code works locally on Apache/PHP5 without any issues.
FatalThrowableError in Akismet.php line 427:
Wrong parameters for Exception([string $message [, long $code [, Throwable $previous = NULL]]])

in Akismet.php line 427
at Exception->__construct(array('Empty "blog" value')) in Akismet.php line 427
at Akismet->getResponseData('https://xxxxxxxx.rest.akismet.com/1.1/comment-check') in Akismet.php line 375


[tf2.0] decode numpy array with bytes format stored in tensor

I can not found any documentation about this.
I saved some numpy arrays into tfrecord, it stored as bytes (using np.array().to_bytes() method)
When read it back, there were no way to decode that bytes back to numpy.
Anyone knows how to solve it?

@jinfagang tensor.numpy() --> gives numpy array of the given tensor in TF2.0.
a=tf.constant([1,2,3,4,5])
a # <tf.Tensor: id=32, shape=(5,), dtype=int32, numpy=array([1, 2, 3, 4, 5], dtype=int32)>
a.numpy() # array([1, 2, 3, 4, 5], dtype=int32)

This is not Build/Installation or Bug/Performance issue. Please post this kind of support questions at Stackoverflow. There is a big community to support and learn from your questions. GitHub is mainly for addressing bugs in installation and performance. Thanks!

I think it was resolved. I am closing the issue. But, please let me know if I'm mistaken. Thanks!


ext_authz: upgraded to the latest Envoy ext-authz with support to allowed authorization headers

I'm not sure what I'm doing wrong when running KAT tests. Here is the problem:

Run DOCKER_REGISTRY=quay.io/gsagula make clobber all

# Everything passes
=== 96 passed, 9 xfailed, 20 warnings in 197.75 seconds =======

DOCKER_REGISTRY=quay.io/gsagula make shell
3 pytest -sk AuthenticationTest --disable-pytest-warnings -v

Traceback (most recent call last):
  File "/Users/gsagula/src/datawire/ambassador/kat/kat/harness.py", line 513, in setup
    self._setup_k8s()
  File "/Users/gsagula/src/datawire/ambassador/kat/kat/harness.py", line 599, in _setup_k8s
    action()
  File "/Users/gsagula/src/datawire/ambassador/ambassador/tests/kat/abstract_tests.py", line 197, in post_manifest
    result.check_returncode()
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 409, in check_returncode
    self.stderr)
subprocess.CalledProcessError: Command '('docker', 'run', '-d', '--name', 'ratelimittest', '-p', '8877:8877', '-p', '8080:80', '-p', '8443:443', '-v', '/tmp/ratelimittest-ambassadormixin-secret:/var/run/secrets/kubernetes.io/serviceaccount', '-e', 'KUBERNETES_SERVICE_HOST=kubernetes', '-e', 'KUBERNETES_SERVICE_PORT=443', '-e', 'AMBASSADOR_ID=ratelimittest', 'quay.io/gsagula/ambassador:test-branch-d0eb420-dirty')' returned non-zero exit status 125.
Exit: setup failed

From inside make shell make clean all

go get github.com/gorilla/websocket
AMBASSADOR_DOCKER_IMAGE=quay.io/gsagula/ambassador:test-branch-d0eb420-dirty \
	KUBECONFIG=/Users/gsagula/src/datawire/ambassador/cluster.yaml \
	AMBASSADOR_DEV=1 \
	bash --init-file releng/init.sh -i
(venv) bash-3.2$ make clean all
rm -f cluster.yaml
test -x venv/bin/kubernaut && venv/bin/kubernaut claims delete kat-gsagula || true
rm -f kubernaut-claim.txt
rm -rf docs/yaml docs/_book docs/_site docs/package-lock.json
rm -rf helm/*.tgz
rm -rf app.json
rm -rf ambassador/ambassador/VERSION.py*
rm -rf ambassador/build ambassador/dist ambassador/ambassador.egg-info ambassador/__pycache__
find . \( -name .coverage -o -name .cache -o -name __pycache__ \) -print0 | xargs -0 rm -rf
find . \( -name *.log \) -print0 | xargs -0 rm -rf
find ambassador/tests \
		\( -name '*.out' -o -name 'envoy.json' -o -name 'intermediate.json' \) -print0 \
		| xargs -0 rm -f
rm -rf end-to-end/ambassador-deployment.yaml end-to-end/ambassador-deployment-mounts.yaml
find end-to-end \( -name 'check-*.json' -o -name 'envoy.json' \) -print0 | xargs -0 rm -f
rm -f end-to-end/1-parallel/011-websocket/ambassador/service.yaml
find end-to-end/1-parallel/ -type f -name 'ambassador-deployment*.yaml' -exec rm {} \;
# TODO: validate version is conformant to some set of rules might be a good idea to add here
Generating and templating version information -> test-branch-d0eb420-dirty
sed \
		-e 's!{{VERSION}}!test-branch-d0eb420-dirty!g' \
		-e 's!{{GITBRANCH}}!test_branch!g' \
		-e 's!{{GITDIRTY}}!dirty!g' \
		-e 's!{{GITCOMMIT}}!d0eb420!g' \
		-e 's!{{GITDESCRIPTION}}!0.50.0-ea3-85-gd0eb420!g' \
		< VERSION-template.py > ambassador/ambassador/VERSION.py
go get github.com/gorilla/websocket
docker build -q  -t quay.io/gsagula/ambassador:test-branch-d0eb420-dirty .
sha256:58ed7b0d9048adf19b906222823c5313098c11d521e13a4069b75afc962ddbd7
PUSH quay.io/gsagula/ambassador:test-branch-d0eb420-dirty
.....................................................................
venv/bin/kubernaut claims delete kat-gsagula
venv/bin/kubernaut claims create --name kat-gsagula --cluster-group main

 Your clusters kubeconfig has been written to the below path:

 /Users/gsagula/.kube/kat-gsagula.yaml

 Usage Instructions:

 There are several ways you can use this kubeconfig file:

 1. Pass --kubeconfig to the `kubectl` command:

    kubectl --kubeconfig=/Users/gsagula/.kube/kat-gsagula.yaml ...

 2. Set the KUBECONFIG environment variable:

    export KUBECONFIG=/Users/gsagula/.kube/kat-gsagula.yaml

cp ~/.kube/kat-gsagula.yaml cluster.yaml
rm -f /tmp/k8s-*.yaml
venv/bin/teleproxy -kubeconfig /Users/gsagula/src/datawire/ambassador/cluster.yaml 2> /tmp/teleproxy.log &
Sleeping for Teleproxy cluster
sleep 10

========================================================= test session starts =========================================================
platform darwin -- Python 3.7.0, pytest-3.10.0, py-1.7.0, pluggy-0.8.0
rootdir: /Users/gsagula/src/datawire/ambassador, inifile: pytest.ini
plugins: cov-2.6.0
collected 105 items

tests/ambassador_test.py FFFFFFFFFFFFFFFF................                                                                       [ 30%]
tests/corner_case_test.py xxxxx

 .. then generate a bunch of tests!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! _pytest.outcomes.Exit: setup failed !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
==================================== 16 failed, 16 passed, 5 xfailed, 21 warnings in 58.21 seconds ====================================

Thoughts?
cc @rhs

I just saw that we have this README:
https://github.com/datawire/ambassador/blob/release/0.50.0/ambassador/README.md
I will try to go over it tomorrow.

Hmmm. I need to look at that README too -- lots of doc really is out of date in 0.50.0 still. 😦
This error:
subprocess.CalledProcessError: Command '('docker', 'run', '-d', '--name', 'ratelimittest', '-p', '8877:8877', '-p', '8080:80', '-p', '8443:443', '-v', '/tmp/ratelimittest-ambassadormixin-secret:/var/run/secrets/kubernetes.io/serviceaccount', '-e', 'KUBERNETES_SERVICE_HOST=kubernetes', '-e', 'KUBERNETES_SERVICE_PORT=443', '-e', 'AMBASSADOR_ID=ratelimittest', 'quay.io/gsagula/ambassador:test-branch-d0eb420-dirty')' returned non-zero exit status 125.

usually, for me, means that kat is confused about which Docker pods are still running. make clean all should indeed fix that, BUT:
There's also a bug right now where make tries to do pip install -e ./ambassador and, for some reason I don't yet know, that doesn't actually work on a Mac. I'll try to sort that out today.
Here's the workaround, which is ugly but works: after you do make shell and make clean all, then:
pip install ./ambassador   # note no -e here!
pytest --tb=short    # or whatever other options you like

That pip install will clean up the broken -e install and you'll be able to run the tests. The downside is that any time you change Ambassador code, you need to rerun pip install ./ambassador to update the Python package stuff. So your dev loop becomes

hack on some Ambassador code
pip install ./ambassador
pytest --tb=short
repeat

and that's not great, but it'll work.

@kflynn Thank you for the instructions.
$ make shell
$ make clean all
$ pip install ./ambassador   # note no -e here!

Here is what happened:
Installing collected packages: setuptools, oauthlib, requests-oauthlib, python-dateutil, pyasn1, rsa, pyasn1-modules, cachetools, google-auth, kubernetes, ambassador
  Found existing installation: setuptools 18.5
    Uninstalling setuptools-18.5:
Could not install packages due to an EnvironmentError: [('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc', '/private/tmp/pip-uninstall-cnmD6A/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc', "[Errno 1] Operation not permitted: '/private/tmp/pip-uninstall-cnmD6A/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py', '/private/tmp/pip-uninstall-cnmD6A/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py', "[Errno 1] Operation not permitted: '/private/tmp/pip-uninstall-cnmD6A/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py', '/private/tmp/pip-uninstall-cnmD6A/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py', "[Errno 1] Operation not permitted: '/private/tmp/pip-uninstall-cnmD6A/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc', '/private/tmp/pip-uninstall-cnmD6A/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc', "[Errno 1] Operation not permitted: '/private/tmp/pip-uninstall-cnmD6A/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib', '/private/tmp/pip-uninstall-cnmD6A/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib', "[Errno 1] Operation not permitted: '/private/tmp/pip-uninstall-cnmD6A/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib'")]

Well, it looks like a real failure -- if you can make the request, and you're even getting the correct HTTP status, that implies that there's some issue with header whitelisting. Back to the Envoy logs?


Why does the symbol S mean?

I got some S in my status dots in the grid, they wont continue, what does the S mean? cant fix it without any information :D

Switch. You might have run out of extra accounts because of captchas or empty scans.

Well yea, i have zero extra accounts because i used a 5x5 grid with 25 accounts. if ill have 30 accounts so i got 5 extra accounts, i cant be sure that all are on level 2 after 16-20 spins can i?


Fixed clipping of unbounded upper releases version numbers

Old code would assume that a missing upper or lower bound should be
treated the same but effectively they have different meanings:

Missing lower bound should just return current high bound
Missing upper bound should be used like a normal comparison baseline
which may not return a result

This change should fix the resulting output of unreleased changes while
having no effect on other outputs.
Connected to #154

CC is green, raising broken reporting with @cyberark/conjur-infra-team


Cross-compiling Nix is incomplete

Describe the bug
I cross-compiled Nix using nix-build '<nixpkgs>' -A pkgs.pkgsCross.remarkable1.nix and copied the closure over to my tablet, however at least one of the dependencies failed to cross-compile, bash. Here's an example session
To Reproduce
reMarkable: ~/ nix-env -i /nix/store/7ffamdhnw49fkcdix9bldvl6wvilwikn-nix-2.3.9-armv7l-unknown-linux-gnueabihf
replacing old 'nix-2.3.7pre0_0000000'
installing 'nix-2.3.9-armv7l-unknown-linux-gnueabihf'
building '/nix/store/p5nzpkhrq72w5dmvza2azlvk8cxmg9i2-user-environment.drv'...
created 38 symlinks in user environment
reMarkable: ~/
reMarkable: ~/ nix-channel --update
unpacking channels...
while setting up the build environment: executing '/nix/store/516z50fm1jbpcl32qnzy7kynrh0vl22w-bash-4.4-p23/bin/bash': Exec format error
builder for '/nix/store/2492z962y1cm6l5mr2v72244mqnp6rvh-nixpkgs-21.03pre257666.12beaac9337.drv' failed with exit code 1
error: build of '/nix/store/2492z962y1cm6l5mr2v72244mqnp6rvh-nixpkgs-21.03pre257666.12beaac9337.drv' failed
error: program '/nix/store/7ffamdhnw49fkcdix9bldvl6wvilwikn-nix-2.3.9-armv7l-unknown-linux-gnueabihf/bin/nix-env' failed with exit code 100
reMarkable: ~/

@tadfisher let me know if you can replicate this on rM 2.
Expected behavior
Nix should cross-compile all of its dependencies correctly.
Notify maintainers
@edolstra
Metadata
system: `"x86_64-linux"`
 - host os: `Linux 5.4.80, NixOS, 21.03pre257339.83cbad92d73 (Okapi)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.9`
 - channels(root): `"nixos-21.03pre255513.8ee7c2505f9"`
 - channels(siraben): `"home-manager"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

This should be fixed in https://github.com/NixOS/nix/commit/045708db4343174f30f3647776971c852f72a9e8. nixUnstable includes that commit.

@matthewbauer I tried to cross-compile nixUnstable, my channel has the commit at https://github.com/NixOS/nix/commit/79aa7d95183cbe6c0d786965f0dbff414fd1aa67, and the build fails:

build log
/nix/store/bfnd0jymd687lz50lmg5h33vy802w4f6-armv7l-unknown-linux-gnueabihf-stage-final-gcc-debug-9.3.0/armv7l-unknown-linux-gnueabihf/include/c++/9.3.0/bits/vector.tcc: In member function 'void std::vector<_Tp, _Alloc>::_M_realloc_insert(std::vector<_Tp, _Alloc>::iterator, _Args&& ...) [with _Args = {const nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long long int, long long unsigned int, double, std::allocator, nlohmann::adl_serializer>&}; _Tp = nlohmann::basic_json<>; _Alloc = std::allocator<nlohmann::basic_json<> >]':
/nix/store/bfnd0jymd687lz50lmg5h33vy802w4f6-armv7l-unknown-linux-gnueabihf-stage-final-gcc-debug-9.3.0/armv7l-unknown-linux-gnueabihf/include/c++/9.3.0/bits/vector.tcc:426:7: note: parameter passing for argument of type 'std::vector<nlohmann::basic_json<>, std::allocator<nlohmann::basic_json<> > >::iterator' {aka '__gnu_cxx::__normal_iterator<nlohmann::basic_json<>*, std::vector<nlohmann::basic_json<>, std::allocator<nlohmann::basic_json<> > > >'} changed in GCC 7.1
  426 |       vector<_Tp, _Alloc>::
      |       ^~~~~~~~~~~~~~~~~~~
In file included from /nix/store/bfnd0jymd687lz50lmg5h33vy802w4f6-armv7l-unknown-linux-gnueabihf-stage-final-gcc-debug-9.3.0/armv7l-unknown-linux-gnueabihf/include/c++/9.3.0/vector:67,
                 from src/libutil/types.hh:9,
                 from src/libutil/util.hh:3,
                 from src/nix/installables.hh:3,
                 from src/nix/command.hh:3,
                 from src/nix/profile.cc:1:
/nix/store/bfnd0jymd687lz50lmg5h33vy802w4f6-armv7l-unknown-linux-gnueabihf-stage-final-gcc-debug-9.3.0/armv7l-unknown-linux-gnueabihf/include/c++/9.3.0/bits/stl_vector.h: In member function 'std::string ProfileManifest::toJSON(nix::Store&) const':
/nix/store/bfnd0jymd687lz50lmg5h33vy802w4f6-armv7l-unknown-linux-gnueabihf-stage-final-gcc-debug-9.3.0/armv7l-unknown-linux-gnueabihf/include/c++/9.3.0/bits/stl_vector.h:1195:4: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<nlohmann::basic_json<>*, std::vector<nlohmann::basic_json<>, std::allocator<nlohmann::basic_json<> > > >' changed in GCC 7.1
 1195 |    _M_realloc_insert(end(), __x);
      |    ^~~~~~~~~~~~~~~~~
In file included from /nix/store/bfnd0jymd687lz50lmg5h33vy802w4f6-armv7l-unknown-linux-gnueabihf-stage-final-gcc-debug-9.3.0/armv7l-unknown-linux-gnueabihf/include/c++/9.3.0/vector:72,
                 from src/libutil/types.hh:9,
                 from src/libutil/util.hh:3,
                 from src/nix/installables.hh:3,
                 from src/nix/command.hh:3,
                 from src/nix/profile.cc:1:
/nix/store/bfnd0jymd687lz50lmg5h33vy802w4f6-armv7l-unknown-linux-gnueabihf-stage-final-gcc-debug-9.3.0/armv7l-unknown-linux-gnueabihf/include/c++/9.3.0/bits/vector.tcc: In member function 'bool nlohmann::detail::parser<BasicJsonType>::sax_parse_internal(SAX*) [with SAX = nlohmann::detail::json_sax_dom_parser<nlohmann::basic_json<> >; BasicJsonType = nlohmann::basic_json<>]':
/nix/store/bfnd0jymd687lz50lmg5h33vy802w4f6-armv7l-unknown-linux-gnueabihf-stage-final-gcc-debug-9.3.0/armv7l-unknown-linux-gnueabihf/include/c++/9.3.0/bits/vector.tcc:121:4: note: parameter passing for argument of type '__gnu_cxx::__normal_iterator<nlohmann::basic_json<>*, std::vector<nlohmann::basic_json<>, std::allocator<nlohmann::basic_json<> > > >' changed in GCC 7.1
  121 |    _M_realloc_insert(end(), std::forward<_Args>(__args)...);
      |    ^~~~~~~~~~~~~~~~~
builder for '/nix/store/0hjwyakp5vwmxcv980bwac3j839d15kn-nix-2.4pre20201118_79aa7d9-armv7l-unknown-linux-gnueabihf.drv' failed with exit code 2
error: build of '/nix/store/0hjwyakp5vwmxcv980bwac3j839d15kn-nix-2.4pre20201118_79aa7d9-armv7l-unknown-linux-gnueabihf.drv' failed

I don't get that error, but do get a different one. Can you try https://github.com/NixOS/nixpkgs/pull/107909 ?

That worked (I cross-compiled to pkgsCross.remarkable1) and after copying the closure, Nix ran, as before, and nix-channel --update works now. Thanks!
reMarkable: ~/ nix --version
nix (Nix) 2.4pre20201228_abbf9df
reMarkable: ~/ nix-channel --update
unpacking channels...
reMarkable: ~/

Closing because resolved.

Closing because resolved.