Prometheus mTLS scraping seems broken, fails to startup in Istiod model

Using istiod we can remove citadel and instead use direct SDS from istio-agent. However, this won't work for prometheus which has no sidecar but mounts the certs.
I believe the intent here is to allow scraping pods with mTLS. As far as I can tell this does NOT actually work. We are looking for this mtls label that never exists: https://github.com/istio/istio/blob/80bee4e1e8c9da71a27dccaf62ccf655d132c0d6/install/kubernetes/helm/istio/charts/prometheus/templates/configmap.yaml#L269. On my cluster, I see that there are 0 targets for kubernetes-pods-secure, and all of the kubernetes-pods fail (presumably due to mTLS).
I think there are a few action items here. If it is broken, and we don't care, we could just remove the kubernetes pod scraping (note - this is scraping user pods, envoy stats are different). However, presumably users need a way to scrape their mTLS pods, so we still need some solution for the certs.
If we do want to keep this, we need to find a more reliable way to identify if a pod is mTLS ready. This could be done easily with auto-mtls, but I am not sure how it would work with just standard strict mTLS.
We also need a way to get the certs to prometheus. This problem may extend beyond prometheus as well for user applications that need access to the certs outside of Envoy. One possibility here is to have an SDS-only mode for istio-agent, that just fetches the certs and writes them to disk as an init container.
cc @myidpt @costinm @douglas-reid

Prom with mTLS was working and tested. Was your test cluster using Permissive mode? I believe there was an issue identified earlier with requiring the "istio" ALPN for Permissive that obviously isn't being sent by Prom.

Its using strict mode. My understand is it will only scrape pods with this istio-mtls label which does not exist as far as I know. I'll send you my prom instance

@howardjohn The pods are scraped even when the value=istio-mtls is not present, since it was never present. There is something else wrong in your setup.
I run my tests with prom app scraping enabled from time to time. Like @douglas-reid mentions, permissive mode does not work anymore.
Have you added scrape annotation to your deployment?
You need to change prometheus_scrape: true in value.yaml.

I misunderstood what was going on. On my cluster i have 60 pods in the k8s-service-monitor (plain text) that all fail. I thought this was from tls, but its the client which has no sidecar. There are 0 in secure.
Adding the scrape label to a pod with sidecar now shows up in the secure monitor, but it fails with "received HTTP response to HTTPS request".
How does it know to scrape it with plaintext or tls? It seems the select is the same for both? https://github.com/istio/installer/blob/72b94963d4fe3e37ef713c3d856745835ce46547/istio-telemetry/prometheus-operator/templates/servicemonitors.yaml#L129-L130

Received http response for https means

Permissive mode
Endpoint with some ports work and some do not

I think this is fixed. @lei-tang


[AAP-5991] Add API permission checks

Modify test suite to make permission checks testing easier.
Add permission checks and update tests for existing API endpoints.
Configure app fixture for API tests to authenticate as admin user.
Move test_create_rulebook test case to respective tests file.
Move admin_user fixture to API's conftest.py.
Add check_permission_spy fixture to test permission checks.
Add new resource types to sample initial_data.yml

Had to re-open this PR, to fix the incorrect issue number in the branch name.


25386 Explained that debug in class Engine defaults to settings.debug

Ticket 25386 pointed out that options.debug defaults to settings.DEBUG, but the documentation for configuring a class engine and TEMPLATE_DEBUG both claim that they default to False without elaborating on that point. I added a line to the section of documentation configuring a class engine explaining this.
I did not change the TEMPLATE_DEBUG documentation, because it is deprecated. I will explain how this affects options debug in another pull request.

I think it would be better to include a clearer message under the "Configuring an engine" header that if you are using the DjangoTemplates backend, this isn't the documentation that you're look for, and that any defaults mentioned below are overridden by what's passed by DjangoTemplates. What do you think?

Created my proposal in https://github.com/django/django/pull/5347. Feel free to leave feedback there, thanks!

@timgraham Thanks for your suggestion!  I've had a lot of deadlines this last week, and this took a back burner. I hope they delay did not cause you inconvenience.

Not a problem, thanks for the review!


App 1 - Support Agent Bot not working

Installed genai-stack in Ubuntu 22.04. The App3 - Q/A with a local PDF worked great with one test case. The App2 - Loader worked and the neo4j graph was generated. However, the App 1 did not appear to work. Only the Select RAG mode section was displayed and  the buttons seemed to work. However, no other info was displayed. So could not (or did not know how to) do the Q&A part.

Thank you Stefan for the suggestion. How did you get the display as shown in your image.png file? I could only see the small bottom section for "Select RAG mode", and the rest of the page was just blank. I certainly did not have a good understanding of the sequence of actions needed to have the multiple apps/containers work together. I would greatly appreciate for more detailed instructions on how to operate the apps in order to get the desired results.

If you're in dark mode you first need to switch to light mode (see my comment above).

Thank you. I was in light mode. I got the tiny grey area above the Select RAG mode section as a kind of separator in the initial screen after the App 1 was launched. However, as I tried to go through the rest of the App 1 demo, the display did not change to show different content such as ticket generation. So, I thought that I might not have done it correctly procedure-wise.

I've been able to run the Support Bot demo using App 5 - Static Front End without having to change any code. The App 1 with Streamlit UI still did not work for me.

I've been able to run the Support Bot demo using App 5 - Static Front End without having to change any code. The App 1 with Streamlit UI still did not work for me.

I had the same issue as you and tried the following step and then the App1 display becomes normal:
Search "Streamlit UI" in bot.py file, and comment out the markdown style like this:
Streamlit UI
styl = f"""

"""
st.markdown(styl, unsafe_allow_html=True)
Run "docker compose up --build" in your terminal.

Thank you for posting your workaround. I have posted a different approach in my recent response to Issue #144 'Streamlit Text Box Covered by "Select RAG mode"'.

Ran into the same issue. Worked around the problem by giving the fixed position to the chatbox, instead of the "Select RAG mode" pane. This is done by modifying the CSS style for Streamlit UI in bot.py file: 1) commenting out the "position: fixed" line for the '.element-container:has(aria-lable="Select RAG mode"' element, and 2) adding "position: fixed" line for the '.stChatFloatingInputContainer' element.
With these changes, the chatbox takes a fixed position at the bottom of the page, while the "Select RAG mode" pane becomes a floating element. So, the chatbox is no longer covered by the "Select RAG mode". This workaround allowed me to continue with the demo app successfully.
--- Streamlit UI style sheet modification in bot.py file ---
Streamlit UI
styl = f"""


play WARN alsa: can't encode 0-bit Unknown or not applicable

OS: Elementary OS 5.1 Hera (Ubuntu 18.04 base)
When running the script, alsa gives the error message of "play WARN alsa: can't encode 0-bit Unknown or not applicable_".

I fixed this issue with this pull request: https://github.com/fohtla/Fallout3Terminal/pull/15
Unfortunately the author seems to have lost interest?


Upgrade to Font Awesome Version 5

I would like to have font awesome version 5 installed within the following file path for additional icons:
MagicMirror/vendor/node_modules/
This would also need to have the MagicMirror/vendor/vendor.js file updated to reflect the upgrade as well:
var vendor = {
"moment.js" : "node_modules/moment/min/moment-with-locales.js",
"moment-timezone.js" : "node_modules/moment-timezone/builds/moment-timezone-with-data.js",
"weather-icons.css": "node_modules/weathericons/css/weather-icons.css",
"weather-icons-wind.css": "node_modules/weathericons/css/weather-icons-wind.css",
"font-awesome.css": "node_modules/font-awesome_5/css/all.css",
"nunjucks.js": "node_modules/nunjucks/browser/nunjucks.min.js"
};

I'm not sure how this would affect existing modules. Any idea?

I have needed to use the upgraded font awesome symbol specification in my calendar module shown below:
{ 						symbol: "fas fa-calendar-alt", 						url: "webcal://www.calendarlabs.com/templates/ical/US-Holidays.ics",                         color: "#ffffff" 					},
The default modules I use have not been affected: Clock, Current Weather, Weather Forecast and News Feed. Unsure of the effects of the Compliments, Hello World or Alert modules.

It seems that fa5 isn't backwards compatible to fa4. This means that if we upgrade it, some icons stop working for mirrors that are currently configured. What might be an option, is to make the version of font awesome (in the calendar) configurable, with fa4 as the default.

Doing some testing, it looks like we could have both FA4 and FA5 icons available by adding another line to the MagicMirror/vendor/vendor.js file:
var vendor = {
"moment.js" : "node_modules/moment/min/moment-with-locales.js",
"moment-timezone.js" : "node_modules/moment-timezone/builds/moment-timezone-with-data.js",
"weather-icons.css": "node_modules/weathericons/css/weather-icons.css",
"weather-icons-wind.css": "node_modules/weathericons/css/weather-icons-wind.css",
"font-awesome.css": "node_modules/font-awesome_4/css/font-awesome.min.css",
"font-awesome5.css": "node_modules/font-awesome_5/css/all.css",
"nunjucks.js": "node_modules/nunjucks/browser/nunjucks.min.js"
};
Then adding this into the MagicMirror/modules/default/calendar/calendar.js file:
getStyles: function () {
	return ["calendar.css", "font-awesome.css", "font-awesome5.css"];
},

This would give the user the ability to select symbols from either FA4 or FA5 within the config.js file, just formatting them correctly as shown below:
{
    symbol: "calendar", //FA4 format
    symbol: "fas fa-baseball-ball", //FA5 format
},

Sounds nice. Can you send a PR to the dev branch? Dont forget the updated changelog. :)

Closing dure to inactivity. Feel free to reopen or resubmit a PR.


G8 and g9 characters have crippled hands and feet when imported into unreal

The hands and feet of g8 and g9 characters are crippled and bent when importing into UE5.3 with zero pose mode . Any fix for this?

(1) Transfer Animation into UE: I found the hands are ok but feets are deformed, perhaps because of heels?
(Genesis 8.1, UE 5.1 Daz 4.22, newest plugin)
Transfer Model (Skeleton Mesh) into UE is fine.
(2) Genesis 8 Animation Re-target does not work (retarget from UE 5.1 third-person template animations), the re-targeted animations are deformed.
Old version(update from Daz Install Manager): retarget works fine,  but animation does not...


On Windows install .dll libraries in /bin

On Windows it is important to install the .dll libraries (RUNTIME in CMake jargon) in <prefix>/bin, as otherwise the Windows dynamic loader is not able to find them.
See https://github.com/ros-planning/moveit/pull/1575 for details on this.

Thanks @rhaschke !


Question about byte alignment

In your document, Core Animation can work with a image which has been made byte alignment directly without having to create a copy.
I found that by we can get a byte alignment image by setting the right bytesPerRow para of CGImageCreate function.
My question is that should we call "CGImageCreate" every time when we load a image from disk?
Or after I get a byte alignment image in memory and save it to disk, the next time if I just call the function "initWithContentOfFile", can I get the byte alignment image?

You're asking about implementation details of UIImage. Not sure what the answer is. Maybe ask on an Apple mailing list?


UAT | Content in Payment for E-filing

The Payment modal has a note which mentions it is possible to pay offline as well
The content of the same needs to be updated as per design
Design:  https://www.figma.com/design/fwFYdlymVjggLOltqnXWsY/Hand-off-(Digit)?node-id=179-33277&t=LEbLxBNkKETYHzmQ-0

Issue is fixed, please find the reference


Delete incremental directory

cargo sweep -r path leaves the target/<debug>/incremental directory as-is, which can amass a lot of redundant builds. I would expect sweep to clean this directory too.

My understanding is that rustc does a pretty good job of cleaning up files in that directory even across versions. I think that if there are files that are not reliably getting cleaned up that can be reported on the rust issue tracker mentioning incremental. I do know that Cargo has no tracking we can use about the files in that folder.
Do you have a suggestion for how you would like to see that cleaned?

With the -t switch, it should be simple enough to just look at the file times.

I think this is relevant: cargo ./target fills with outdated artifacts as toolchains are updated/changed  #5026
Build files across toolchains are not automatically cleaned, and this includes the incremental directory.

Do you have a suggestion for how you would like to see that cleaned?

@Eh2406 I had to reverse engineer how cargo-sweep works from the code, so I might be wrong, but my understanding of how it works today is:

It looks at all .json files in target/debug/.fingerprint
It stores the rustc version hash in the file.
Remove all files that don't have a selected version hash. For example, if I pass --toolchains nightly and there's a target/debug/.fingerprint/unicode-ident-e4c3093778eadcfe/lib-unicode-ident.json file with the rustc hash of nightly, it will keep target/debug/deps/unicode_ident-e4c3093778eadcfe.d but not target/debug/deps/winapi-7507b8d119f8d10f.d.

The problem with incremental is that there's no file in the .fingerprint directory that corresponds to that hash; for example I currently have a target/debug/incremental/cargo_sweep-1sor5mg0j6ouy directory, but there's no target/debug/.fingerprint/cargo_sweep-1sor5mg0j6ouy/lib-cargo_sweep.json file.

Ok, assuming all that's true, I think my preferred solution is for cargo to start emitting those .json fingerprint files. Is that doable? If not, can it at least include the hash of the compiler version in the incremental directory name?

Your description of how cargo-sweep works sounds correct to my recollection. Sorry I left the code an undocumented mess.

Fundamentally the incremental folder is not maintained by Cargo. The files are created by rustc, and usually cleaned up by rustc. I have no idea what structure rustc uses for organizing these files, or how we could introspect them to dell files that fell through the cracks.

Ah, I see - cargo is only passing -C incremental=/Users/jyn/src/cargo-sweep/target/debug/incremental, it doesn't control the subdirectories.
That's ... quite unfortunate. I suppose cargo could do something like incremental-<rustc version hash> but that's probably the most it can do.

It would be convenient if there was added some way for cargo-sweep to remove these, whether with --all (right now cargo sweep -r -t 0 still keeps dozens of GB of incremental folders on my machine) or with a new separate option.
One workaround I got working is to run:
fd -I incremental -x rm -R

fd -I incremental -x rm -R

Thanks, this was a good suggestion.
However I found it was matching some files with 'incremental' in the name (surprisingly common) so here is my extra safe and verbose version:
fd -I "target/.*/incremental$" --type directory --full-path

Try a dry run, then add -x rm -R to do the deletion


pdf is truncated to first page

Configuration.pdf
Each pdf was truncated to the first page.
When I combine them into one pdf, I get a multi-page pdf that includes the first page of each individual pdf.  The html pages are rendered with all the content of the markdown.  The pdf renders beautifully... it just stops at the end of page 1.

I was able to get multiple pdf pages to render when I removed this from my mkdocs.yml file:
theme: readthedocs

this plugin only support the material theme. see here.
You will have to add support for other themes yourself, see here


Update @context of all Agri Entities to point to FIWARE @context

The LD @context provided by GSMA is incomplete as it does not include the Entity Type. So the best option is to point to the FIWARE @context as we do in other examples.

fixed


fixes #309 example in readme and makes a more complete example

See issue here #309
https://github.com/livekit/python-sdks/issues/309
This fixes the readme example and adds a more complete example so users don't waste time trying to use an invalid example.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Chris Wilson seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


[firebase_messaging] can't find com.google.firebase.messaging.FirebaseMessagingService (6.0.9)

in android ,com.google.firebase.messaging.FirebaseMessagingService

Running Gradle task 'assembleDebug'...
无法访问FirebaseMessagingService
FlutterFirebaseMessagingService.setPluginRegistrant(this);
^
找不到com.google.firebase.messaging.FirebaseMessagingService的类文件
1 个错误
FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':app:compileDebugJavaWithJavac'.

Compilation failed; see the compiler error output for details.

Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

Get more help at https://help.gradle.org

BUILD FAILED in 17s
Finished with error: Gradle task assembleDebug failed with exit code 1

[✓] Flutter (Channel stable, v1.12.13+hotfix.5, on Mac OS X 10.14.6 18G1012, locale zh-Hans-US)
• Flutter version 1.12.13+hotfix.5 at /Users/jackcui/Desktop/flutter
• Framework revision 27321ebbad (6 days ago), 2019-12-10 18:15:01 -0800
• Engine revision 2994f7e1e6
• Dart version 2.7.0

any update?

Hey all 👋
As part of our roadmap (#2582) we've just shipped a complete rework of the firebase_messaging plugin that aims to solve this and many other issues.
If you can, please try out the dev release (see the migration guide for upgrading and for changes) and if you have any feedback then join in the discussion here.
Given the scope of the rework I'm going to go ahead and close this issue in favor of trying out the latest plugin.
Thanks everyone.


[Impeller] Concurrent VkPipeline creation is slow on Mokey Device.

Impeller will create min(4, hardware_concurrency) threads to boot strap VkPipelines after creating the context_vk. On the Mokey deivce, using 4 threads is actually much slower than using 1.  The Mokey is only a 4 core device so at a minimum we should reduce the max concurrency to something like hardware_concurrency / 2.
4 thread (~3 seconds)

1 thread ( ~ 2 seconds)

We should figure out how to adjust the concurrency level based on device performance.

We could potentially add a "lazy bootstrap" setting, though we'd need to make sure the context can handle shaders with no immediate default.

A potentially better solution:

one pipeline cache per thread. Not externally sychronized, no internal synchronization.
Use https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/vkMergePipelineCaches.html at the end.

FYI @chinmaygarde  ^

Another option for speeding this up that jonah pointed out is that vkCreateGraphicsPipelines can take in multiple arguments.  Currently it is called once for each pipeline we create.
The vulkan spec states:

Applications can group together similar pipelines to be created in a single call, and implementations are encouraged to look for reuse opportunities when creating a group.

That means there may be savings that can only be achieved with batching vkCreateGraphicsPipelines.  It's plausible that creating a triangle and a triangle strip variant of a pipeline may have overlapping calculations that can be reused.
Letting the driver manage parallelism may lead to worse performance on particular devices though.

Here is a profile that Jia Hao did too.  We can see that while we have 2 worker threads, they are mostly operating serially.

Here is a doc with a class diagram and a sequence diagram for pipeline creation for background information to help understand how this works today: https://docs.google.com/document/d/1MzL_U4M63BEyg35asxksZ-fQTi0gLBHiCO48kVXXMe8/edit#heading=h.l5nbn6yk543e

I think I initially misunderstood @jonahwilliams suggestion about the pipeline caches per thread that are eventually merged. I was incorrectly operating under the assumption that this would need VK_PIPELINE_CACHE_CREATE_EXTERNALLY_SYNCHRONIZED_BIT from VK_EXT_pipeline_creation_cache_control. But the synchronization overhead isn't the issue, its the actual contention. So I do think that a fine start and should give us an immediate improvement on the Mokey. On subsequent runs, we can initialize the TLS caches using the same merged blob and then re-merge before saving back to disk.
@gaaclarke s suggestion is interesting and I haven't tried if it actually improves things. I did try VK_PIPELINE_CREATE_DERIVATIVE_BIT for the pipeline variants but that didn't move the needle at all. So it may depend on the driver. But, it should be possible to build this on top of @jonahwilliams suggestion.

It looks like every time we create a pipeline we also allocate a new render pass (see renderPass in  https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VkGraphicsPipelineCreateInfo.html).
Another optimization that we could do is to use the same render pass when allocating all the default pipelines.  That field seems to be used for setup and as long as all the render passes are the same setup, they should be equivalent.

I profiled on a pixel 4a and I don't see the same serialization problem happening.  Creating the pipelines is taking up 550ms.

https://ui.perfetto.dev/#!/?s=3a3959a7cc3a6e2f315c6e7b95414b2090f3b360ccd80fd6f0487186655ed23d
That seems to indicate that the serialization is happening in the driver.
Profiled at 259bc4b44987f0cc14a24e84c273dd3898fd2212

As an experiment I turned off the cache and it ended up taking 2x as long.
--- a/impeller/renderer/backend/vulkan/pipeline_cache_vk.cc
+++ b/impeller/renderer/backend/vulkan/pipeline_cache_vk.cc
@@ -121,7 +121,7 @@ vk::UniquePipeline PipelineCacheVK::CreatePipeline(
   }

   auto [result, pipeline] =
-      strong_device->GetDevice().createGraphicsPipelineUnique(*cache_, info);
+      strong_device->GetDevice().createGraphicsPipelineUnique(nullptr, info);
   if (result != vk::Result::eSuccess) {
     VALIDATION_LOG << "Could not create graphics pipeline: "
                    << vk::to_string(result);
@@ -137,7 +137,7 @@ vk::UniquePipeline PipelineCacheVK::CreatePipeline(
   }

   auto [result, pipeline] =
-      strong_device->GetDevice().createComputePipelineUnique(*cache_, info);
+      strong_device->GetDevice().createComputePipelineUnique(nullptr, info);
   if (result != vk::Result::eSuccess) {
     VALIDATION_LOG << "Could not create compute pipeline: "
                    << vk::to_string(result);

profile: https://ui.perfetto.dev/#!/?s=d803805446dbf6e7c4ebaae9467acf98036700a8b486e27b59b594765d958fc4

Okay, I was able to get batching working.  I batched up a handful of pipelines, whichever were convenient to batch.  The results are negative.  It looks like it takes the same amount of time but the bigger batch size leads to worse parallelization where ultimately all threads were left waiting on a single batch job to finish.

https://ui.perfetto.dev/#!/?s=7dd410c583a55a8fb77c4d6ee5f3996b14491a00c2b56d61f9b81be11b3ebfe2

Is there a trace for the version captured earlier in #143540 (comment)?

It's in this internal doc: https://docs.google.com/document/d/1n7im-6EaeRYlerbuBtJayg2zWEDjgVMXuJx7elpT0JU/edit?tab=t.0#heading=h.ybl0gfg8pr0r

It looks like your version is getting core utilization.

Yea, I was seeing that on the pixel 4a before this change though too.  It is possible that batching does make a difference with the mokey.  I'm not sure we'd want to accept something that regresses other devices though.

I'm not sure we'd want to accept something that regresses other devices though.

Yeah. I think squeezing the pipeline counts seems like a better way to go. Did we try the per thread pipeline cache with subsequent merge?

Yeah. I think squeezing the pipeline counts seems like a better way to go. Did we try the per thread pipeline cache with subsequent merge?

Not yet.  I can try that out starting tomorrow.  I need a little breather from this.

I have a few more clean ups that will reduce shader count, namely the removal of points.comp (which doesn't provide much speedup over the CPU computation right now), the removal of uv.comp, and the removal of position_color.vert and vertices.frag.
We could:

Combine glyph_atlas.frag and glyph_atlas_color.frag (not working on this now).
Go back to using the solid_fill shader for clip.frag.
Find a way to do the texture_fill_strict_src math on the CPU.
Remove KernelDecal/NoDecal on platforms that only need 1 due to natively supported decal.

Beyond that, we could explore lazy initialization for certain infrequently used shader variants. This has a risk of introducing jank though.

@gaaclarke when I'm at the office tomorrow I'll try your patch on a mokey. If its negative there I say we probably have to abandon the effort.

Merge pipeline caches design sketch
Implementing the merge behavior is going to be tricky:

There needs to be a mechanism to switch over worker threads to the new merged
cache.  In order to preserve paralellism, it would have to be a
stop-the-world mechanism for the worker threads.
Using a pipeline cache will now dynamically have to decide if a mutex is
necessary, as is the case for the merged cache, or not.

It will have to look something like this:
static PipelineCache merged_cache;
static Mutex merged_mutex;
static atomic<bool> did_merge = false;
static thread_local PipelineCache tls_cache;
static vector<PipelineCache*> caches;
static Mutex caches_mutex;

void ContentContext::InitializeDefaultPipelines() {
  foo_pipeline_.CreateDefault();
  bar_pipeline_.CreateDefault();
  //...

  shared_ptr<atomic<int>> counter(worker_pool_.count());
  // `ForEachWorker` doesn't exist but posts the same task to all workers.
  worker_pool_.ForEachWorker([counter]{
    if (counter->fetch_sub(1) == 0) {
      // Now we know that all default pipelines have finished.

      // `PostStopTheWorld` doesn't exist today but blocks all threads except
      // one who executes.
     worker_pool_->PostStopTheWorld([]{
        MergePipelines();
      });
    }
  });
}

ContentContext::~ContentContext() {
  worker_pool_.ForEachWorker([]{
    tls_cache.reset();
  });
}

Pipeline PipelineCacheVK::CreatePipeline(PipelineCreateInfo info) {
  if (did_merge.load()) {
    Lock lock(merged_mutex);
    return device_->CreatePipeline(merged_cache, info);
  }
  if (!tls_cache) {
    tls_cache = device_->CreatePipelineCache();
    Lock lock(caches_mutex);
    caches.push_back(&tls_cache);
  }
  return device_->CreatePipeline(tls_cache, info);
}

void PipelineCacheVK::MergePipelines() {
  // No need for caches_mutex or merged_mutex since this is executed in
  // StopTheWorld.
  merged_cache = device_->MergePipelines(caches);
  did_merge.store(true);
}

Explored alternatives:

Don't use StopTheWorld -- This resulted in serializing usage of tls_cache
because we had to make sure they weren't used while a merge was happening.

I think the TLS cache is likely not worth pursuing. Since the cacheless initial bootstrap had the same or worse performance, its likely that we're serialized in the driver shader compilation and not in cache read/write.

I think the TLS cache is likely not worth pursuing. Since the cacheless initial bootstrap had the same or worse performance, its likely that we're serialized in the driver shader compilation and not in cache read/write.

There was the hypothesis that the worse performance without a cache was because the cache provides a speed boost when creating subsequent pipelines.  As a quick check we could try profiling with just a tls cache, but no merge.

Yeah seems like a reasonable idea

Thread local caches without merges were slower too on a pixel4a:
https://ui.perfetto.dev/#!/?s=2bec2aca8611ae2e454d1c0043ad1a8b7cfd41091b50ddd53453da6a719f1df2

diff --git a/impeller/entity/contents/content_context.cc b/impeller/entity/contents/content_context.cc
index 1a7a8a9b02..dd93f2f8e0 100644
--- a/impeller/entity/contents/content_context.cc
+++ b/impeller/entity/contents/content_context.cc
@@ -266,6 +266,11 @@ ContentContext::ContentContext(
     return;
   }

+  {
+    TRACE_EVENT0("flutter", "WaitingForPipelines");
+    sleep(15);
+  }
+
   auto options = ContentContextOptions{
       .sample_count = SampleCount::kCount4,
       .color_attachment_pixel_format =
diff --git a/impeller/renderer/backend/vulkan/pipeline_cache_vk.cc b/impeller/renderer/backend/vulkan/pipeline_cache_vk.cc
index 49ee504e5b..fc9af4052c 100644
--- a/impeller/renderer/backend/vulkan/pipeline_cache_vk.cc
+++ b/impeller/renderer/backend/vulkan/pipeline_cache_vk.cc
@@ -13,45 +13,74 @@

 namespace impeller {

+static thread_local vk::UniquePipelineCache tls_cache;
+static std::mutex s_caches_mutex;
+static std::vector<vk::PipelineCache*> s_caches;
+static vk::UniquePipelineCache s_merged_cache;
+static std::mutex s_merged_cache_mutex;
+
+static vk::PipelineCache& GetPipelineCache(
+    const std::shared_ptr<DeviceHolderVK>& device_holder) {
+  if (!tls_cache) {
+    vk::PipelineCacheCreateInfo cache_info;
+    cache_info.flags = vk::PipelineCacheCreateFlagBits::eExternallySynchronized;
+
+    auto [result, cache] =
+        device_holder->GetDevice().createPipelineCacheUnique(cache_info);
+    FML_CHECK(result == vk::Result::eSuccess);
+    {
+      std::scoped_lock lock(s_caches_mutex);
+      s_caches.push_back(&cache.get());
+    }
+    tls_cache = std::move(cache);
+  }
+  return tls_cache.get();
+}
+
+void MergePipelineCaches() {
+  std::scoped_lock lock(s_caches_mutex);
+
+}
+
 static constexpr const char* kPipelineCacheFileName =
     "flutter.impeller.vkcache";

-static bool VerifyExistingCache(const fml::Mapping& mapping,
-                                const CapabilitiesVK& caps) {
-  return true;
-}
+// static bool VerifyExistingCache(const fml::Mapping& mapping,
+//                                 const CapabilitiesVK& caps) {
+//   return true;
+// }

 static std::shared_ptr<fml::Mapping> DecorateCacheWithMetadata(
     std::shared_ptr<fml::Mapping> data) {
   return data;
 }

-static std::unique_ptr<fml::Mapping> RemoveMetadataFromCache(
-    std::unique_ptr<fml::Mapping> data) {
-  return data;
-}
-
-static std::unique_ptr<fml::Mapping> OpenCacheFile(
-    const fml::UniqueFD& base_directory,
-    const std::string& cache_file_name,
-    const CapabilitiesVK& caps) {
-  if (!base_directory.is_valid()) {
-    return nullptr;
-  }
-  std::unique_ptr<fml::Mapping> mapping =
-      fml::FileMapping::CreateReadOnly(base_directory, cache_file_name);
-  if (!mapping) {
-    return nullptr;
-  }
-  if (!VerifyExistingCache(*mapping, caps)) {
-    return nullptr;
-  }
-  mapping = RemoveMetadataFromCache(std::move(mapping));
-  if (!mapping) {
-    return nullptr;
-  }
-  return mapping;
-}
+// static std::unique_ptr<fml::Mapping> RemoveMetadataFromCache(
+//     std::unique_ptr<fml::Mapping> data) {
+//   return data;
+// }
+
+// static std::unique_ptr<fml::Mapping> OpenCacheFile(
+//     const fml::UniqueFD& base_directory,
+//     const std::string& cache_file_name,
+//     const CapabilitiesVK& caps) {
+//   if (!base_directory.is_valid()) {
+//     return nullptr;
+//   }
+//   std::unique_ptr<fml::Mapping> mapping =
+//       fml::FileMapping::CreateReadOnly(base_directory, cache_file_name);
+//   if (!mapping) {
+//     return nullptr;
+//   }
+//   if (!VerifyExistingCache(*mapping, caps)) {
+//     return nullptr;
+//   }
+//   mapping = RemoveMetadataFromCache(std::move(mapping));
+//   if (!mapping) {
+//     return nullptr;
+//   }
+//   return mapping;
+// }

 PipelineCacheVK::PipelineCacheVK(std::shared_ptr<const Capabilities> caps,
                                  std::shared_ptr<DeviceHolderVK> device_holder,
@@ -65,39 +94,41 @@ PipelineCacheVK::PipelineCacheVK(std::shared_ptr<const Capabilities> caps,

   const auto& vk_caps = CapabilitiesVK::Cast(*caps_);

-  auto existing_cache_data =
-      OpenCacheFile(cache_directory_, kPipelineCacheFileName, vk_caps);
-
-  vk::PipelineCacheCreateInfo cache_info;
-  if (existing_cache_data) {
-    cache_info.initialDataSize = existing_cache_data->GetSize();
-    cache_info.pInitialData = existing_cache_data->GetMapping();
-  }
-
-  auto [result, existing_cache] =
-      device_holder->GetDevice().createPipelineCacheUnique(cache_info);
-
-  if (result == vk::Result::eSuccess) {
-    cache_ = std::move(existing_cache);
-  } else {
-    // Even though we perform consistency checks because we don't trust the
-    // driver, the driver may have additional information that may cause it to
-    // reject the cache too.
-    FML_LOG(INFO) << "Existing pipeline cache was invalid: "
-                  << vk::to_string(result) << ". Starting with a fresh cache.";
-    cache_info.pInitialData = nullptr;
-    cache_info.initialDataSize = 0u;
-    auto [result2, new_cache] =
-        device_holder->GetDevice().createPipelineCacheUnique(cache_info);
-    if (result2 == vk::Result::eSuccess) {
-      cache_ = std::move(new_cache);
-    } else {
-      VALIDATION_LOG << "Could not create new pipeline cache: "
-                     << vk::to_string(result2);
-    }
-  }
-
-  is_valid_ = !!cache_;
+  // auto existing_cache_data =
+  //     OpenCacheFile(cache_directory_, kPipelineCacheFileName, vk_caps);
+
+  // vk::PipelineCacheCreateInfo cache_info;
+  // if (existing_cache_data) {
+  //   cache_info.initialDataSize = existing_cache_data->GetSize();
+  //   cache_info.pInitialData = existing_cache_data->GetMapping();
+  // }
+
+  // auto [result, existing_cache] =
+  //     device_holder->GetDevice().createPipelineCacheUnique(cache_info);
+
+  // if (result == vk::Result::eSuccess) {
+  //   cache_ = std::move(existing_cache);
+  // } else {
+  //   // Even though we perform consistency checks because we don't trust the
+  //   // driver, the driver may have additional information that may cause it
+  //   to
+  //   // reject the cache too.
+  //   FML_LOG(INFO) << "Existing pipeline cache was invalid: "
+  //                 << vk::to_string(result) << ". Starting with a fresh
+  //                 cache.";
+  //   cache_info.pInitialData = nullptr;
+  //   cache_info.initialDataSize = 0u;
+  //   auto [result2, new_cache] =
+  //       device_holder->GetDevice().createPipelineCacheUnique(cache_info);
+  //   if (result2 == vk::Result::eSuccess) {
+  //     cache_ = std::move(new_cache);
+  //   } else {
+  //     VALIDATION_LOG << "Could not create new pipeline cache: "
+  //                    << vk::to_string(result2);
+  //   }
+  // }
+
+  is_valid_ = true;
 }

 PipelineCacheVK::~PipelineCacheVK() {
@@ -121,7 +152,8 @@ vk::UniquePipeline PipelineCacheVK::CreatePipeline(
   }

   auto [result, pipeline] =
-      strong_device->GetDevice().createGraphicsPipelineUnique(*cache_, info);
+      strong_device->GetDevice().createGraphicsPipelineUnique(
+          GetPipelineCache(strong_device), info);
   if (result != vk::Result::eSuccess) {
     VALIDATION_LOG << "Could not create graphics pipeline: "
                    << vk::to_string(result);
@@ -137,7 +169,8 @@ vk::UniquePipeline PipelineCacheVK::CreatePipeline(
   }

   auto [result, pipeline] =
-      strong_device->GetDevice().createComputePipelineUnique(*cache_, info);
+      strong_device->GetDevice().createComputePipelineUnique(
+          GetPipelineCache(strong_device), info);
   if (result != vk::Result::eSuccess) {
     VALIDATION_LOG << "Could not create compute pipeline: "
                    << vk::to_string(result);
@@ -154,8 +187,8 @@ std::shared_ptr<fml::Mapping> PipelineCacheVK::CopyPipelineCacheData() const {
   if (!IsValid()) {
     return nullptr;
   }
-  auto [result, data] =
-      strong_device->GetDevice().getPipelineCacheData(*cache_);
+  auto [result, data] = strong_device->GetDevice().getPipelineCacheData(
+      GetPipelineCache(strong_device));
   if (result != vk::Result::eSuccess) {
     VALIDATION_LOG << "Could not get pipeline cache data to persist.";
     return nullptr;

We've reduced the shader count as much as we can (~50 down to ~30) and moved bootstrapping as early as possible. Closing as fixed (as well as it can be).


Backport "Remove artificial CURSOR added to code in the completions" to LTS

Backports #20899 to the 3.3.5.
PR submitted by the release tooling.
[skip ci]

No regressions detected in the community build up to backport-lts-3.3-21857.
Reference


[question] HystrixGatewayFilterFactory retry

Question
HystrixGatewayFilterFactory with RetryGatewayFilterFactory Is it possible to retry the service on the basis of Hystrix, or how to retry the service on the gateway?

Do you want to retry a request when Hystrix times out?

emmmmmm.....yes

Generally, we recommend retry within the circuit rather than outside.

Spring cloud gateway Is it possible to configure the Ribbon to retry, and if so, what should I do?

Gateway doesn't use ribbon for retry. See http://cloud.spring.io/spring-cloud-static/Finchley.SR1/multi/multi__gatewayfilter_factories.html#_retry_gatewayfilter_factory

Thank you for your busy schedule to reply to me, because we did not find a timeout setting in the RetryGatewayFilterFactory provided by spring cloud gateway, we did not use it.

Retry does not have a timeout associated with it.

@spencergibb Set the retry for N times to trigger hystrix invalid？

Using the retry filter

Generally, we recommend retry within the circuit rather than outside.

Hystrix configuration will cause retry to fail

Put hystrix before retry

Thank you, I didn't think about the reason for the loading order.

Closing as answered


Updates to opentelemetry JS packages used in the website break netlify

This happened before: We update the opentelemetry-*-js packages used in tracing.js, and since tracing.js is disabled in the pre-prod built we do not catch errors. Currently builds are affected by that, see
https://app.netlify.com/sites/opentelemetry/deploys/65e9d0e4cf5af70008c332a1

3:37:24 PM: Error: error building site: JSBUILD: failed to transform "js/tracing.js" (text/javascript): "/opt/build/repo/assets/js/tracing.js:10:9": No matching export in<EMAIL_ADDRESS>for import "SEMRESATTRS_SERVICE_NAME"

There are 2 things we need to fix:

The current problem: updating the code with the changes to semconv
The general problem: figure out how we can enable tracing.js safely for pre-prod, to verify that this does not happen, maybe we can influence the used exporter instead of disabling tracing by not including tracing.js ?

Closing this now, if this happens again we might want to revisit to address the general problem


403 error when I want to send form

I have custom form(with custom php, built on phpmailer) when I added my template to batflat, I've got problem with form redirections. When attachment is too big it should  redirect to my custom html error page.  Instead of this I have error 403,  without batflat it works fine.

HI @jaroslav-mor ,
You could possibly fix this by implementing a quick filesize check on client-side using JavaScript if you know the max upload availability. Just update the form HTML accordingly referencing the example below:
<input type="file" id="file">


Add health checks for DynamoDB, SQS, and LDAP

Background
Presently, we only health check that we can talk to MySQL.  This is done via the /health HealthRoute.  We will not be system aware of an outage on other dependencies.
Solution
Use drop wizard metrics health checking to check all dependencies.  This should be in the module itself (MySQL, DynamoDB).
Will need to add LDAP authentication checks to the portal.
Notes
https://metrics.dropwizard.io/3.1.0/manual/healthchecks/
If we run health checks on demand (by the Load Balancer), it maybe too frequent.
I would recommend having a daemon service that polls those health checks once every few seconds, and have the HealthRoute just ping that daemon.  Can use FS2 and things like Ref for concurrently setting and fetching state.

#325 did databases, working on queues now


Typo under "Create a virtual network" Step #6

Typo under "Create a virtual network"
Bookmark: https://learn.microsoft.com/en-us/azure/dns/dns-private-resolver-get-started-portal#create-a-virtual-network
Step #6
Bullet #2
Typo: "IPv4 address range: <IP_ADDRESS>.16" <-- (last dot must be a forward slash "/")
Correction: "IPv4 address range: <IP_ADDRESS>/16"

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 67376353-3eff-47d3-3e52-13600075ebd4
Version Independent ID: 88cd7c90-2a27-a389-a751-99b5dc37120e
Content: Quickstart - Create an Azure DNS Private Resolver using the Azure portal
Content Source: articles/dns/dns-private-resolver-get-started-portal.md
Service: dns
GitHub Login: @greg-lindsay
Microsoft Alias: greglin

@rene-pjd
Thanks for your feedback! We will investigate and update as appropriate.

@rene-pjd
Thanks for reporting this! We have created a PR for this issue and the changes should go live soon.


The operator should support related images

What would you like to be added:
If the operator is deployed using a bundle, and that bundle specifies related images to use, those images should be used instead of the defaults or those specified in the Submariner CR.
Why is this needed:
This is needed for air-gapped deployments: images are provided using a local registry, and pointers to those images are provided as related images in the corresponding bundle.
How to implement this:
Related images are made available to the operator as environment variables, e.g. RELATED_IMAGE_submariner-gateway. See https://github.com/submariner-io/submariner-operator/pull/2240/files#diff-1bf04c286473ed674abe64d3d8de769ee5e499ad02d4f8a709b856d3e1aad3a1 for an example of a full bundle with related images.

This is already implemented, see https://github.com/submariner-io/submariner-operator/pull/1702.


Need an option for users to use the client lib only

This repo contains both a server library and a client library to share policy/requirement based authorization.
Certain users might only want to use the client library - for example the server stack might not be using asp.net core.
Providing an option for users to use the client library standalone will maximize the potential benefits.
One potential solution is to let user provide their AuthZyinContext object as part of the initializeAuthZyinContext call.

This is done with this commit.
If a context object is passed to initializeAuthZyinContext, the library will use it as is and don't make any calls to the api provided by the server lib, like below:
initializeAuthZyinContext(context);

You'll have to construct your own AuthZyinContext object on the client side with user context, policies/requirements though.


add<EMAIL_ADDRESS>to compat

Shouldn't require any code changes I think

This would be very nice to have as the<EMAIL_ADDRESS>requirement is blocking the use of AOG for me - as it leads to version downgrades for a lot of other packages (e.g. RxInfer.jl)

Thanks!


[VMR] Component Governance errors tracking

NuGet problems
⚠ NuGet Feed Configuration

[x] fsharp/tests/FSharp.Build.UnitTests/NuGet.Config
[x] sdk/src/Assets/NonRestoredTestProjects/AppThrowingException/AppDependingOnOtherAsTool/NuGet.Config
[x] sdk/src/Assets/TestProjects/TrimmedAppWithReferences/App/nuget.config
[x] sdk/src/Assets/TestProjects/PackageValidationTestProject/NuGet.Config

⚠ NuGet security analysis - Potential upstreams in a feed

Seems like there might be a feed it cannot access so it cannot decide whether it has upstreams (and potentially leads to nuget.org)

[x] nuget-client/Nuget.Config

Plus there are more but once we sync the removal of FileSystem and Common submodules, they will go away.
⚠ CFS0013 - Package source has value that is not an Azure Artifacts feed

Usually, these usually have nuget.org inside. I am not sure how this ties to https://github.com/dotnet/source-build/issues/3170

[ ] format/tests/projects/for_analyzer_formatter/analyzer_project/NuGet.config
[ ] nuget-client/test/EndToEnd/NuGet.Config
[ ] sdk/template_feed/Microsoft.DotNet.Common.ItemTemplates/content/Nuget/nuget.config
I think this one needs an exemption as it's a template for new projects
[x] source-build-externals/src/application-insights/BASE/NuGet.config
[x] source-build-externals/src/application-insights/NETCORE/NuGet.config
[x] source-build-externals/src/application-insights/NuGet.config
[x] source-build-externals/src/newtonsoft-json/Src/NuGet.Config

⚠ CFS0011 - C# project(s) are missing feed configuration

These all seem to be our files from the VMR bootstrap that don't resolve to no root NuGet.config file.

[x] eng/bootstrap/buildBootstrapPreviouslySB.csproj
https://github.com/dotnet/source-build/issues/3230
[x] test/Microsoft.DotNet.SourceBuild.SmokeTests/Microsoft.DotNet.SourceBuild.SmokeTests.csproj
https://github.com/dotnet/source-build/issues/3230
[x] tools-local/tasks/Microsoft.DotNet.SourceBuild.Tasks.XPlat/Microsoft.DotNet.SourceBuild.Tasks.XPlat.csproj
https://github.com/dotnet/source-build/issues/3230
[x] tools-local/tasks/SourceBuild.MSBuildSdkResolver/SourceBuild.MSBuildSdkResolver.csproj
https://github.com/dotnet/source-build/issues/3230

NPM problems
⚠ CFS0001 - Node.js project(s) are missing feed configuration

Missing .npmrc files

[x] aspnetcore/src/submodules/MessagePack-CSharp/src/MessagePack.UnityClient/Assets/RuntimeUnitTestToolkit/package.json
Issue to be created, I reached out to dougbu
[x] aspnetcore/src/submodules/MessagePack-CSharp/src/MessagePack.UnityClient/Assets/Scripts/MessagePack/package.json
Issue to be created, I reached out to dougbu
[x] aspnetcore/src/submodules/spa-templates/src/content/Angular-CSharp/ClientApp/package.json
Issue to be created, I reached out to dougbu
[x] aspnetcore/src/submodules/spa-templates/src/content/React-CSharp/ClientApp/package.json
Issue to be created, I reached out to dougbu

Just for book-keeping, the variables used to suppress the scans are following:

cfsNPMWarnLevel: none
cfsNugetWarnLevel: none
myGetWarnLevel: none
NuGetSecurityAnalysisWarningLevel: none

So, we've got the Source Build NuGet.config without a root config fixed.
We also fixed the source-build-externals ones by cloaking them from the VMR.
I have also dismissed all of the multi-feed AzDO alerts based on https://dev.azure.com/dnceng/internal/_wiki/wikis/DNCEng Services Wiki/843/SingleFeedFAQ

@brettfo we are seeing an alert in the VMR for this file: fsharp/tests/FSharp.Build.UnitTests/NuGet.Config which is synchronized from the fsharp repo.
I checked the history of the dotnet-fsharp repo and the alert was possibly suppressed wrongly. I can see a number of suppressed alerts in CG due to the multi-feed configuration which CG wrongly detects for our repos where we reference several public AzDO feeds with no upstreams (https://dev.azure.com/dnceng/internal/_wiki/wikis/DNCEng Services Wiki/843/SingleFeedFAQ). These suppressions are in place and the right step.
However, this linked one I think is not of the same kind. This one is probably missing the <clear /> statement so it's a valid alert and should be resolved differently. I propose reinstating the alert and fixing the NuGet.config file which will in turn fix this in the VMR.

@brettfo we are seeing an alert in the VMR for this file: fsharp/tests/FSharp.Build.UnitTests/NuGet.Config which is synchronized from the fsharp repo.

Ah, so it seems that we are probably fine to dismiss this as the root NuGet.config should be enough to functionally match copying the required feeds into the offending config.

All critical and high alerts have been resolved so I am closing this now.
I have also set up a weekly email that I get with a report of CG alerts for the VMR.


added mikalcallahan.me

Webring text on right side of footer.

It says "restructuring in process"?
How to I access the site :)

I am doing an iterative development on it but on anything but phone screens there should be a link visible on the footer. More to be updated today! The rest of the site is coming :)

I checked again, it still only says "Reconstruction in progress" with no link to any content whatsoever.


Category for "High-Performance Computing"

I wanted to submit my new research, and found that none of the categories is relevant. Could you please create a "High-Performance Computing" category?

High-Performance Computing added as a category, tnx!


[FHIR] How to ensure that the record in a resource is not duplicated?

hi LinuxForHealth,
How are you doing? I'm learning and implementing the FHIR server. There is a question about duplicated records avoiding. If you can provide a solution or any suggestion, I'll really appreciate it.
How to ensure that the record in a resource is not duplicated?
I know the field ID in a resource is unique and represents a record.
However, I'm looking for any other way to set a field that makes business sense as a unique judgment.
Taking a resource Organization as an example, There is already a record of "Tufts Medical Center" in the database.
Is there any way to avoid the next record having the same name?
I'm trying to find a way except for doing a GET action before the POST.
Thanks again, and have a great one

Hi @cpeter0702.  There's a few ways to do that.
The mechanism defined by the FHIR specification is called conditional create.  Unfortunately, our implementation (and most others') can't provide absolute uniqueness guarantees if the creates are executed in parallel.
For cases where you need a stronger level of assurance, we have a special feature implemented for "conditional create-on-update".  This will only work in controlled environments where it is safe to use "client-assigned" resource ids.
In this case, if you pass the HTTP header If-None-Match: * then the server will perform the "update" only if the resource doesn't already exit.  See https://linuxforhealth.github.io/FHIR/Conformance#http-headers for more info.

Worth mentioning:  both of the above options require the client to "do the right thing".  At the server level, its possible to write an interceptor to enforce whatever business logic you'd like.  So I guess that would be the way to ensure it at the server level.

hi @lmsurpre, thank you so much for your reply. Using "conditional create-on-update" is helpful and solves my primary key issue. In addition,  because I also have needs to make one of the columns or fields unique other than the primary key,  I'm trying to implement the "interceptor". Thanks again, and have a great one.


vscode's search and other functions are very sluggish

Version: 1.73.0 (Universal)
Commit: 8fa188b2b301d36553cbc9ce1b0a146ccb93351f
Date: 2022-11-01T15:38:50.881Z (6 days ago)
Electron: 19.0.17
Chromium: 102.0.5005.167
Node.js: 16.14.2
V8: <IP_ADDRESS>-electron.0
OS: Darwin arm64 22.1.0
Sandboxed: No
My vscode is working fine when I first open the project, searching for files, source control, etc., but as soon as I switch branches, everything becomes loading, very slow, occasionally successful. Switching back to a branch doesn't work either, my system is the latest Mac Ventura.

Even if I remove vscode completely and reinstall it, I still have this problem, as long as everything changes branches, all functions become laggy.

Which process is slow? See here to debug performance issues https://github.com/microsoft/vscode/wiki/Performance-Issues

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.
Happy Coding!


Adds ClusterID parameters to on-demand credential generation

This PR adds a ClusterID parameter to on-demand credential generation for SRE UX so they don't need to know the inner workings of the AWS Account Operator

/lgtm

/retest

/hold
Need to test with new build because of the go module missing.

/lgtm

/hold

/lgtm

/hold cancel


BUG: Validator is showing in active set, but not signing any blocks after upgrade to Paloma v1.13.3

What is happening?

Section description
Provide as much context as you can. Give as much context as you can to make it easier for the developers to figure what is happening.

BUG: Validator is showing in active set, but not signing any blocks after upgrade to Paloma v1.13.3
Paloma Logs:
https://gist.github.com/pcheliniy/5de28dfdeb927073d965456ce474d72a paloma logs
Pigeon Logs:
https://gist.github.com/pcheliniy/2f1d4d0f4dbb8170f69de91ef4beead7 pigeon logs
Paloma and pigeon versions and logs

Section description
Write down paloma version.
Write down pigeon version.
Copy and paste pigeon config file as well as relevant ENV variables.

Paloma: v1.13.3
commit: 1ba2ba70804f490ba2512a00506265cf7b40d51e
Pigeon: 1.11.0
How to reproduce?

Section description
Please write detailed steps of what you were doing for this bug to appear.

Not able to reproduce. Validator shows as active: https://paloma.explorers.guru/validator/palomavaloper1p4r04ek0jnngegmw76rpx66xu9ep0kuzvcwxyy
But is not signing.
What is the expected behaviour?

Section description
If you know, please write down what is the expected behaviour. If you don't know, that's ok. We can have a discussion in comments.

Active validators must sign.

Closing as fixed by @pcheliniy

It was a my mistake and not really a problem, I messed up with node keys, during all this cleanups / inits.


Session operation has timed out inside Azure Automation

I am trying to use Posh-SSH in an Azure Automation Runbook in order to run something on a Linux VM.  My script
$password = Convertto-SecureString -String "xxxxxxx" -AsPlainText -Force
$cred = New-Object -Typename System.Management.Automation.PSCredential -ArgumentList numbers, $password
$ssh = New-SSHSession -ComputerName "example.com" -Credential $cred -AcceptKey -ConnectionTimeOut 100
$(Invoke-SSHCommand -Index 0 -Command "echo dog").Output

runs fine from my command line, but in Azure Automation I get the error
Session operation has timed out

which is why I extended the timeout.  Do you know whether Posh-SSH operate in this environment?

The full error is
New-SSHSession : Session operation has timed out
At line:3 char:8
+ $ssh = New-SSHSession -ComputerName "example.com" -Credentia ...
+        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [New-SSHSession], SshOperationTimeoutException
    + FullyQualifiedErrorId : Renci.SshNet.Common.SshOperationTimeoutException,SSH.NewSshSession

I have had more luck with this SSH client implementation for PowerScript.

I don’t use Azure so no clue there. Only thing that comes to mind is they auth it does to check if the key it is one it knows about and you are not auto accepting the key.

On Dec 17, 2015, at 7:25 PM, oliverbock<EMAIL_ADDRESS>wrote:
I have had more luck with this SSH client implementation for PowerScript https://gallery.technet.microsoft.com/scriptcenter/SSH-PowerShell-Module-17616e1a.
—
Reply to this email directly or view it on GitHub https://github.com/darkoperator/Posh-SSH/issues/83#issuecomment-165614201.

Thanks.  Am I not accepting the key properly?  I included "-AcceptKey" alone because when I tried "-AcceptKey 1" or "-AcceptKey $true" then PowerShell (v3.0) complains about unexpected command-line parameters.

basic usage would be
New-SSHSession -ComputerName fooserver -AcceptKey
I have run into issues with devices with a different key then what was previously accepted (due to re-image etc) and have to either cleanup after each script with the Remove-SSHTrustedHost function.  Or I can purge all keys with
foreach($i in Get-SSHTrustedHost){Remove-SSHTrustedHost -SSHHost $i.SSHHost -Verbose}

Thinking if I should add a -Force parameter that will not check the key at all and simply accept a session if the credentials are valid

Drd3
On 28 Feb 2016 02:06, "Carlos Perez"<EMAIL_ADDRESS>wrote:

Thinking if I should add a -Force parameter that will not check the key at
all and simply accept a session if the credentials are valid
—
Reply to this email directly or view it on GitHub
https://github.com/darkoperator/Posh-SSH/issues/83#issuecomment-189764397
.

It would be good if you can fix this Carlos so we can use the full capabilities of this module in Azure Automation.
Besides that it is a great module and thank you for your work!

added -Force on the 1_7_5 branch, once I have some other issues fixed for that version I will push the version to GitHub and to the Gallery.

Uploaded 1.7.5 to the gallery

Even with the -Force parameter I am getting a timeout..

Anything on the target logs?
Sent from my iPhone

On May 30, 2017, at 7:17 AM, rrbruin<EMAIL_ADDRESS>wrote:
Even with the -Force parameter I am getting a timeout..
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or mute the thread.

The target is a Fortigate firewall. But at the moment it's not producing any usable logs in the GUI.
I will try to get more info.
Using the Posh-SSH module in a Powershell ISE session works perfect when connecting to the Fortigate.
It only fails when I use it from a Azure runbook.

The module requires to be ran under the context of a user since it accesses the registry for some tasks, do Azure Runbooks run under a user?

On May 30, 2017, at 8:16 AM, rrbruin<EMAIL_ADDRESS>wrote:
The target is a Fortigate firewall. But at the moment it's not producing any usable logs in the GUI.
I will try to get more info.
Using the Posh-SSH module in a Powershell ISE session works perfect when connecting to the Fortigate.
It only fails when I use it from a Azure runbook.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub https://github.com/darkoperator/Posh-SSH/issues/83#issuecomment-304859945, or mute the thread https://github.com/notifications/unsubscribe-auth/AAf0HhXvWIMCnDcWfmGVxYlEGjq0y6rVks5r_AiVgaJpZM4G3IIC.

No, it completely runs inside the Azure environment. So, I don’t think there is even a register to access.
I will try to run the runbook tough an Azure HybridWorker. In this case, it will run on a Windows VM using a service account.
This will add extra step which I rather avoid. But if that’s the only way to get it working, I have no choice. Or I should look for another SSH solution
I will let you know.

Ok, I do know the dev branch of version 2.0 does not have this restriction but 2.0 is not ready yet and it will introduce braking changes, that is why it will be a major number release

On May 30, 2017, at 10:15 AM, rrbruin<EMAIL_ADDRESS>wrote:
No, it completely runs inside the Azure environment. So, I don’t think there is even a register to access.
I will try to run the runbook tough an Azure HybridWorker. In this case, it will run on a Windows VM using a service account.
This will add extra step which I rather avoid. But if that’s the only way to get it working, I have no choice. Or I should look for another SSH solution
I will let you know.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub https://github.com/darkoperator/Posh-SSH/issues/83#issuecomment-304891701, or mute the thread https://github.com/notifications/unsubscribe-auth/AAf0Hqtnr52oxaljOus2Cc2FC28q3Zweks5r_CR8gaJpZM4G3IIC.

Under what context does Azure Automation run under? Never used Azure so I have little knowledge on it.

On Mar 21, 2016, at 8:48 AM, Stanislav Zhelyazkov<EMAIL_ADDRESS>wrote:
It would be good if you can fix this Carlos so we can use the full capabilities of this module in Azure Automation.
Besides that it is a great module and thank you for your work!
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub https://github.com/darkoperator/Posh-SSH/issues/83#issuecomment-199257105

I think it will run under some kind of SYSTEM account.
It does work when I use the Azure HybridWorker option.
For now I can use this as an option.

I am running into this issue as well.  Do you have any information on when version 2.0 will be released?

Hello,
I've the same problem on my Azure Automation account, with the -Force parameter, any idea? Thanks

The only workaround I found was to use the Azure HybridWorker option as @rrbruin mentioned

Version 2 was released are you using the latest version?
Sent from my iPhone

On Sep 14, 2017, at 10:01 AM, rbyrnedugan<EMAIL_ADDRESS>wrote:
The only workaround I found was to use the Azure HybridWorker option as @rrbruin mentioned
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or mute the thread.

Yes I downloaded it this afternoon.
Florent
From: Carlos Perez<EMAIL_ADDRESS>Reply-To: darkoperator/Posh-SSH<EMAIL_ADDRESS>Date: Thursday, 14 September 2017 at 16:33
To: darkoperator/Posh-SSH<EMAIL_ADDRESS>Cc: Flodu31<EMAIL_ADDRESS>Comment<EMAIL_ADDRESS>Subject: Re: [darkoperator/Posh-SSH] Session operation has timed out inside Azure Automation (#83)
Version 2 was released are you using the latest version?
Sent from my iPhone

On Sep 14, 2017, at 10:01 AM, rbyrnedugan<EMAIL_ADDRESS>wrote:
The only workaround I found was to use the Azure HybridWorker option as @rrbruin mentioned
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or mute the thread.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

The to be honest dont know what I could else do. I’m no longer checking the resgitry or trying to open it. Since I do not have Azure access there is no way for me to test to see what else it could be. Does it at least give you a specific error?

On Sep 14, 2017, at 11:16 AM, Flodu31<EMAIL_ADDRESS>wrote:
Yes I downloaded it this afternoon.
Florent
From: Carlos Perez<EMAIL_ADDRESS>Reply-To: darkoperator/Posh-SSH<EMAIL_ADDRESS>Date: Thursday, 14 September 2017 at 16:33
To: darkoperator/Posh-SSH<EMAIL_ADDRESS>Cc: Flodu31<EMAIL_ADDRESS>Comment<EMAIL_ADDRESS>Subject: Re: [darkoperator/Posh-SSH] Session operation has timed out inside Azure Automation (#83)
Version 2 was released are you using the latest version?
Sent from my iPhone

On Sep 14, 2017, at 10:01 AM, rbyrnedugan<EMAIL_ADDRESS>wrote:
The only workaround I found was to use the Azure HybridWorker option as @rrbruin mentioned
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or mute the thread.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub https://github.com/darkoperator/Posh-SSH/issues/83#issuecomment-329515041, or mute the thread https://github.com/notifications/unsubscribe-auth/AAf0HpPOG8GTJnuWF2gytVv1ApH3hdfZks5siUNcgaJpZM4G3IIC.

No, I've exactly the same. I'll try another module to see if the problem is the same and will keep you updated.

I'll use the workaround of @rbyrnedugan because I can't find a solution to this problem.


docs(README.md): fix typo in CPM section

'"' was missing in the markdown
Looked weird

@elbeno reached here from the Microsoft Reactor talk
Was fun!


Constructor for linear dimensions

https://strusoft.freshdesk.com/a/tickets/26364
https://github.com/strusoft/femdesign-api/blob/1c4106ac2f6e2b51dde46894b46e24e8f4a08f33/FD 21.01.003.xsd#L4964-L4983
User wants to be able to add linear dimensions to their model for visualization purposes.

[x] C# constructor
[x] Grasshopper component
[x] Set font-size
[x] Set arrow-type
[x] Set number of decimals
[x] Set unit type
[x] String override
[x] Add test to see if distances are calculated correctly
[ ] Report: Incorrect position type in xsd-classes?
[ ] Report: Why does the measuement value have to be set?
[ ] Bug: Can't set measurement value position.
[ ] Add icons.
[ ] Link files to Dynamo.
[ ] Allow user to set custom layer

Issues:
Position type in schema is incorrect
This issue is fixed in 22.00.001 schema.
Why do I have to calculate measurement?
It seems an necessary that we have to define the actual measure in the schema as this can be done inside the app anyways.
This is a great source for error.

Can't position measurement text correctly

Will not be finished in 22.1.0. Still waiting on fd-developer.


test: update ami image id

This PR is to update ami id to latest according to compose update.

Hi @cgwalters! Test is runing into an error Failed to invoke skopeo proxy method FinishPipe. This error should be the same as issue https://github.com/coreos/rpm-ostree/issues/4567.
I retired twice and can't make it green.

It's always been a race condition AFAICS, but unfortunate we're seeing it here too.

As far as these AMI updates, I think we should probably configure renovate to do it.

As far as these AMI updates, I think we should probably configure renovate to do it.

This ami id will be used by at least two repos(bootc and bootc-workflow-test), I'll save ami id into AWS Systems Manager (SSM) Parameter Store. There's a CI in https://github.com/virt-s1/bootc-workflow-test/blob/main/.github/workflows/upload_rhel9_ami.yml. The CI includes a fully test for new ami. So I'll update this CI to deploy test passed ami id to AWS Systems Manager (SSM) Parameter Store instead of update the ami id in playbook.


Accessory plural

this PR needs db migration

@jonzee what's up with this?

before merge this pr we have to have migration flow #37


basic data collection

For the last 30 days
On Sun, Nov 20, 2022 at 1:56 PM rithikchalla @.***>
wrote:

@.**** commented on this pull request.
In data_collection.ipynb
https://github.com/smurffers/finance-ETL-Process/pull/3#discussion_r1027336474
:

  "28  Oct 04, 2022  250.52  257.50  242.01  249.44    249.44  109,578,500\n",

  "29  Oct 03, 2022  254.50  255.16  241.01  242.40    242.40   98,363,500"

 ]

},

"execution_count": 21,

"metadata": {},

"output_type": "execute_result"

}
],
"source": [
"driver = webdriver.Chrome(executable_path="chromedriver.exe")\n",
"url = 'https://finance.yahoo.com/quote/TSLA/history?p=TSLA'\n",
"driver.get(url)\n",
"\n",
"data =[]\n",
"for i in range(1,31):\n",

Why are we hardcoding for i in range of 1 to 31. What exactly is this 1 to
31?
—
Reply to this email directly, view it on GitHub
https://github.com/smurffers/finance-ETL-Process/pull/3#pullrequestreview-1187307306,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AOZFKTO6PGQKUTRRTPGOYTDWJJX5TANCNFSM6AAAAAASFS2MQA
.
You are receiving this because you authored the thread.Message ID:
@.***>

--
AMBURI SAI GOWTHAM BABU

Can pass it as a value that should be given as input
On Sun, Nov 20, 2022 at 1:56 PM sai gowtham babu amburi <
@.***> wrote:

For the last 30 days
On Sun, Nov 20, 2022 at 1:56 PM rithikchalla @.***>
wrote:

@.**** commented on this pull request.
In data_collection.ipynb
https://github.com/smurffers/finance-ETL-Process/pull/3#discussion_r1027336474
:

  "28  Oct 04, 2022  250.52  257.50  242.01  249.44    249.44  109,578,500\n",

  "29  Oct 03, 2022  254.50  255.16  241.01  242.40    242.40   98,363,500"

 ]

},

"execution_count": 21,

"metadata": {},

"output_type": "execute_result"

}
],
"source": [
"driver = webdriver.Chrome(executable_path="chromedriver.exe")\n",
"url = 'https://finance.yahoo.com/quote/TSLA/history?p=TSLA'\n",
"driver.get(url)\n",
"\n",
"data =[]\n",
"for i in range(1,31):\n",

Why are we hardcoding for i in range of 1 to 31. What exactly is this 1
to 31?
—
Reply to this email directly, view it on GitHub
https://github.com/smurffers/finance-ETL-Process/pull/3#pullrequestreview-1187307306,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AOZFKTO6PGQKUTRRTPGOYTDWJJX5TANCNFSM6AAAAAASFS2MQA
.
You are receiving this because you authored the thread.Message ID:
@.***>

--
AMBURI SAI GOWTHAM BABU

--
AMBURI SAI GOWTHAM BABU

Since,
We are using selenium to extract data, we can also use the html tags to
load the data
On Sun, Nov 20, 2022 at 1:57 PM sai gowtham babu amburi <
@.***> wrote:

Can pass it as a value that should be given as input
On Sun, Nov 20, 2022 at 1:56 PM sai gowtham babu amburi <
@.***> wrote:

For the last 30 days
On Sun, Nov 20, 2022 at 1:56 PM rithikchalla @.***>
wrote:

@.**** commented on this pull request.
In data_collection.ipynb
https://github.com/smurffers/finance-ETL-Process/pull/3#discussion_r1027336474
:

  "28  Oct 04, 2022  250.52  257.50  242.01  249.44    249.44  109,578,500\n",

  "29  Oct 03, 2022  254.50  255.16  241.01  242.40    242.40   98,363,500"

 ]

},

"execution_count": 21,

"metadata": {},

"output_type": "execute_result"

}
],
"source": [
"driver = webdriver.Chrome(executable_path="chromedriver.exe")\n",
"url = 'https://finance.yahoo.com/quote/TSLA/history?p=TSLA'\n",
"driver.get(url)\n",
"\n",
"data =[]\n",
"for i in range(1,31):\n",

Why are we hardcoding for i in range of 1 to 31. What exactly is this 1
to 31?
—
Reply to this email directly, view it on GitHub
https://github.com/smurffers/finance-ETL-Process/pull/3#pullrequestreview-1187307306,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AOZFKTO6PGQKUTRRTPGOYTDWJJX5TANCNFSM6AAAAAASFS2MQA
.
You are receiving this because you authored the thread.Message ID:
@.***>

--
AMBURI SAI GOWTHAM BABU

--
AMBURI SAI GOWTHAM BABU

--
AMBURI SAI GOWTHAM BABU

The function can be created.
But, I am testing it on multiple test cases that we discussed.
So, didn't create a function for now.
On Sun, Nov 20, 2022 at 1:58 PM sai gowtham babu amburi <
@.***> wrote:

Since,
We are using selenium to extract data, we can also use the html tags to
load the data
On Sun, Nov 20, 2022 at 1:57 PM sai gowtham babu amburi <
@.***> wrote:

Can pass it as a value that should be given as input
On Sun, Nov 20, 2022 at 1:56 PM sai gowtham babu amburi <
@.***> wrote:

For the last 30 days
On Sun, Nov 20, 2022 at 1:56 PM rithikchalla @.***>
wrote:

@.**** commented on this pull request.
In data_collection.ipynb
https://github.com/smurffers/finance-ETL-Process/pull/3#discussion_r1027336474
:

  "28  Oct 04, 2022  250.52  257.50  242.01  249.44    249.44  109,578,500\n",

  "29  Oct 03, 2022  254.50  255.16  241.01  242.40    242.40   98,363,500"

 ]

},

"execution_count": 21,

"metadata": {},

"output_type": "execute_result"

}
],
"source": [
"driver = webdriver.Chrome(executable_path="chromedriver.exe")\n",
"url = 'https://finance.yahoo.com/quote/TSLA/history?p=TSLA'\n",
"driver.get(url)\n",
"\n",
"data =[]\n",
"for i in range(1,31):\n",

Why are we hardcoding for i in range of 1 to 31. What exactly is this 1
to 31?
—
Reply to this email directly, view it on GitHub
https://github.com/smurffers/finance-ETL-Process/pull/3#pullrequestreview-1187307306,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AOZFKTO6PGQKUTRRTPGOYTDWJJX5TANCNFSM6AAAAAASFS2MQA
.
You are receiving this because you authored the thread.Message ID:
@.***>

--
AMBURI SAI GOWTHAM BABU

--
AMBURI SAI GOWTHAM BABU

--
AMBURI SAI GOWTHAM BABU

--
AMBURI SAI GOWTHAM BABU

Ok.
Will work on it.
The provided code is just a sample of the work and needs a lot of work to
be done.
On Sun, Nov 20, 2022 at 2:01 PM rithikchalla @.***>
wrote:

@.**** commented on this pull request.
Please restructure all this code into a proper python functions and I
would love to see a .py file rather than a .ipynb and would also love to
see unit test cases included.
—
Reply to this email directly, view it on GitHub
https://github.com/smurffers/finance-ETL-Process/pull/3#pullrequestreview-1187307815,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AOZFKTNZDB36GYJG3L3HYGTWJJYQLANCNFSM6AAAAAASFS2MQA
.
You are receiving this because you authored the thread.Message ID:
@.***>

--
AMBURI SAI GOWTHAM BABU

If the code is written, it might not work on all the test cases. But, the
code can be worked for basic code to extract the data.
Does that work for us on a minimum basis?
On Sun, Nov 20, 2022 at 2:02 PM rithikchalla @.***>
wrote:

@.**** commented on this pull request.
In Makefile
https://github.com/smurffers/finance-ETL-Process/pull/3#discussion_r1027337255
:

@@ -0,0 +1,19 @@
+install:

When would this file execute on a PR or on a push?
—
Reply to this email directly, view it on GitHub
https://github.com/smurffers/finance-ETL-Process/pull/3#pullrequestreview-1187307966,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AOZFKTLMFWMWS27BCMRKPM3WJJYVZANCNFSM6AAAAAASFS2MQA
.
You are receiving this because you authored the thread.Message ID:
@.***>

--
AMBURI SAI GOWTHAM BABU

Please let me know the new reuirements for the project. List them all so that i could work on them and let me update with the dealine.

Please let me know the new reuirements for the project. List them all so that i could work on them and let me update with the dealine.

The requirements should still be the same as we discussed earlier and I guess thats in the issues.

A new data_collection.py file have been added to vvk_goosebumps branch. Using that file, we can scrape data till date or we can scrape data selective days with in the last 30 days.

The code just prints the data. It does not convert the data into a csv format. Please do mention the required format ( shall i conver it to csv/ just print?)

We need to figure out a data warehouse location or a storage location like object storage and then dump the CSV file there. Once the CSV file is left in there, we will further process the data and transform that into a final table that we would like to use for reporting or further data analytics.


feat(mount): copy {script,link} tags

Allows us to create root views that look like:

function mainView (state, prev, send) {
  return html`

choo choo

  `
}

And these will have like all the


Add support for tsconfig.json.

https://github.com/Microsoft/TypeScript/wiki/tsconfig.json
tsconfig.json is the official project type for TypeScript projects. In particular, it is where compiler options are stored. In many situations, there may be several tools that are running the TypeScript compiler and they should all use the same options for a given project. In order for this to not become a maintenance nightmare, each of the tools should leverage a shared configuration file, which is tsconfig.json.
I would like to see SystemJS's TypeScript transpiler support tsconfig.json so I don't have to store my compiler options in two places.

This sounds reasonable, so I can add a setting to typescriptOptions tsconfig which will contain the path of a tsconfig.json file or just be set to true to load /tsconfig.json?
typeScriptOptions: {
  tsconfig: true,
  tsconfig: "/.settings/tsconfig.json",
  resolveAmbientRefs: true
}

I'm guessing that a person would either use a tsconfig.json or provide the options through typeScriptOptions.  I can't think of a situation where you would want to mix the two.  I suppose if a user did want to override some settings it would probably be specific to runtime plugin, so maybe the merge strategy would be to pull in all of the information from tsconfig.json and then overwrite any explicit options provided via typeScriptOptions?
tsconfigPath: 'relative/path/to/tsconfig.json',
typeScriptOptions: {
	resolveAmbientRefs: true
}

I will admit, I haven't really thought this through very hard, I just started using the typescript plugin with SystemJS and the first thing I looked into was using my existing tsconfig.json settings since I already have those setup so my IDE can correctly do type checking.

+1 for @Zoltu's recommendation above.  Seems to make the most sense intuitively.

+1 so I don't have to use ///references everywhere :(
Also getting cannot find module errors when I can see the file (transpiled) in the browser!

@frankwallis if tsconfig support is supported, would this be possible using in browser compilation?
src/app.ts
then src/module/component/comp1/comp1.ts import {default as d} from 'app'; work?
Right now you must do `import {default as d} from '../../../app';

@robertbaker - I am not sure if this works even when building with the TypeScript compiler, it seems to me that the code would fail at runtime unless the compiler rewrites the import in the generated JavaScript to point to the correct file. Support for resolution mappings is being handled under https://github.com/Microsoft/TypeScript/issues/2338.
Currently my plan for tsconfig support is just to load the compiler options (which will actually just be the subset of the options which do not conflict with those used by plugin-typescript) and to use it to automatically load any reference files contained in the 'files' array.

Just my $0.02 about having both tsconfig and specifying compiler options:  The command-line tsc supports this specifically, with command line options overriding options in the tsconfig file.

I prefer tsconfig: /path/to/tsconfig. There is discussion about this here : https://github.com/TypeStrong/meta/issues/1 :rose:

Ok I have finally added tsconfig.json support (in 2.1.0), you can use
tsconfig: true

to load from "tsconfig.json", or set it to a path which will be resolved to locate the file:
tsconfig: "config/mytsconfig.json"

The plugin loads compiler options and declaration files from the files array when type-checking is enabled. I have updated some of the examples to show it working. Let me know if you have any problems, thanks.

awesome!


How are target_queries generated?

Sorry if I missed this in the repo somewhere, but could you let me know if there is a script to generate the JSON files in target_queries were generated (along with the correct formatting)?

Hi, Thank you! In Section 5.1 of our paper, we discussed "Target questions and answers". We first used the ground truth context (qrel) to generate a correct answer for each query in the dataset using GPT-4, and then since we have already got the correct answer, we could use GPT-4 again to randomly generate an incorrect answer for each query. This is how we made the target_queries.
The script is a python file called gen_adv.py, we are modifying it and will upload later in this week. You could also email me if you have an urgent need for this file.
Thanks again for pointing it out!

Thanks for clarifying! I also wanted to ask if you generated incorrect answers via GPT-4 manually, and whether this was in a batch or independently, as I noticed that there are  100 target questions and answers for the QA datasets used in the paper.

Hello, I just uploaded the script for generating adv_targeted_results in the latest commit. We found that target_queries and adv_targeted_results are redundant so we removed target_queries and just keep adv_targeted_results now.
And yes, as we discussed in Section 5.1, we mannually checked the quality of the generated incorrect answers via GPT-4.
Now you could refer to gen_adv.py for more details.

Hi @tamtakapanadze, I've fixed the path bug in the latest commit. Thanks for pointing it out!


Moving FAQ - Using compressed data as tool inputs

This FAQ is part of the Understanding compressed fastq data (fastq.gz).
The first section of the FAQ has already been covered by this GTN FAQ.

thank you @Melkeb!


feat: Velero integration

Checklist

[ ] Architecture Design Records have been added as adr/*.md and appended to list in adr/_index.md, if applicable.
[ ] The values-schema.yaml file and test/** fixtures have been updated to reflect code changes, if applicable.
[ ] The OpenApi Schema from redkubes/otomi-api project is compatible with definitions from values-schema.yaml file, if applicable.
[ ] Helm releases are meeting otomi's baseline security policies, if applicable.
[ ] Helm chart and helmfile changes are tested against upgrade scenario, if applicable.

@j-zimnowoda I see no commits since my last review so I will wait for those to come in first


Bump OSA-OPS repo

This includes, for the leapfrog:

A fix to the openstack-release discovery
A change to the mirror used by default
A fix to mysql-python/pip requirements dependencies

Partial-issue: LA-322

recheck_cit_artifact_xenial_apt

recheck_cit_trusty_leapfrogupgrade_swift

recheck_cit_trusty_leapfrogupgrade_swift

recheck_cit_trusty_leapfrogupgrade_swift

recheck_cit_trusty_leapfrogupgrade_swift

recheck_cit_trusty_leapfrogupgrade_swift

Depends on https://github.com/rcbops/rpc-openstack/pull/2411

Depends on https://github.com/rcbops/rpc-openstack/pull/2414

recheck_cit_trusty_leapfrogupgrade_swift


Fixes the Django 1.11 ORM compatibility issue.

This pull request fixes the remaining issue blocking 1.11 compatibility (see #3314).
It is due to the changes done in django/django@1bc249c.

Wouldn't this break cases where the value is not a subquery? For example:
MyModel.objects.filter(category__in=['foo', 'bar']).search()

That’s right, we have to add a test for such use cases.
I’ll update the pull request & add a test.

I pushed tests and handled other cases than subqueries.

Looks good to me, thanks!


Bug in Dumping a computational graph including LayerNormalization

Added LayerNomalization to mnist example network.
class MLP(chainer.Chain):

    def __init__(self, n_units, n_out):
        super(MLP, self).__init__(
            # the size of the inputs to each layer will be inferred
            l1=L.Linear(None, n_units),  # n_in -> n_units
            ln1=L.LayerNormalization(),
            l2=L.Linear(None, n_units),  # n_units -> n_units
            ln2=L.LayerNormalization(),
            l3=L.Linear(None, n_out),  # n_units -> n_out
        )

    def __call__(self, x):
        h1 = F.relu(self.ln1(self.l1(x)))
        h2 = F.relu(self.ln2(self.l2(h1)))
        return self.l3(h2)

Dumped computational graph shows components of LayerNormalization.

Hi. TL;DR, it is difficult in current implementation that a link is expanded if Link.__call__ is implemented with other links, functions, and variables.
It is Chainer functions and variables, not links that are appeared in dumped computational graphs. I think it would be natural to bundle Chainer functions and variables with the link to which these modules belong to . But currently, there is no way to tell Chainer functions and variables which link it belongs to automatically (i.e. without hints by the developer of the link). Also there is no way to tell a link which Chainer functions and variables belong to it.
I think analyzing such a hierarchical structure mentioned above without user's help would be as difficult as parsing Python scripts. On the other hand, it would be cumbersome for users to annotate links. It would be good to support "hierarchical" computational graphs. I am considering what is the appropriate compromising point.

Thank you for reply.
I understood its difficulty.
I agree with "it would be natural to bundle Chainer functions and variables with the link to which these modules belong to."
I expect that it will be improved in the future.

@niboshi Should we reopen this issue?

@delta2323 I think so, as this is not solved yet.

This issue was solved by  layer normalization implemented as a function. (#2857).
So,  it would be nice to close this issue and make another issue ( about "hierarchical" computational graphs etc.) if necessary.

Thank you for your comments. The problem of hierarchical computational graphs is addressed in #1424. Then we can close this issue with referencing the issue. Do you think it is reasonable? > @niboshi

Yes, I agree with you.


@ts-ignore not work for typedoc

My project uses TypeScript. And I compile the project with "webpack": "^3.3.0", "typescript": "^2.6.1" and  "awesome-typescript-loader": "^3.4.1".
In some cases, I need to use __DEV__ to write conditional code. Like the following code snippet.
let activeInfo = this._uniformActiveInfosMap[name];
if (!activeInfo) {
    // @ts-ignore
    if (__DEV__) {
        // https://stackoverflow.com/questions/21911836/opengl-shaders-uniform-variables-count-incorrect
        let warningMessage = `uniform ${name} is not active for program ${this.type}`;
        if (!this._warningMessagMap[warningMessage]) {
            this._warningMessagMap[warningMessage] = true;
            console.warn(`uniform ${name} is not active for program ${this.type}`);
        }
    }
}

The value of __DEV__ is defined by webpack.DefinePlugin in webpack.config.js.
vscode will show Cannot find name __DEV__ error, so I add the comment // @ts-ignore and vscode doesn't show this error any more.
Unfortunately, when I use typedoc, the error Cannot find name __DEV__ appears. It seems // @ts-ignore doesn't work for typedoc.
Here is the npm script of typedoc.
"doc:api": "typedoc --out _site src/"

typedoc vertion is "typedoc": "^0.9.0"
I know typedoc supports option --ignoreCompilerErrors  and it does work for my case. But I don't want to use this option because it will ignore all compile errors.
So, my question is how to make @ts-ignore works for typedoc.
Any comments are appreciated.

Hello @iSpring, TypeDoc uses it's own version of TypeScript due to tight bundling. I believe typedoc v0.9.0 was using TypeScript v2.5. Can you try again with the latest version of TypeDoc?

Hi @aciccarello , thanks for your hint. typedoc 0.11.0 does honor @ts-ignore. Close this issue.


[Candidate 1v1 Arab Tragedy ZHv3] Map name needs changing

Issue Description
Map name is unacceptable. Arab Tragedy can be interpreted as an insult to Arabs.
Expected behavior
Map name fits the overall style of Official Maps and ReLaX Maps.

Some suggestions from the discord:
Tragedy Tracks
Arab Prosperity
A New Tragedy
My vote goes for "Tragedy Tracks", got a really nice ring to it and makes sense.

Map will be renamed to [RANK] A New Tradegy ZH v1


Metrodr e2e

@nirs I have made some changes to this draft PR as it is something I am using for my current sync dr testing. I have addressed some of your comments as part of the changes but I don't intend to bring this PR out of draft status.
Please continue working on your changes using drenv. My goal is to have this working so that we can use it as a reference when your drenv code is merged to enable sync dr in that.


UI styling

Hi @atomotic @ikreymer!
I've update main.html & loader.html with the new UI.
These are the states I had in mind for the css:

App first loads: main.html in default state

on opening a collection / returning to collection view: Add .viewCollection class to #topBar and #viewCollection, remove .replay if necessary

After hitting play: Add .replay class to #topBar and #viewCollection, remove .viewCollection if necessary

Totally not sure if this is the right approach, especially where I've put "Project by Rhizome" div into "webview". If changes are needed, lmk! :)

[ ] Add big load button
[ ] Resize logo? (Not 100% sure I understand the changes needed. Discuss this more over a call?)
[ ] Add "back to collection / list view" icon


Won't start debugging program.

Environment data

VS Code version: XXX
Extension version (available under the Extensions sidebar): 2018.6.0
OS and version: Windows 10 64 bit
Python version 3.6.5
Type of virtual environment used: virtualenv
Relevant/affected Python packages and their versions: XXX

Actual behavior
\.vscode\extensions\ms-python.python-2018.6.0\pythonFiles\PythonTools\visualstudio_py_launcher.py", line 29, in <module>
    import traceback
ModuleNotFoundError: No module named 'traceback'

Expected  behavior
Run the program and execute code, program works.
Steps to reproduce:

XXX

Logs
Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)
XXX

Output from Console under the Developer Tools panel (toggle Developer Tools on under Help)
XXX

Whenever i want to launch the code i get the error above, i have unzipped the project and put it on my desktop, can't find any solutions..

Please do fill out the "Steps to reproduce", the "Logs", sections of the issue template for us, and if you could also please attach you launch.json file.
Also consider: Which module: is specified in the current debugger profile selected? Should it be a module? Or should you simply be running the currently opened file?

Can you also run any code? The fact that PTVSD can't import traceback suggests there is something wrong with your Python environment as that's a module from Python's standard library.

Hi, yes i am trying to run one file named Pychat.py, it works at school but at home it does not, here is my launch.json file:
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "Python: Current File",
"type": "python",
"request": "launch",
"program": "${file}"
}
]
}
The file i am trying to run is included in the Folder itself.
Importing traceback in python console does work yes.

"Importing traceback in python console does work yes." as in Run Python File in Terminal works, or you can just manually run it?
My point is that the only way this fails is because your Python environment is broken to the point that standard library modules are not accessible.
You can also try our experimental debugger by changing "type": "python" to "type": "pythonExperimental".

I have tried it in the python terminal (Python 3.6 (32 Bit)) and it worked there, i've just repaired everything and the modules should be accessible because it works within the terminal

When i try to run it as pythonExperimental i get the same error but instead of it being traceback, it is now runpy

Please try running your code from the Run Python File in Terminal command (it can be as simple as import traceback). The key point is you need to test running code from the Python environment you have set in the extension, not on your system overall.

Because we have not heard back with the information we requested, we are closing this issue for now. If you are able to provide the info later on then we will be happy to re-open this issue to pick up where we left off.


Transaction details endpoint should validate hash parameter

The endpoint which processes /transactions/{hash} requests does not validate the hash parameter.
If you use a hash which is not a valid utf8 sequence this will result in horizon responding with a 500 status. Here is an example of this error which was logged recently:
time="2020-03-12T17:30:39.411Z" level=error msg="loading transaction record: get failed: pq: invalid byte sequence for encoding \"UTF8\": 0x00" pid=5777 req=horizon-testnet-green-002c/xuy67jVTC6-44865331 stack="[main.go:43 transaction.go:118 action.go:217 value.go:460 value.go:321 handler.go:100 handler.go:69 handler.go:222 server.go:2007 mux.go:418 server.go:2007 mux.go:69 mux.go:295 server.go:2007 mux.go:418 server.go:2007 mux.go:69 mux.go:295 server.go:2007 mux.go:418 server.go:2007 http.go:73 server.go:2007 cors.go:190 server.go:2007 compress.go:66 server.go:2007 middleware.go:220 server.go:2007 middleware.go:231 timer.go:227 middleware.go:230]"

I can be reproduce this with curl http://horizonhost/transactions/%00
@tamirms did stumble into this through unescaped URLs as well?

yeah, this is the request which triggered the error earlier:
/transactions/%00%1E4%5E%EF%BF%BD%EF%BF%BD%EF%BF%BDpVP%EF%BF%BDI&R%0BK%EF%BF%BD%1D%EF%BF%BD%EF%BF%BD=%EF%BF%BD%3F%23%EF%BF%BD%EF%BF%BDl%EF%BF%BD%1El%EF%BF%BD%EF%BF%BD```

The URL parameter does in fact get validated as utf8 (i.e. 0x00 is a valid string, see https://play.golang.org/p/rkcxiep0rFO ). I think the problem is the leading 0 when passing the transaction ID on.
Looking into it further ...

I suspect there is a bug in squirrel's parameter substitution (or the way we use it).
When doing:
https://github.com/stellar/go/blob/06b6a58cfd92639deecdb6f1e1529d0b8fa53727/services/horizon/internal/db2/history/transaction.go#L22-L24
With a 0x00-led hash, the resulting string is:
SELECT ht.id, ht.transaction_hash, ht.ledger_sequence, ht.application_order, ht.account, ht.account_sequence, ht.max_fee, COALESCE(ht.fee_charged, ht.max_fee) as fee_charged, ht.operation_count, ht.tx_envelope, ht.tx_result, ht.tx_meta, ht.tx_fee_meta, ht.created_at, ht.updated_at, ht.successful, array_to_string(ht.signatures, ',') AS signatures, ht.memo_type, ht.memo, lower(ht.time_bounds) AS valid_after, upper(ht.time_bounds) AS valid_before, hl.closed_at AS ledger_close_time FROM history_transactions ht LEFT JOIN history_ledgers hl ON ht.ledger_sequence = hl.sequence WHERE ht.transaction_hash = ? LIMIT 1

(I think that the zero is injected after the ?
I could simply fix the problem for transactions by checking transaction IDs only have hex chars.
However, I think we should understand the problem above better and check that it doesn't happen for other parameters since it could potentially constitute a security problem.

It turns out there is nothing wrong with Squirrel. It turns out that Postgres uses 0x00 internally as a terminator and you cannot be part of text (text/char/varchar) passed to it, even if it's a valid utf8 :S.

Uhm, maybe this should be fixed when merging back to master?

@2opremio we usually close issues when we have merged them into a release branch

👍
It seems that GitHub's automatic closing only works when merging in master. I'll investigate.

Yeah automatic closing doesn't work except to the chosen repo root branch.


Change logic to allow genomes without GFFs

Currently you have to provide a ID, Refseq ID or ID, genome, gff.
We may have species without gff from NCBI or your local files.
Also, we use the GFFREAD module to pull out the ID,Genome for TE classification, this should be done by a channel manipulation instead

Fixed in f2996e3
Now you can enter species,genome.fa only  - though genome must have the following endings for it to be recognised:
def fastaExtensions = ['.fa', '.fasta', '.fna', '.fa.gz', '.fasta.gz', '.fna.gz']


[VS Code] Joining collaboration session: Could not connect to the server. Live Share requires openssl 1.0 for authentication, please verify you have the dependency installed.

LiveShareLogs.zip
LiveShareLogs.zip

Error:
Joining collaboration session: Could not connect to the server. Live Share requires openssl 1.0 for authentication, please verify you have the dependency installed.
Steps to Reproduce:

Version Data

extensionName
VSLS

extensionVersion
1.0.18

protocolVersion
2.2

applicationName
VSCode

applicationVersion
1.33.0

platformName
Linux

platformVersion
4.19.0-4-amd64

@pablodanielrey This looks similar to https://github.com/MicrosoftDocs/live-share/issues/1876. Can you try the recommendation there of installing openssl manually? Please let us know if that doesn't work.


Add the google-oauth2 connection to the new_tenant.js recipe

The current new_tenant.js recipe properly creates a Default App client and a Username-Password-Authentication connection, but it should also enable the Google social connection in order to replicate what you get by default when you create a new tenant.

on it.

still working on it don't fire me!


Describe precise algorithm with numbers and scenarios for "High risk"

The app only seems to provide a binary risk assessment: low and high risk.
While the general algorithm is described in the Exposure API docs, it would be very valuable to know what precise values are used in this particular implementation and what this means in a variety of scenarios.
What values were chosen and why? How long do I need to be in contact for High risk to be triggered?
If I remember it correctly, you can define various risk values from 0-9 for a variety of buckets and categories (encounter 14 days ago, 13 days ago,... / low risk person, high risk person / 0-5min, 5-10min,... / signal strength, ...)
I guess I could look up the numerical details in the code, but it would be helpful if someone who actually chose the numbers explained why they chose the exact values they did.

Please see the information in the architecture document and the Epidemiological Motivation of the Transmission Risk Level.
Mit freundlichen Grüßen/Best regards,
SW
Corona Warn-App Open Source Team

Sorry, what you shared is interesting but doesn't answer the question about what the weights are you are setting in the app.
Also, it doesn't allow one to understand where you set the threshold between low and high risk.
@SebastianWolf-SAP How can you close issues so aggressively without properly answering the question? Please get someone else to help with request triaging. I'm reopening this issue.

This is somewhat of a duplicate/followup to #229.

@corneliusroemer You might be interested in comparing this across countries.
This is what we do in this project:
https://github.com/PersonalDataIO/CoronaRiskScoring/issues?q=is%3Aissue+is%3Aopen+label%3AExposureConfiguration
Immuni, for instance, changed the parameters suddenly with no public communication about it.
Consider also helping for the other issues!

This question is still unanswered, but was consolidated in #336 alongside all the others.


portal should set a non-zero exit status when it fails

Hi!
When portal fails it should set a non-zero exit code. This would make it easier to handle failures i.e in a CI environment.
$ portal deploy demo
✔ Wiped all Files from demo
Deploying demo:
Deployed ✔ configs
Deploying content ✖ Request failed with status code 400
$ echo $?
0

@johanhammar agreed! Would you be willing to submit a PR?

Hi @kinman. Unfortunately I won't be able to but it is reported as FTI-1972