jumbo is no more

Just to let you all know, that jumbo is now buried:

ERROR at //build/config/jumbo.gni:16:1: Assertion failed.
assert(!use_jumbo_build,
^-----
The jumbo build is no longer supported. Please remove any uses of 'use_jumbo_build', 'jumbo_build_excluded', or 'jumbo_file_merge_limit' from your args.gn file and re-run GN.
See //media/media_options.gni:8:1: whence it was imported.
import("//build/config/jumbo.gni")
^--------------------------------
See //BUILD.gn:21:1: whence it was imported.
import("//media/media_options.gni")
^---------------------------------

Yeah just realized that when testing v80. This will make automatic building and testing even more difficult to do.

jumbo was already removed some releases ago, maybe just now google removed the legacy support code.

When I made #927, I was very clear on that. All distros will need to remove those jumbo GN flags now, which (unfortunately) does increase compile time.

… sigfnificantly increase compile time :(
2019-09-01T00:25:51 >>> www-client/ungoogled-chromium-72.0.3626.122_p1: 1 hour, 50 minutes, 5 seconds
2019-09-11T23:59:01 >>> www-client/ungoogled-chromium-76.0.3809.132_p1: 1 hour, 56 minutes, 39 seconds
2019-09-20T01:12:30 >>> www-client/ungoogled-chromium-77.0.3865.90_p1: 2 hours, 41 minutes, 24 seconds
2019-09-25T19:37:27 >>> www-client/ungoogled-chromium-77.0.3865.90_p1: 2 hours, 9 minutes, 15 seconds
2019-10-15T18:58:33 >>> www-client/ungoogled-chromium-77.0.3865.120_p1: 2 hours, 12 minutes, 9 seconds
2019-11-02T21:09:47 >>> www-client/ungoogled-chromium-78.0.3904.70_p1: 2 hours, 16 minutes, 58 seconds
2019-11-03T23:59:10 >>> www-client/ungoogled-chromium-78.0.3904.87_p1: 2 hours, 17 minutes, 48 seconds
2019-11-10T16:32:11 >>> www-client/ungoogled-chromium-78.0.3904.97_p1: 2 hours, 15 minutes, 51 seconds
2019-12-01T01:20:57 >>> www-client/ungoogled-chromium-78.0.3904.108_p2: 2 hours, 1 minute, 52 seconds
2019-12-19T05:19:56 >>> www-client/ungoogled-chromium-79.0.3945.79_p1: 2 hours, 56 minutes, 42 seconds
2019-12-21T18:45:44 >>> www-client/ungoogled-chromium-79.0.3945.88_p1: 2 hours, 36 minutes, 2 seconds
2020-01-10T02:25:50 >>> www-client/ungoogled-chromium-79.0.3945.117_p1: 2 hours, 36 minutes, 57 seconds
2020-01-20T22:56:26 >>> www-client/ungoogled-chromium-79.0.3945.130_p2: 2 hours, 38 minutes, 26 seconds
2020-02-07T19:06:21 >>> www-client/ungoogled-chromium-80.0.3987.87_p1: 6 hours, 15 minutes, 11 seconds

yeah, same, it doubled my compile times

Did anybody try building completely in tmpfs? Theoretically this could speed up the process.

When I build through github actions it's (almost) completely tmpfs (well except the whole thing runs in a docker). It still takes more than 6 hours. I can try again on my own machine but I don't think it will magically speed up things a lot.
By the way will ccache speed up things? I'm thinking if it does then maybe we can share the cache somehow?

@PF4Public
I used to use a vps with enough resources with arch in order to build, and it took ~1 hour (when jumbo was still a thing).
@wchen342
I think that is because in the PKGBUILD ninja is only using a single thread, maybe using the -j $(nproc) option could help. And afaik ccache only helps with subsequent builds, not initial ones. I think Github uses a clean env for every build.

@jstkdng

I think that is because in the PKGBUILD ninja is only using a single thread

That is not true. ninja defaults to multi thread, unlike make. I just tested on my own machine and can confirm that is the case.

ccache only helps with subsequent builds, not initial ones.

So my idea is, if each build doesn't differ too much (like between minor versions), then once we have an initial build with ccache, maybe we can put the cache from that inital build into a tarball and use it for all the builds afterwards?

That is a really crazy idea, and the tarball would be huge in size. I'm not sure if it would work tho, just go for it if you have the internet speed to upload the tarball somewhere.

tmpfs seems to be of no help unfortunately :(
One system:
2020-03-11T10:45:41 >>> www-client/ungoogled-chromium: 9 hours, 48 minutes, 24 seconds
2020-03-15T22:05:16 >>> www-client/ungoogled-chromium: 9 hours, 56 minutes, 22 seconds

Another one:
2020-03-11T13:02:02 >>> www-client/ungoogled-chromium: 6 hours, 16 minutes, 5 seconds
2020-03-16T03:41:29 >>> www-client/ungoogled-chromium: 6 hours, 16 minutes, 44 seconds

Last one is on the tmpfs
Comparing other packages:
2020-02-10T19:29:36 >>> app-office/libreoffice: 1 hour, 12 minutes, 43 seconds
2020-03-15T23:12:02 >>> app-office/libreoffice: 1 hour, 8 minutes, 55 seconds

2020-03-03T21:14:45 >>> dev-lang/rust: 48 minutes, 33 seconds
2020-03-15T21:58:30 >>> dev-lang/rust: 45 minutes, 30 seconds

2019-11-18T17:31:22 >>> app-emulation/wine-vanilla: 16 minutes, 22 seconds
2020-03-16T00:26:24 >>> app-emulation/wine-vanilla: 17 minutes, 41 seconds

2020-03-15T06:01:10 >>> www-client/firefox: 1 hour, 35 minutes, 1 second
2020-03-16T02:02:40 >>> www-client/firefox: 1 hour, 29 minutes, 42 seconds

Would there be interest in a patch to put jumbo builds back in?
I think it can be done..

I think it can be done..

True, but with Google removing more and more code, it could become more and more painful maintaining such a patch.

@sinetek I have an idea kicking around but I don't know how feasible it is.
I believe main problem with Jumbo is that non-exported symbols can conflict when you combine translation units together. If this is the only problem with Jumbo builds, then it is possible to determine the translation units that can be combined. (Or inversely, the translation units that cannot belong together. I think the frequency of the collisions should determine which implementation is more efficient.) Once you have this information, then you'll need to do is make GN produce jumbo'd Ninja build files, and everything else is pretty straight-forward.
However, there are two problems we'd need to overcome:

I don't know how much work we'd need to make GN produce Jumbo builds.
I might be wrong and there may be more problems that make it difficult to automatically determine which translation units can be combined. It's entirely possible that after considering all the potential issues, there's not much of an advantage to this approach.

This is not a high-priority for me, so this might be all I want to contribute for now.

@Eloston Yes, that is correct.
I went ahead and investigated this. The result is a patch that enables jumbo-build on Ungoogled Chromium. There are a few instances of conflicting symbols, currently there are three.
chromium-80-jumbo-builds.patch :
--- a/build/config/jumbo.gni  2020-04-11 19:11:51.979131201 +0200
+++ b/build/config/jumbo.gni  2020-04-11 19:17:26.472451060 +0200
@@ -6,18 +6,42 @@
 import("//build/toolchain/goma.gni")

 declare_args() {
+  # If true, use a jumbo build (files compiled together) to speed up
+  # compilation.
   use_jumbo_build = false
+
+  # A list of build targets to exclude from jumbo builds, for optimal
+  # round trip time when frequently changing a set of cpp files. The
+  # targets can be just the short name (in which case it matches any
+  # target with that name), a directory prefixed with the root
+  # specifier //, or a full build target label.
+  #
+  # Example:
+  # These would all exclude the "browser" target in a file
+  # content/browser/BUILD.gn, and potentially more.
+  #
+  # jumbo_build_excluded = [ "browser" ]
+  # jumbo_build_excluded = [ "//content/browser" ]
+  # jumbo_build_excluded = [ "//content/browser:browser" ]
   jumbo_build_excluded = []
+
+  # How many files to group on average. Smaller numbers give more
+  # parallellism, higher numbers give less total CPU usage. Higher
+  # numbers also give longer single-file recompilation times.
+  #
+  # Recommendations:
+  # Higher numbers than 100 does not reduce wall clock compile times
+  # even for 4 cores or less so no reason to go higher than 100.
+  # Going from 50 to 100 with a 4 core CPU saves about 3% CPU time and
+  # 3% wall clock time in a tree with blink, v8 and content
+  # jumbofied. At the same time it increases the compile time for the
+  # largest jumbo chunks by 10-20% and reduces the chance to use all
+  # available CPU cores. So set the default to 50 to balance between
+  # high and low-core build performance. -1 means do the default which
+  # varies depending on whether goma is enabled.
   jumbo_file_merge_limit = -1
 }

-# TODO(crbug.com/994387): Remove all uses of the jumbo_* templates from the
-# BUILD.gn files, and then remove this whole file.
-assert(!use_jumbo_build,
-       "The jumbo build is no longer supported. Please remove any uses of " +
-           "'use_jumbo_build', 'jumbo_build_excluded', or " +
-           "'jumbo_file_merge_limit' from your args.gn file and re-run GN.")
-
 # Normal builds benefit from lots of jumbification
 jumbo_file_merge_default = 50

--- a/v8/gni/v8.gni   2020-04-11 19:11:32.232465323 +0200
+++ b/v8/gni/v8.gni   2020-04-11 19:56:48.159023051 +0200
@@ -2,6 +2,7 @@
 # Use of this source code is governed by a BSD-style license that can be
 # found in the LICENSE file.

+import("//build/config/jumbo.gni")
 import("//build/config/sanitizers/sanitizers.gni")
 import("//build/config/v8_target_cpu.gni")
 import("split_static_library.gni")
@@ -70,7 +71,7 @@
   # build configuration. This allows us to set v8_use_multi_snapshots=true on
   # all bots, and e.g. no-snapshot bots will automatically do the right thing.
   v8_use_multi_snapshots =
-      v8_use_external_startup_data && !build_with_chromium
+      v8_use_external_startup_data && !build_with_chromium && !use_jumbo_build
 }

 if (v8_enable_backtrace == "") {
@@ -139,9 +140,9 @@
       defined(v8_static_library) && v8_static_library && is_win) {
     link_target_type = "split_static_library"
   } else if (defined(v8_static_library) && v8_static_library) {
-    link_target_type = "static_library"
+    link_target_type = "jumbo_static_library"
   } else {
-    link_target_type = "source_set"
+    link_target_type = "jumbo_source_set"
   }
   target(link_target_type, target_name) {
     forward_variables_from(invoker,
@@ -160,7 +161,7 @@
 }

 template("v8_header_set") {
-  source_set(target_name) {
+  jumbo_source_set(target_name) {
     forward_variables_from(invoker, "*", [ "configs" ])
     configs -= v8_remove_configs
     configs += v8_add_configs
--- a/third_party/blink/renderer/platform/fonts/font_matching_metrics.cc 2020-04-11 19:11:33.505798604 +0200
+++ b/third_party/blink/renderer/platform/fonts/font_matching_metrics.cc 2020-04-11 20:31:54.822271813 +0200
@@ -15,7 +15,7 @@
 enum FontLoadContext { kTopLevel = 0, kSubFrame };

 template <typename T>
-HashSet<T> Intersection(const HashSet<T>& a, const HashSet<T>& b) {
+HashSet<T> HSIntersection(const HashSet<T>& a, const HashSet<T>& b) {
   HashSet<T> result;
   for (const T& a_value : a) {
     if (b.Contains(a_value))
@@ -52,16 +52,16 @@
   ukm::builders::FontMatchAttempts(source_id_)
       .SetLoadContext(top_level_ ? kTopLevel : kSubFrame)
       .SetSystemFontFamilySuccesses(ukm::GetExponentialBucketMin(
-          Intersection(successful_font_families_, system_font_families_).size(),
+          HSIntersection(successful_font_families_, system_font_families_).size(),
           kUkmFontLoadCountBucketSpacing))
       .SetSystemFontFamilyFailures(ukm::GetExponentialBucketMin(
-          Intersection(failed_font_families_, system_font_families_).size(),
+          HSIntersection(failed_font_families_, system_font_families_).size(),
           kUkmFontLoadCountBucketSpacing))
       .SetWebFontFamilySuccesses(ukm::GetExponentialBucketMin(
-          Intersection(successful_font_families_, web_font_families_).size(),
+          HSIntersection(successful_font_families_, web_font_families_).size(),
           kUkmFontLoadCountBucketSpacing))
       .SetWebFontFamilyFailures(ukm::GetExponentialBucketMin(
-          Intersection(failed_font_families_, web_font_families_).size(),
+          HSIntersection(failed_font_families_, web_font_families_).size(),
           kUkmFontLoadCountBucketSpacing))
       .Record(ukm_recorder_);
 }
--- a/storage/browser/quota/quota_settings.cc 2020-04-11 19:11:56.809131010 +0200
+++ b/storage/browser/quota/quota_settings.cc 2020-04-11 21:16:14.728831862 +0200
@@ -21,7 +21,7 @@

 namespace {

-const int64_t kMBytes = 1024 * 1024;
+const int64_t QSkMBytes = 1024 * 1024;
 const int kRandomizedPercentage = 10;

 // Skews |value| by +/- |percent|.
@@ -35,7 +35,7 @@
   // The incognito pool size is a fraction of the amount of system memory,
   // and the amount is capped to a hard limit.
   double incognito_pool_size_ratio = 0.1;  // 10%
-  int64_t max_incognito_pool_size = 300 * kMBytes;
+  int64_t max_incognito_pool_size = 300 * QSkMBytes;
   if (base::FeatureList::IsEnabled(features::kIncognitoDynamicQuota)) {
     const double lower_bound = features::kIncognitoQuotaRatioLowerBound.Get();
     const double upper_bound = features::kIncognitoQuotaRatioUpperBound.Get();
--- a/chrome/browser/performance_manager/decorators/process_priority_aggregator.cc       2020-04-11 19:11:54.455797767 +0200
+++ b/chrome/browser/performance_manager/decorators/process_priority_aggregator.cc    2020-04-11 21:39:36.945442260 +0200
@@ -22,15 +22,15 @@

 namespace {

-class DataImpl : public ProcessPriorityAggregator::Data,
-                 public NodeAttachedDataImpl<DataImpl> {
+class PPADataImpl : public ProcessPriorityAggregator::Data,
+                 public NodeAttachedPPADataImpl<PPADataImpl> {
  public:
   using StorageType = ProcessPriorityAggregatorAccess::StorageType;

   struct Traits : public NodeAttachedDataOwnedByNodeType<ProcessNodeImpl> {};

-  explicit DataImpl(const ProcessNode* process_node) {}
-  ~DataImpl() override {}
+  explicit PPADataImpl(const ProcessNode* process_node) {}
+  ~PPADataImpl() override {}

   static std::unique_ptr<NodeAttachedData>* GetUniquePtrStorage(
       ProcessNodeImpl* process_node) {
@@ -106,7 +106,7 @@
 // static
 ProcessPriorityAggregator::Data* ProcessPriorityAggregator::Data::GetForTesting(
     ProcessNodeImpl* process_node) {
-  return DataImpl::Get(process_node);
+  return PPADataImpl::Get(process_node);
 }

 ProcessPriorityAggregator::ProcessPriorityAggregator() = default;
@@ -114,7 +114,7 @@

 void ProcessPriorityAggregator::OnFrameNodeAdded(const FrameNode* frame_node) {
   auto* process_node = ProcessNodeImpl::FromNode(frame_node->GetProcessNode());
-  DataImpl* data = DataImpl::Get(process_node);
+  PPADataImpl* data = PPADataImpl::Get(process_node);
   data->Increment(frame_node->GetPriorityAndReason().priority());
   // This is a nop if the priority didn't actually change.
   process_node->set_priority(data->GetPriority());
@@ -123,7 +123,7 @@
 void ProcessPriorityAggregator::OnBeforeFrameNodeRemoved(
     const FrameNode* frame_node) {
   auto* process_node = ProcessNodeImpl::FromNode(frame_node->GetProcessNode());
-  DataImpl* data = DataImpl::Get(process_node);
+  PPADataImpl* data = PPADataImpl::Get(process_node);
   data->Decrement(frame_node->GetPriorityAndReason().priority());
   // This is a nop if the priority didn't actually change.
   process_node->set_priority(data->GetPriority());
@@ -140,7 +140,7 @@
   // Update the distinct frame priority counts, and set the process priority
   // accordingly.
   auto* process_node = ProcessNodeImpl::FromNode(frame_node->GetProcessNode());
-  DataImpl* data = DataImpl::Get(process_node);
+  PPADataImpl* data = PPADataImpl::Get(process_node);
   data->Decrement(previous_value.priority());
   data->Increment(new_value.priority());
   // This is a nop if the priority didn't actually change.
@@ -160,8 +160,8 @@
 void ProcessPriorityAggregator::OnProcessNodeAdded(
     const ProcessNode* process_node) {
   auto* process_node_impl = ProcessNodeImpl::FromNode(process_node);
-  DCHECK(!DataImpl::Get(process_node_impl));
-  DataImpl* data = DataImpl::GetOrCreate(process_node_impl);
+  DCHECK(!PPADataImpl::Get(process_node_impl));
+  PPADataImpl* data = PPADataImpl::GetOrCreate(process_node_impl);
   DCHECK(data->IsEmpty());
   DCHECK_EQ(base::TaskPriority::LOWEST, process_node_impl->priority());
   DCHECK_EQ(base::TaskPriority::LOWEST, data->GetPriority());
@@ -171,7 +171,7 @@
     const ProcessNode* process_node) {
 #if DCHECK_IS_ON()
   auto* process_node_impl = ProcessNodeImpl::FromNode(process_node);
-  DataImpl* data = DataImpl::Get(process_node_impl);
+  PPADataImpl* data = PPADataImpl::Get(process_node_impl);
   DCHECK(data->IsEmpty());
 #endif
 }

Here is my work. It works for ungoogled-chromium-80.0.3987.163_p1.ebuild, but the -81 needs a few minor fixes.
https://github.com/sinetek/pf4-jumbo
https://github.com/sinetek/pf4-jumbo/commit/403b81ccf9c9af2ddb681b039d05b9b6dc745694
In my opinion this is easy enough to maintain for the gains it brings. It's at most 1-2 fixes per new revision.

@sinetek Have you tried removing jumbo_file_merge_limit? This makes it 50, which should make it even faster, but guarantees more namespace clashes.

I haven't tried, there would be diminishing returns at one point. The old
behaviour in your ebuild is 8. That is easily tunable.
Regarding the commit, it seems I have a bit more work to do on it because a
few other conflicts have come up.
There is a way to simply add files to the ignore list instead of manually
patching every little compiler nit. A few exceptions will not extend the
build time by anything, that's what I will go back and do.
Help appreciated for integration, bug fixing or benching.
On Sun, Apr 12, 2020, 02:21 PF4Public<EMAIL_ADDRESS>wrote:

@sinetek https://github.com/sinetek Have you tried removing
jumbo_file_merge_limit? This makes it 50, which should make it even
faster, but guarantees more namespace clashes.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Eloston/ungoogled-chromium/issues/928#issuecomment-612542068,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABNCTC4SHV27GRGJBRCHEXTRMECQPANCNFSM4KRC2M4A
.

The old behaviour in your ebuild is 8

I had to lover it from default 50 at some point, when namespace clashes started to occur.

there would be diminishing returns at one point.

… at around 100 ;) But you will anyway need lots (of lots) of RAM then.
Jumbo - bringing shorter compile times to you

Awesome that you have this information. This is an incredible speedup. For
the ebuild we should have it either ask for N or have it pulled out of an
environment variable.
All the conflicts can just go to the exception list.
Thoughts?
On Sun, Apr 12, 2020, 02:48 PF4Public<EMAIL_ADDRESS>wrote:

The old behaviour in your ebuild is 8
I had to lover it from default 50 at some point, when namespace clashes
started to occur.
there would be diminishing returns at one point.
… at around 100 ;) But you will anyway need lots (of lots) of RAM then.
Jumbo - bringing shorter compile times to you
https://docs.google.com/document/d/19jGsZxh7DX8jkAKbL1nYBa5rcByUL2EeidnYsoXfsYQ/edit#
[image: image]
https://user-images.githubusercontent.com/10319700/79057872-ca452f80-7c6e-11ea-9305-39a482df801a.png
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Eloston/ungoogled-chromium/issues/928#issuecomment-612544818,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABNCTC6EMIRMOA2IWVX52FTRMEFU5ANCNFSM4KRC2M4A
.

One came up to mind just after posting. As you know the OOM response on
Linux is extremely poor, we should check RAM or at least make it hard to
load N=500 on a 8GB system... That would require a reboot, probably.
On Sun, Apr 12, 2020, 02:53 Philippe Michaud-Boudreault<EMAIL_ADDRESS>wrote:

Awesome that you have this information. This is an incredible speedup. For
the ebuild we should have it either ask for N or have it pulled out of an
environment variable.
All the conflicts can just go to the exception list.
Thoughts?
On Sun, Apr 12, 2020, 02:48 PF4Public<EMAIL_ADDRESS>wrote:

The old behaviour in your ebuild is 8
I had to lover it from default 50 at some point, when namespace clashes
started to occur.
there would be diminishing returns at one point.
… at around 100 ;) But you will anyway need lots (of lots) of RAM then.
Jumbo - bringing shorter compile times to you
https://docs.google.com/document/d/19jGsZxh7DX8jkAKbL1nYBa5rcByUL2EeidnYsoXfsYQ/edit#
[image: image]
https://user-images.githubusercontent.com/10319700/79057872-ca452f80-7c6e-11ea-9305-39a482df801a.png
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Eloston/ungoogled-chromium/issues/928#issuecomment-612544818,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABNCTC6EMIRMOA2IWVX52FTRMEFU5ANCNFSM4KRC2M4A
.

As you know the OOM response on
Linux is extremely poor, we should check RAM or at least make it hard to
load N=500 on a 8GB system...

I agree, also there's no good way to communicate memory usage to the compiler. I think the best we can do is to use cgroups to limit the memory usage to some percentage of the total memory (or a fixed number). There are some nice examples on ArchWiki: https://wiki.archlinux.org/index.php/cgroups

have it pulled out of an environment variable

EXTRA_GN is already available in Gentoo:
set -- gn gen --args="${myconf_gn} ${EXTRA_GN}" out/Release

All the conflicts can just go to the exception list.

That's fine, several exceptions are fine, I suppose. But if this list grows bigger and bigger due to Google efforts…

we should check RAM

use cgroups

I'd suggest against overcomplicating things. It used to work fine with 50, cut almost half the compile time and nobody requested to lower it (on Gentoo and from my ebuild at least). I have never had a single OOM while compiling myself in a kvm-guest. Even RAM usage was never observed to be higher than 4-6 gigs. I admit, that compiling Chromium is a lengthy process and RAM usage might have peaked at some point, when I was not looking at it. I also admit, that I compile it in a kvm-guest with 16-19 GB of dedicated RAM, but still… The biggest issue here is to patch back jumbo support.
If @sinetek succeeds in making this patch, we will have some more things to discuss.

Where does this patch belong? Options are:

Here in this particular repo. If that's the case, @Eloston might have a say on the subject.
Platform repos. In this case, there would be a issue on how to syncronise this patch across all the repos. My Gentoo overlay for example does not use portable-linux repo, only this one.

This patch needs to be maintained from version to version. Who's going to be doing this?

I don't think the patch proposed in https://github.com/Eloston/ungoogled-chromium/issues/928#issuecomment-612523039 is sustainable in the long-term. Patching the source code to fix symbol conflicts can only become more difficult after each new version, especially as Google has stated that they will not guarantee Jumbo compatibility. Not to mention renaming symbols will probably break tests too. We don't want another patch that keeps growing in complexity like Safe Browsing patches.
My approach in https://github.com/Eloston/ungoogled-chromium/issues/928#issuecomment-612317405 does not require modifying the source code (aside from some GN changes), and instead statically analyzes all symbol conflicts before the build. However, this approach is also non-trivial; creating this tool, testing it, and optimizing its performance (I suspect it might be challenging in both runtime and memory usage) will take some time.

I think, I will close this issue, as there was not much of a discussion recently and re-implementing jumbo seems prohibitively complicated at the moment.
If at a later time something changes, one could continue the discussion and reopen the issue.

Perhaps Clang modules can do the thing?

The primary user-level feature of modules is the import operation, which provides access to the API of software libraries. However, today’s programs make extensive use of #include, and it is unrealistic to assume that all of this code will change overnight. Instead, modules automatically translate #include directives into the corresponding module import. For example, the include directive
#include <stdio.h>

will be automatically mapped to an import of the module std.io. Even with specific import syntax in the language, this particular feature is important for both adoption and backward compatibility: automatic translation of #include to import allows an application to get the benefits of modules (for all modules-enabled libraries) without any changes to the application itself. Thus, users can easily use modules with one compiler while falling back to the preprocessor-inclusion mechanism with other compilers.

I'm trying to build Chromium with -fmodules in cflags. It seems to work, as the directory I specified in -fmodules-cache-path is populated. I'll check out the size of the cache produced and whether this out of the box affects build speed.

Perhaps the Clang modules will do the trick?
Maintaining the patch would probably be much easier than with jumbo build's one.

The primary user-level feature of modules is the import operation, which provides access to the API of software libraries. However, today’s programs make extensive use of #include, and it is unrealistic to assume that all of this code will change overnight. Instead, modules automatically translate #include directives into the corresponding module import. For example, the include directive
#include <stdio.h>

will be automatically mapped to an import of the module std.io. Even with specific import syntax in the language, this particular feature is important for both adoption and backward compatibility: automatic translation of #include to import allows an application to get the benefits of modules (for all modules-enabled libraries) without any changes to the application itself. Thus, users can easily use modules with one compiler while falling back to the preprocessor-inclusion mechanism with other compilers.

Just need to make a module map for all (or almost) header files that are used in-tree when building Chromium.
Extra Clang utility modularize generates a module map based on the list of header files.

Perhaps the Clang modules will do the trick?

Could you please elaborate on how clang's module system could help? I have an impression of it being orthogonal to the problem of long building time of Chromium.

Could you please elaborate on how clang's module system could help?

Clang modules use PCH (Precompiled Headers) to automatically cache built modules. Because of this, each module only needs to be built once.
There's a problem with modules that header files have to conform to certain practices.

Note that headers with dependents will be ignored with a warning, as the Clang module mechanism doesn’t support headers that rely on other headers being included first.

I wrote a script to generate a dependency tree of header files. According to rough calculations (just using find), out of ~120,000 header files, ~40,000 are being excluded by modularize.
While effective use of Clang modules does not require major changes to the build system, it will probably require code refactoring, which is why the Chromium development team rejected this option as an alternative to jumbo builds.
However, it should be possible to use PCH, which should not require changes in the code, but in the build system definitely.
Cmake is supported natively by PCH. Libreoffice has been providing support for building with PCH for quite some time now

@PF4Public, Clang modules use PCH (Precompiled Headers) to automatically cache built modules. Because of this, each module only needs to be built once.

You're not telling that among 50-60 thousand compile targets, that Chromium builds, most are duplicates, are you?

You're not telling that among 50-60 thousand compile targets, that Chromium builds, most are duplicates, are you?
@PF4Public, when building C/C++ code dependencies are ‘included’, i.e. the compiler ends up processing one big ‘file’. In this case, headers are processed more times than necessary, because for each translation unit (target) the compiler needs to process all included dependencies. Doesn't it? That's why jumbo build helps, because it reduces the number of duplicate header processing.

By the way, there is already some support for precompiled headers in Chromium, but it was implemented long before LLVM 11 was released in 2020, in which PCHs were redesigned, and offers no real advantage, especially on Linux.

While effective use of Clang modules does not require major changes to the build system, it will probably require code refactoring, which is why the Chromium development team rejected this option as an alternative to jumbo builds.

The relevant discussion can be found here.

As a data point: Inside Chromium the time to process headers is typically 80-95% of the total time processing a cc file. Maybe not surprising when the headers are around 240k lines, and the cc files themselves 50-500 lines.

I don’t think there is much we can do at this point. Perhaps Google will work on it.
Sent from Proton Mail for iOS
On Sun, Oct 6, 2024 at 02:10, Benjamino Masyura @.***(mailto:On Sun, Oct 6, 2024 at 02:10, Benjamino Masyura < wrote:

While effective use of Clang modules does not require major changes to the build system, it will probably require code refactoring, which is why the Chromium development team rejected this option as an alternative to jumbo builds.

The relevant discussion can be found here.

As a data point: Inside Chromium the time to process headers is typically 80-95% of the total time processing a cc file. Maybe not surprising when the headers are around 240k lines, and the cc files themselves 50-500 lines.

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: @.***>


No network, no staking.

Issue description
Network doesn't work, staking doesnt work. Pls held
Firefly version
v1.3.0
OS details
Win 64
Can the issue be reliably reproduced?
No
Steps to reproduce the issue
I have 109 mi on a balance. Wanted to stake that in firefly wallet. Did that yesturday. But today no network and no staking.
Expected behaviour
Staking started yesterday
Actual behaviour
No network, no staking.
Errors
0

This is not a bug with firefly, the network is currently not issues new milestones. Please follow official IOTA channels to be notified when this issue is resolved. https://twitter.com/iota/status/1476126255506501632?s=20

Hey! I staked that yesterday! And it was OK! I have a screenshot with it. But today there is no staking! I can show.

Here it is


add _magnitude to ureg().to() when passing arrays to ase.Atoms

as mentioned here;
https://github.com/nomad-coe/nomad/issues/71
this warnings are raised when normalizing vasp outputs  in script:
'''
/scratch/AnacondaEnvironments/anaconda3/envs/nomad/lib/python3.9/site-packages/ase/cell.py:97: UnitStrippedWarning: The unit of the quantity is stripped when downcasting to ndarray.
cell = np.array(cell, float)
'''
'''
/scratch/AnacondaEnvironments/anaconda3/envs/nomad/lib/python3.9/site-packages/ase/atoms.py:424: UnitStrippedWarning: The unit of the quantity is stripped when downcasting to ndarray.
a = np.array(a, dtype, order='C')
'''
this fixes:
https://github.com/nomad-coe/nomad/commit/1e2200c9f5647b70641a4589f809f5d8c7685306

This is now fixed in the develop branch by this commit.


Why isn't the ajax synchronized？

Vue.js version
2.0.2
Reproduction Link

Steps to reproduce
What is Expected?
What is actually happening?

the ajax is  synchronized，but actually is asynchronization，the description of problem in the picture


ebpf_hash_table should initialize ebpf_random if needed

Describe the feature you'd like supported
Currently, use of ebpf_hash_table requires ebpf_random to be initialized.
Proposed solution
ebpf_hash_table should check if random has been initialized and then initialize it if not.
Additional context
No response

Consider: Removing the dependency on ebpf_random instead if unnecessary.

K-12: Static initialization sequence QED. 🌻


[Feat]: Add GitHub Actions for Release and Changelog

What feature?
I have an idea to use GitHub Actions to automate the release and changelog creation process.
Add screenshots
Add screenshots
Code of Conduct

[ ] I agree to follow this project's Code of Conduct

I think its a duplicate issue. I'll close it.


act_runner registration fails with gitea hosted on OpenBSD-current

Description
Involved systems

gitea is running on an OpenBSD-current VPS
the runner was built as documented on my archlinux desktop machine

Versions
Runner
$ act_runner -v
act_runner version v0.2.5+4-ged35b09

$ uname -a
Linux 6.4.11-zen2-1-zen #1 ZEN SMP PREEMPT_DYNAMIC Sat, 19 Aug 2023 15:38:10 +0000 x86_64 GNU/Linux

# gitea -v
Gitea version 1.20.2 built with go1.21.0 : purego, sqlite, sqlite_unlock_notify

# uname -a
OpenBSD 7.3 GENERIC#1290 amd64

Config
Actions are enabled and gitea itself is run with:
[server]
PROTOCOL = fcgi+unix
HTTP_ADDR = /var/www/run/gitea.sock
[...]

Incoming connections are handled by OpenBSD's httpd:
# cat /etc/httpd.conf.gitea
server "XXXX" {
    listen on * port 80
        location "/.well-known/acme-challenge/*" {
                root "/acme"
                request strip 2
        }
                location * {
                block return 302 "https://$HTTP_HOST$REQUEST_URI"
        }
    }

server "XXXX" {
        listen on * tls port 443
        tls {
                certificate "/etc/ssl/XXXX"
                key "/etc/ssl/private/XXXX"
        }
        connection { max requests 500, timeout 3600 }
        location "*" {
            fastcgi socket "/run/gitea.sock"
        }
}

Issue
When trying to register the runner it errors and complains about a malformed http response:
$ act_runner register --no-interactive --instance https://XXX --token $GITEA_ACTIONS_TOKEN
INFO Registering runner, arch=amd64, os=linux, version=v0.2.5+4-ged35b09.
WARN Runner in user-mode.
INFO Runner name is empty, use hostname 'XXX'.
ERRO Cannot ping the Gitea instance server         error="unavailable: net/http: HTTP/1.x transport connection broken: malformed HTTP response \"8\""
ERRO Cannot ping the Gitea instance server         error="unavailable: net/http: HTTP/1.x transport connection broken: malformed HTTP response \"8\""

Logs
Gitea doesn't log anything about the runner's registration request even if I set the logging mode to Trace. Also the webserver doesn't log anything except of the incoming request itself:
server XXX, client 1 (1 active), X.X.X.X:50786 -> X.X.X.X:443, closed

Gitea Version
1.20.2 built with go1.21.0 : purego, sqlite, sqlite_unlock_notify
Can you reproduce the bug on the Gitea demo site?
No
Log Gist
No response
Screenshots
No response
Git Version
git version 2.41.0
Operating System
OpenBSD current GENERIC#1290 amd64
How are you running Gitea?
As described gitea is running on a OpenBSD-current VPS and installed via pkg_add
Database
SQLite

Is this related to fcgi+unix? Have you tried other ways? 🤔


Finding Matlab

At the moment, we plan on natively supporting Matlab in a thorough manner. Therefore, we need to be able to reliably call out to Matlab. We could just say that users have to sort this themselves, but that is very unfriendly on OSX and Windows where matlab is not on the path by default. We should probably have some sort of logic that tries to work out which Matlab is installed, and if it fails it should fallback to something like the caching mechanism and store it in the .menpobench config file.
We should probably add Matlab existence to YAML validation as well.

Addressed with #8


Update README.md for sublevel version 6

see fixed issue: https://github.com/levelgraph/levelgraph/issues/184#issue-417665214

Coverage remained the same at 95.413% when pulling a3252d2973355e8c526160275fd5b32ab836595f on vanvuongngo:patch-1 into 33ece87a70e03c4ad2d93fc6d73ad5a81a8075a2 on levelgraph:master.


[WFLY-17871] added test-case for managed executor service defined via…

… annotation
This is a test-case for managed executor service defined via annotations. I can't see any such test in the test-suite.
Issue: https://issues.redhat.com/browse/WFLY-17871

@istudens I see a failure in the new test case https://ci.wildfly.org/buildConfiguration/WF_PullRequest_GalleonLinuxJdk11/366184?expandBuildProblemsSection=true&hideProblemsFromDependencies=false&expandBuildTestsSection=true&hideTestsFromDependencies=false&expandBuildChangesSection=true

@emmartins it looks like this one is waiting for a review from you?

/retest

@bstansberry done

Thanks @istudens


Override date type with IDEA

Versions:

ide-helper Version: <IP_ADDRESS>

Description:
Currently, a lot of things in the generated blocks indicate that a Carbon object is returned for dates.
Would it be possible to detect if Date::use(CarbonImmutable::class) is set, and then generate the respective docblocks with CarbonImmutable hints?

Sorry, this should actually be a feature req I think.


nodePackages: update to latest

Description of changes
$ pkgs/development/node-packages/generate.sh

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 242263 run on x86_64-linux 1

55 packages built:

balanceofsatoshis
bat-extras.prettybat
bitwarden-cli
breitbandmessung
castnow
commitlint
concurrently
create-cycle-app
emanote
emanote.data
fast-cli
flexoptix-app
graphite-cli
gtop
hueadm
joplin
lemmy-ui
lighthouse
lv_img_conv
morgen
openmoji-black
openmoji-color
pm2
postcss-cli
pot
pscid
pulsar
redoc-cli
reveal-md
shepherd
slack
teams
theLoungePlugins.plugins.closepms
theLoungePlugins.plugins.giphy
theLoungePlugins.themes.flat-blue
theLoungePlugins.themes.flat-dark
uppy-companion
vimPlugins.coc-clangd
vimPlugins.coc-metals
vimPlugins.coc-sqlfluff
vimPlugins.coc-stylelint
vimPlugins.coc-toml
vimPlugins.coc-vetur
vimPlugins.markdown-preview-nvim
vscode
vscode-extensions.rust-lang.rust-analyzer
vscode-fhs
vscode-langservers-extracted
vscode-with-extensions
vscodium
vscodium-fhs
wasm-text-gen
wrangler
yaml-language-server
zx


no-use-before-declare doesn't handle class members

Bug Report

TSLint version: 5.5.0
TypeScript version: 2.4.1
Running TSLint via: gulp

TypeScript code being linted
class A { b: B = new B(); }
class B { }

with tslint.json configuration:
no-use-before-declare: true

Actual behavior
variable B is used before declaration
Expected behavior
No error

I also experience the same issue inside class members functions.
class A {
  test() {
      const b = new B(); // variable 'B' used before declaration
  }
}

class B {}

What about the code
export class One {
    toSecond() {
        return new Two();
    }
}

export class Two {
    toFirst() {
        return new One();
    }
}

It will not fail at runtime.


UPSTREAM:  Add standard Makefile targets

These Makefile targets should make building and CI easier.

I'm going to close this out, since https://github.com/kubernetes/cloud-provider-gcp/pull/590 is opened upstream.


Add filter and sortBy to category.products #10917

Note: This is a 3.9 port of #10997

I want to merge this change because it adds filtering and sorting capabilities to category.products query.

Impact

[ ] New migrations
[x] New/Updated API fields or mutations
[ ] Deprecated API fields or mutations
[ ] Removed API types, fields, or mutations
[ ] Documentation needs to be updated

Pull Request Checklist

[ ] Privileged queries and mutations are guarded by proper permission checks
[ ] Database queries are optimized and the number of queries is constant
[ ] Database migration files are up to date
[x] The changes are tested
[x] GraphQL schema and type definitions are up to date
[x] Changes are mentioned in the changelog

Hi @maarcingebala ,
Please check this PR as this is a port of #10997 to 3.9
Best and thank you,
Ogün


Add && overload for Instance::FullName and Database::FullName.

Some of the code can discard the instance and database once they get the full name, seems like a waste to copy in those cases.

Since https://github.com/googleapis/google-cloud-cpp-spanner/pull/652, these classes no longer store the full_name as a member, so moving out of that member wouldn't be very beneficial. Instead, they compute the FullName() only when asked. So I think this issue is no longer applicable.


Feature Request: Document how to change field value data types

Current Situation: At present (v 1.0), it's not possible to change a field value data type from (say) a boolean to a float64, and #2651 makes it clear that this won't happen soon.  That's okay.
Feature Request: In the meantime, it would be useful to document one or more techniques that have the overall effect of changing data types.  For example (from #2651) you can:

use a new field name,
use a new retention policy,
wait for a new shard to be created and drop the old shards
... etc.

Use case: Judging from comments on the web, a number of people stumble across this behaviour and would like a workaround.

@rdpoor Sounds like docs issue. Opening mirror issue on docs repo.


Bunch of new commands revolving around exports

Export-DbaInstance needs a smidge more work, then will be ready

We not going to add the warning on deprecated parameters (e.g. FilePath to just Path)?

naw, just gonna leave the alias in forever. it's useful for gci piping

thank you for looking

also wrt get-dbatrigger, just gonna make an announcement or something because the new proper commands are so drastically different. there's no new replacement.


Update S3 to JSON job to use v4.6.1 of archive-map.json

@thomasyu888 It's unlikely that archive-map.json will be updated in the future considering that we are moving towards self-referencing data.


⚠ Bofrost.at has degraded performance

In fc31c68, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1894 ms

Resolved: Bofrost.at performance has improved in 45828c5 after 13 minutes.


 How would it feel if someone told that to yourself?

If you don't like DICCS don't install it. The documentation clearly states:

While we believe most people are fond of DICSS, we realize that DICSS isn't for everyone. There are plenty of alternatives. Feel free to use them instead. And please, never force someone to use DICSS if they're not willing.

Oh no! A JavaScript library made a joke that triggered me!

 People like you give social justice warriors the condescending stereotype they currently have.  HINT: it's not just for popular speech ass!


Solve the CVE-2019-13990?

what a wondeful thing it is If someone can solve this security problem, and release on 2.3.x.

Hi, please see: https://github.com/quartz-scheduler/quartz/issues/467

@zhaowenxishi  please close this ticket as #467 was fixed.

Fixed under #467


SonarQube Api Importer support multi branch

Is your feature request related to a problem? Please describe
SonarQube api/hotspots/search support branch param, however DefectDojo SonarQube Api Import only use projectKey param to request bugs and hotspots.
https://github.com/DefectDojo/django-DefectDojo/blob/master/dojo/tools/sonarqube_api/importer.py

Describe the solution you'd like
Holp to add branch param into function find_issues and find_hotspots, default branch value can be master.

duplicate of #5348 ?

duplicate of #5348 ?

Yes you're right.


Customisation: Ability to change cursor colour

It seems, for some colour, it's hard to the read, through the grey cursor colou

r
Can we have rectangle?

I've the same issue...

or can we invert colors ?

The pull request above should solve this.
Don't mind the other settings in the toml file. Those are additional features I'm working on.


Include individual count in bullfrog dataset on gbif

@DimEvil Currently data concerning individual counts is lost in translation from recorder to gbif
https://www.gbif.org/occurrence/1936700473.
Is it possible to include this info in a following update ?

Individual count is added


This is a test commit which adds a file

Added testfile.txt

@aditirajagopal

@jecarey
Was pretty skeptical that this would work. Looks like in order to submit a PR of a "fork of a fork," you need to go to the parent fork's repo page, and submit the PR there. if you do it from your child fork, it will (by default) try to submit a PR for the grandparent repo, and the filter doesn't let you find the parent fork.
What a weird dysfunctional family of repos.

Please sign your commits following these rules:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "moxiegirl-patch-README" git@github.com:docker/docker.git somewhere
$ cd somewhere
$ git commit --amend -s --no-edit
$ git push -f
Ammending updates the existing PR. You DO NOT need to open a new one.

@MHBauer added the signature.

LGTM!

looks good. success!


Installing to a different $NODE_PATH gives "Cannot resolve module" error for loaders

I expected the loaders to work the same way as require() and check for node modules in the env var $NODE_PATH if it's set.
In my webpack.config.js:
var config = {
  module: {
    loaders: {
      test: /\.js$/,
      loader: 'babel',
    }
  }
}
module.exports = config;

This doesn't work if I npm install to a different NODE_PATH instead of the local ./node_modules. It does work if I set the loader name to an absolute path, e.g.  loader: process.env.NODE_PATH + '/babel-loader'.
Can the loaders be updated to check NODE_PATH when specified by name?

You're looking for the resolveLoaders option.

You can add process.env.NODE_PATH to your config.
Adding it to resolve.root will cause this to be a top-level search-point for modules (likely what you want).
Adding it to resolve.modulesDirectories would include a folder name for each subsequent search (probably not what you want, but mentioned for completeness).
So, in your case, I'd probably do:
module.exports = {
  resolve: {
    root: [ process.env.NODE_PATH || "node_modules" ]
  }
}

Closing given this was answered.


Move usegalaxy.eu to traefik – reduce TTL

https://github.com/usegalaxy-eu/issues/issues/441

⚠ Merge this only ~ 3h before the switch is done and increase directly after the switch.
Lowering the TTL results in more DNS requests in Route53 and consequently higher costs.

Always feels so wrong to create the branch in upstream 😅

Actually; infrastructure does not run regularily, so I would consider to reduce the TTL manually and just change the IP here, when we do the switch


[feature]Supports traversal of Doris FE nodes when searching for Doris BE on branch flink-before-1.13

Proposed changes
Problem Summary:
目前，使用 flink-doris-connector 写入 Doris ，当配置了多个 Doris FE 节点的前提下， 内部通过 FE 节点请求到 BE 时会随机选择其中一个 FE 提交 http 请求 BE 节点地址；在生产环境中，由于机器原因或者是版本升级需要滚动重启 FE 服务时，由于某个 FE 服务暂时不可用，可能会引起 flink 作业异常退出；
为了降低集群变更或偶发异常影响实时流稳定，考虑通过对所有配置的  Doris FE 节点发送 http 请求查询 Doris BE。
Currently, when Flink-doris-Connetor writes Doris, it will queries Doris FE node through one of the Doris BE nodes randomly. It may causing job failures when one of Doris BE node in cluster is out of service. For example, sometime  machine has problem and need to restart or admin need to Actively restart service In the process of upgrading the version
Checklist(Required)

Does it affect the original behavior: (Yes)
Has unit tests been added: (No Need)
Has document been added or modified: (No Need)
Does it need to update dependencies: (No)
Are there any changes that cannot be rolled back: (No)

Further comments

Please update the documentation, both in Chinese and English

Please update the documentation, both in Chinese and English
Hi, which documentation, can you provided a path? I'm sorry I couldn't find the file

LGTM
No need to update document, because we already support specify multi FE nodes before


[Serving][Grammar] Integrate JSON grammar into the generation pipeline

This PR is the 3rd part of the grammar-guided generation. This intregrates the grammar framework into the generation process, and supports JSON output for now.
The API this PR provides is compatible with the OpenAI api.
APIs
Python API
@dataclass
class ResponseFormat:
    type: Literal["text", "json_object"] = "text"
    json_schema: Optional[str] = None

@dataclass
class GenerationConfig:
        response_format: ResponseFormat = ResponseFormat(type="text")

Rest API
response_format: { "type": "text" } # text generation, by default
response_format: { "type": "json_object" } # json generation
response_format: { "type": "json_object", json_schema="..."} # json generation with schema

JSON generation with schema is not supported yet, but has been planned to be realized in the future.
Performance
Without JSON
Single token prefill latency: 891.2234 ms/tok
Single token decode latency: 31.3399 ms/tok
Prefill token throughput: 4693.3077 tok/s
Decode token throughput: 226.4406 tok/s
Overall token throughput: 470.3180 tok/s

With JSON
Single token prefill latency: 219.2287 ms/tok
Single token decode latency: 29.1399 ms/tok
Prefill token throughput: 7392.1555 tok/s
Decode token throughput: 179.2296 tok/s
Overall token throughput: 1052.1996 tok/s

We observed a slight decrease in performance under JSON mode. This will be further optimized in the future.
Thanks to @tqchen and @MasterJH5574

Congratulations! Regarding the performance, what's the benchmarking setting (e.g. which GPU, prefill & decode length)?

BTW, I noticed that this input may cause the engine “running forever”:

def test_batch_generation_hang_forever():
    # Initialize model loading info and KV cache config
    model = ModelInfo(model_path, model_lib_path=model_lib_path)
    kv_cache_config = KVCacheConfig(page_size=16)
    # Create engine
    engine = Engine(model, kv_cache_config)

    prompts = [
        """[INST] <<SYS>>
You are a helpful, respectful and honest assistant.
<</SYS>>

  [INST] Hello! Our project is MLC LLM. [/INST]"""
    ]

    generation_cfg = GenerationConfig(
        temperature=1.0,
        top_p=1.0,
        max_tokens=None,
        stop_strs=["[INST]"],
        stop_token_ids=[2],
        response_format=ResponseFormat(type="json_object"),
    )

    # Generate output.
    output_texts, _ = engine.generate(prompts, generation_cfg)
    for req_id, output in enumerate(output_texts):
        print(f"Prompt {req_id}: {prompts[req_id]}")
        print(f"Output {req_id}: {output}\n")

When it “hangs”, it will get stuck until the generation length reaches the model's max context length and then force quits. This does not happen every time. The frequency is like 2-3 times when I run it for 10 times in a row.
I printed out the delta output each time when it hangs. Seems that each time the engine outputs a new line or some empty spaces. We do need to address this issue in this PR -- just want to mark it so that we can follow up later.

@yzh119 Thanks for the comment. The benchmark was runned on AMD Ryzen Threadripper PRO 5975WX 32-Cores and NVIDIA RTX 6000 Ada Generation with frequency 3100MHz.

BTW, I noticed that this input may cause the engine “running forever”:
def test_batch_generation_hang_forever():
    # Initialize model loading info and KV cache config
    model = ModelInfo(model_path, model_lib_path=model_lib_path)
    kv_cache_config = KVCacheConfig(page_size=16)
    # Create engine
    engine = Engine(model, kv_cache_config)

    prompts = [
        """[INST] <<SYS>>
You are a helpful, respectful and honest assistant.
<</SYS>>

  [INST] Hello! Our project is MLC LLM. [/INST]"""
    ]

    generation_cfg = GenerationConfig(
        temperature=1.0,
        top_p=1.0,
        max_tokens=None,
        stop_strs=["[INST]"],
        stop_token_ids=[2],
        response_format=ResponseFormat(type="json_object"),
    )

    # Generate output.
    output_texts, _ = engine.generate(prompts, generation_cfg)
    for req_id, output in enumerate(output_texts):
        print(f"Prompt {req_id}: {prompts[req_id]}")
        print(f"Output {req_id}: {output}\n")

When it “hangs”, it will get stuck until the generation length reaches the model's max context length and then force quits. This does not happen every time. The frequency is like 2-3 times when I run it for 10 times in a row.
I printed out the delta output each time when it hangs. Seems that each time the engine outputs a new line or some empty spaces. We do need to address this issue in this PR -- just want to mark it so that we can follow up later.

That's a known issue. Sometimes when grammar-guided generation is enabled, the output of LLMs might be unreasonable although comply with the grammar. E.g.
{"name": "value", "name1": "a long string containing nonsense that exceeds the output length

{"name": "value",

// a lot of useless newlines that exceeds the output length

that may be because the LLM cannot understand the json format correctly, but have to output according to it. We are still checking if this problem still occurs for LLMs fine-tuned on json data.


Translation from new logical model into the chase model

Use the new logical model in the reasoner

Added in https://github.com/knowsys/nemo/commit/21501d88a50f846208da315b9d7b80f2966b1a9e


PowerQuery in Excel and PowerBI

I've been told that PowerQuery is the same in both apps. In that case they should have the same list of data source connections.
The PowerBI documentation lists 160 sources
https://docs.microsoft.com/en-us/power-bi/connect-data/desktop-data-sources
While Excel documentation only lists 40 sources
https://support.microsoft.com/en-us/office/import-data-from-data-sources-power-query-be4330b3-5356-486c-a168-b68e9e616f5a
.
If PQ is identical in both applications, please synchronize the lists.
If not, in the excel documentation, explain why Excel PQ cannot access the same set of data sources.
AND as a workaround, document how you could use PBI to access data in the "extra" sources, then access the PBI file from Excel PQ.
Does this workaround allow refresh to work?
Thanks.
sorry about the bold, your editor apply the funky bolding ..
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 9ca836d9-2353-3725-f4ae-73e8d1ca34ba
Version Independent ID: 80ff9e5a-772f-b1ec-61f0-de544ae9e959
Content: Data sources in Power BI Desktop - Power BI
Content Source: powerbi-docs/connect-data/desktop-data-sources.md
Service: powerbi
Sub-service: pbi-data-sources
GitHub Login: @davidiseminger
Microsoft Alias: davidi

Hi @ronssssss, thanks for your comment. I am passing it on to the doc owner for consideration.

@ronssssss Power Query's data sources has an Excel doc and a Power BI doc

Yes, that is my point exactly. The Excel page is a P.O.C. The Excel page only lists a couple of dozen, while the PowerBI page lists hundreds.
.
WHY THE DIFFERENCE?!?
.
Is PQ in Excel dumbed down, or the same as in PowerBI?

From: rootsmusic @.>
Sent: March 25, 2022 9:25 PM
To: MicrosoftDocs/powerbi-docs @.>
Cc: ronssssss @.>; Mention @.>
Subject: Re: [MicrosoftDocs/powerbi-docs] PowerQuery in Excel and PowerBI (Issue #3579)
@ronsssssshttps://github.com/ronssssss Power Query's data sources has an Excel dochttps://support.microsoft.com/en-us/office/power-query-data-sources-in-excel-versions-e9332067-8e49-46fc-97ff-f2e1bfa0cb16 and a Power BI dochttps://docs.microsoft.com/en-us/power-bi/connect-data/power-bi-data-sources
—
Reply to this email directly, view it on GitHubhttps://github.com/MicrosoftDocs/powerbi-docs/issues/3579#issuecomment-1079569885, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AW7JNZ7YMM77ZD77NHNCQILVBZYRTANCNFSM5NV3ZLFA.
You are receiving this because you were mentioned.Message ID: @.***>

And here is another page:
https://docs.microsoft.com/en-us/power-bi/connect-data/desktop-data-sources
.
If PowerQuery is the same, the list of external data sources should be the same. The simplest way to keep it the same is to redirect the various entries to a single page.

From: rootsmusic @.>
Sent: March 25, 2022 9:25 PM
To: MicrosoftDocs/powerbi-docs @.>
Cc: ronssssss @.>; Mention @.>
Subject: Re: [MicrosoftDocs/powerbi-docs] PowerQuery in Excel and PowerBI (Issue #3579)
@ronsssssshttps://github.com/ronssssss Power Query's data sources has an Excel dochttps://support.microsoft.com/en-us/office/power-query-data-sources-in-excel-versions-e9332067-8e49-46fc-97ff-f2e1bfa0cb16 and a Power BI dochttps://docs.microsoft.com/en-us/power-bi/connect-data/power-bi-data-sources
—
Reply to this email directly, view it on GitHubhttps://github.com/MicrosoftDocs/powerbi-docs/issues/3579#issuecomment-1079569885, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AW7JNZ7YMM77ZD77NHNCQILVBZYRTANCNFSM5NV3ZLFA.
You are receiving this because you were mentioned.Message ID: @.***>

No @ronssssss, PQ supports different connectors in every product and even between different versions of a product.

Fine, but it really would help to clarify the issue if you put ALL the connectors on one page in One table. Base the table on the grid for Excel 2016 Win -2019, 2021, 365 on this page.
https://support.microsoft.com/en-us/office/power-query-data-sources-in-excel-versions-e9332067-8e49-46fc-97ff-f2e1bfa0cb16
.
Put ALL of the versions on the page into a SINGLE grid, add PQ in the various PowerBI options: Desktop, enterprise server etc.
.
The idea I'm trying to suggest is to make it easiest for users to identify which connectors are available when they are moving files between various versions of Excel and PowerBI.  For example of someone exports to Excel from PowerBI to do analysis because they are more familiar with it. Then they move the file to another OS, ie Mac ...
Please also suggest adding an "Expand all" button to the top of the page.
Thanks

From: rootsmusic @.>
Sent: March 26, 2022 11:01 AM
To: MicrosoftDocs/powerbi-docs @.>
Cc: ronssssss @.>; Mention @.>
Subject: Re: [MicrosoftDocs/powerbi-docs] PowerQuery in Excel and PowerBI (Issue #3579)
No @ronsssssshttps://github.com/ronssssss, PQ supports different connectors in every product and even between different versions of a product.
—
Reply to this email directly, view it on GitHubhttps://github.com/MicrosoftDocs/powerbi-docs/issues/3579#issuecomment-1079722485, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AW7JNZ5EQZEVVWQKHNJSQDDVB4YHPANCNFSM5NV3ZLFA.
You are receiving this because you were mentioned.Message ID: @.***>


FW: Some bugs in using Wire library

Heya,
someone found some usage bugs of the wire library

File "Sonoff-Tasmota/lib/Mutichannel_Gas_Sensor/src/MutichannelGasSensor.cpp", function "readData()" has a timeout, but there is no need to wait for something after a Wire.requestFrom().

Credits @Koepel
https://github.com/arendst/Sonoff-Tasmota/issues/2442

This part in "readData()" can be removed:
while(Wire.available() == 0)
{
  if(timeout++ > 100)
    return -2;//time out
  delay(2);
}

There is no timeout with Wire.requestFrom().
This part is correct:
if(Wire.available() != 4)
  return -3;//rtnData length wrong

Testing if the same amount of data is received as is requested will capture all possible errors that the Wire library can detect.


feat(relay): Add relay ingest to SFU

Description
This PR brings lots of improvements to ION:

Allows to ingest relayed streams.
Lock free RTP writer
Sender reports time propagation
And some other minor fixes

There still some things to consider while using relay

[ ] No simulcast supported (WIP), atm simulcast is handled as a normal peer so it relays the highest or lower layer according conf
[ ] Audioobserver not supported on relaying SFU
[ ] Datachannel fanouts are supported but not well tested yet

Great work! This is an awesome feature addition 🎉

Is there any example of how to use the relay just to connect two ion-sfu?

Is there any example of how to use the relay just to connect two ion-sfu?

A simple document is here
https://github.com/pion/ion-sfu/tree/master/pkg/relay


core/models/entity.ts has a react dependence

The isReactResponse() implementation depends on a static import of a react function. This dependence means that even headless builds need to ship react!

A fix has been merged into the dev branch. It will be incorporated to master on the next stability swap. https://github.com/starpit/kui/pull/16


[QUESTION] Is there a way get a random video using api.challenge?

Ask your question
I would like to randomly sample videos from a bunch of predefined challenges. Currently when running api.challenge(challenge_name=hashtag, video_limit=1), it will always return the same video back.
E.g. with challenge_name = 'fyp', I always get this video.
How does the API currently choose which videos to grab from a challenge?

Closing this issue because there is no way to do this. TikTok's APIs always serve videos under a hashtag in an order determined by an algorithm.
Your best alternative would be to collect a large amount of videos with api.challenge and then pick a random one.


Initial implementation of jax.numpy.poly

This is an initial jax.numpy.poly implementation intended to help with https://github.com/google/jax/issues/70.
This is my first attempt at contributing, so I apologize in advance for any novice questions or any mistakes made.
There are some differences with the np.poly implementation (I've attempted to describe them in my commit messages), so I anticipate I will need to make changes.
Please let me know if there are any changes I can make to improve this PR.

@googlebot I fixed it.

It looks like some tolerances need to be adjusted.
Additionally, we need to skip the rank=2 tests on GPU and TPU, becuase non-symmetric eigenvalue decomposition is only supported on CPU:
"/build/work/737bb96d2bd1cba8ca8fa60e671f42d64a4c/google3/runfiles/google3/third_party/py/jax/_src/lax/linalg.py", line 381, in eig_translation_rule
    "Nonsymmetric eigendecomposition is only implemented on the CPU backend")
NotImplementedError: Nonsymmetric eigendecomposition is only implemented on the CPU backend


Demo animations in README

It would be nice to have a picture or even smaller audio-less demos in the README. Latter could be recorded with http://asciinema.org/

+1 It makes a big difference, see, for example, pipenv.

So here's on I made today quickly:
https://asciinema.org/a/c4oy6bm9axb8qyonparjmh1w2

Honestly, I'll completely open to criticism and a new script for it, etc. I'm not very creative with making up actions.

A few things:

--due (particularly its human datetime parsing) should be shown off
A typo in your task description: "by" instead of "be".

otherwise LGTM
On Sun, Feb 12, 2017 at 05:02:55AM -0800, Hugo Osvaldo Barrera wrote:

So here's on I made today quickly:
https://asciinema.org/a/c4oy6bm9axb8qyonparjmh1w2

Honestly, I'll completely open to criticism and a new script for it, etc. I'm not very creative with making up actions.
--
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub:
https://github.com/pimutils/todoman/issues/81#issuecomment-279217209

https://asciinema.org/a/5gc9baoqtx6wkgi1ytv90ncrs and https://asciinema.org/a/db3xuhufirhq0g4b22jwmmjeg

Uh oh, the latter actually revealed a bug in todoman. -dtomorrow doesn't work properly outside of UTC

Yeah, I also uncovered another bug (when not passing any summary) when doing this demo.
We need to figure out how to test the urwid bit.

Turns out the default due date got set after setting the user-defined due date 💩 Fixed in masterr

Here's another screencast: https://asciinema.org/a/7ifwbqub2f9e3ntgzk0iszf8g

Back on-topic:

asciinema records your terminal size, so something more convervative (25*80) looks better. Especially if it goes into the README. I use i3wm, and just make the terminal window float for it to take its default size.
I like showing off the config and find, but I'd move that to the end, and start off showing the app itself first.
I mostly liked this one: https://asciinema.org/a/5gc9baoqtx6wkgi1ytv90ncrs
I'd prefer "real-looking" todos. Rather than "hello world", make up something that sounds like an actual task.

Also of importance: I don't see how to download a video/gif from asciinema.

I think we can edit the JSON recording just for resizing the terminal, but I'm not sure.
I suppose we have to record this in some other way because of the GIF though. A real video with sound would be nice

Sound?

Something like the video at http://beets.io/

The voiceover is actually quite helpful, and showing the config makes for sense with it.

ttyrec and ttygif produce really good results. :+1:

Script idea:

Start of with two tasks, with and without due dates, and lists with colors.
todo new SOMETHING
todo new -l work SOMETHING
todo
todo edit 3, edit due date
todo
todo done 2
cat config
tree paths
cat ics file

More details:

Make sure we use the latest version of todoman (unlike my previous recording).
80x25 terminal.
Tasks are better if they sound real ("purchase ticket", "organize party", etc).
I prefer date_format = %Y-%m-%d %H:%M for the demo.

Maybe this issue is without progress for 7 years because the bar is too high. It doesn't take a well-scripted movie. Even a few screenshots may help for a start. Just my two cents.


What is the equivalent of poll or dequeue method

What to do if one need to consume the queue also. Trying to do pub-sub using this.
I know we can easily write one using queue.remove. Still I believe it would be quick task but add meaning to lib.

Exception in thread "Thread-3" Exception in thread "Thread-4" com.squareup.tape.FileException: Failed to peek.
at com.squareup.tape.FileObjectQueue.peek(FileObjectQueue.java:59)
at com.example.FileQueueConsumer.run(FileQueueConsumer.java:19)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.StreamCorruptedException: invalid type code: AC
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1381)
at java.io.ObjectInputStream.readUnshared(ObjectInputStream.java:463)
at com.squareup.tape.SerializedConverter.deserialize(SerializedConverter.java:27)
at com.squareup.tape.SerializedConverter.from(SerializedConverter.java:37)
at com.squareup.tape.SerializedConverter.from(SerializedConverter.java:20)
at com.squareup.tape.FileObjectQueue.peek(FileObjectQueue.java:57)
... 3 more
Here is what I did?
Create producer1, producer2,consumer1,consumer2 all on same FileObjectQueue. I believe there is some issue in multithreading handling.

FileObjectQueue is not thread safe.

Is there a threadsafe file based queue in the current api?

No
On Sat, Aug 6, 2016, 9:51 AM Rohit Verma<EMAIL_ADDRESS>wrote:

Is there a threadsafe file based queue in the current api?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/square/tape/issues/87#issuecomment-238024220, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAEEEQuG8KCB5YO4S626x0UUjG5YRgbfks5qdJFJgaJpZM4JeJwv
.


With latest RGB updates, existing RGB dimmer stops working

Version
Build/Run method

[ ] Docker
[ ] PKG
[X] Manually built (git clone - npm install - npm run build )

zwavejs2mqtt version: 1.0.0-rc.0
Describe the bug
The "master dimmer" of an existing RGBW dimmer (Zooz zen31) stops working with the latest update, the entity must be removed and re-added to HA
To Reproduce
Steps to reproduce the behavior:

Get set up with the 1.0.0-beta zwavejs2mqtt
Ensure an RGBW dimmer has been discovered. It will appear in HA as 5 dimmers: master=1, red=2, green=3, blue=4, white=5
Update to 1.0.0-rc.0
Attempt to control the dimmer. Master dimmer control will have stopped working completely

Expected behavior
I would expect that the dimmer will continue to function at worst, and will be updated to support RGB controls at best
Additional context
From the HA log:
2021-01-20 01:06:10 ERROR (MainThread) [homeassistant.components.light] Platform mqtt does not generate unique IDs. ID zwavejs2mqtt_0xcdbd925c_25-38-1-currentValue already exists - ignoring light.familyroom_tvlight_rgb_dimmer
node_25_beta.zip
node_25_rc.zip

This issue is resolved by #290 ! Thanks!

This issue is resolved by #290 ! Thanks!


🐞 [Bug]: Mnemonic field content is replaced with wrong data after providing a pre-existing Email in case the browser autofills

Is there an existing issue for this?

[X] I have searched the existing issues

which package/s did you face the problem with?
Dashboard
What happened?
while creating a new twin and pressing on Create Account button it generates a new Mnemonic, then after providing a pre-existing Email, the Mnemonic field content is replaced with my previous account's Password.
Steps To Reproduce

Navigate to The Dashboard.
Click on Connect Your Wallet.
Empty the autofilled mnemonics field
Click on Create Account.
Accept terms and conditions
Insert a Pre-exist Email.

which network/s did you face the problem on?
Main
version
2.5.2
Twin ID/s
No response
Node ID/s
No response
Farm ID/s
No response
Contract ID/s
No response
Relevant screenshots/screen records
https://github.com/user-attachments/assets/17f1f72d-d400-4aca-b431-95fff0a4c034
Relevant log output
n/a

could not produce the same issue will also check on different browser

still not producible will check with atef

Verified on devnet
Version: ac03039
The Issue has been fixed and is not producible anymore.

https://github.com/user-attachments/assets/3e325023-aac4-43f9-956a-bf0394dbb577
Related Issue:

https://github.com/threefoldtech/tfgrid-sdk-ts/issues/2630


Deployment target incorrect in rust-std assets on apple platforms

After this PR, the rust-std assets built in CI (nightly/beta/stable/etc) have an incorrect deployment target on darwin. rust-std was relying on. rusts' default targets when IPHONEOS_DEPLOYMENT_TARGET was not set and the CI job does not include that env var.
Meaning that when rust-std is built with the c feature (it is for all assets), the rust-std rlib takes the maximum minos which thanks to libcompile-rt is now the host sdk-version.
Related issue https://github.com/rust-lang/compiler-builtins/issues/650#issuecomment-2267277172
I proposed two solutions in the previous issue, let me know if you think of a third.
cc @BlackHoleFox

cc @BlackHoleFox

From your options I think its better if Rust's CI sets the environment variable to control it. imo compiler-builtins should use the same variables as std, or maybe it would be enough if Rust's CI set IPHONEOS_DEPLOYMENT_TARGET next to the macOS one (seems like a bug that its not there as well).
I do sympathize and apologize for the upstream bug :( I don't think this is for cc to fix, see below for a longer explanation why. I'll ping @NobodyXu in case though.
Long-winded tangent time, though: We have significantly struggled to find the "right default" for everyone now that Rust's deployment target can be different then 7.0 from years ago. We nuked the rustc querying from cc because it broke too many people's build setups where they were relying on the XCode-ish behavior of your current SDK, when they actually should have been setting the variable (like Apple intends, but Rust projects historically haven't cared about this for above reasons). A perfect world would let us restore the rustc querying default (so Rust projects stick with Rust's defaults, instead of Apple's) and tell everyone to set their deployment targets like every XCode and SwiftPM project have you do. Alas though!

Long-winded tangent time, though: We have significantly struggled to find the "right default" for everyone now that Rust's deployment target can be different then 7.0 from years ago. We nuked the rustc querying from cc because it broke too many people's build setups where they were relying on the XCode-ish behavior of your current SDK, when they actually should have been setting the variable (like Apple intends, but Rust projects historically haven't cared about this for above reasons). A perfect world would let us restore the rustc querying default (so Rust projects stick with Rust's defaults, instead of Apple's) and tell everyone to set their deployment targets like every XCode and SwiftPM project have you do. Alas though!

Actually, do you think it would be viable to revisit that now that the default deployment target is 10.12? From what I can tell from #900, it seems like it broke a lot of builds back then when set to macOS 10.7, which was lower than the 10.9 that was require to link certain C++ libraries, but did you encounter other problems?


Describe the bug
The value I passed in is an integer, and the final display is a decimal. This should be a matter of JavaScript's computational accuracy.
To Reproduce
Steps to reproduce the behavior:
<AnimatedCircularProgressBar
  max={100}
  min={0}
  value={14}
  gaugePrimaryColor="rgb(79 70 229)"
  gaugeSecondaryColor="rgba(0, 0, 0, 0.1)"
/>

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots

Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Do you know why it behaved like that? Cause our demo component didn't behave like this on the website.
And I see you have made a PR, I will look into it.

The issue is fixed now. Thanks for your contribution.


flutter_gallery__back_button_memory_diff-median regression

First recent jump at https://github.com/flutter/flutter/pull/49406. Previous jump at https://github.com/flutter/flutter/pull/49847 looks a little more likely to me.

Spoke to @jason-simmons who is investigating, and agrees that this is likely correct behavior and needs to be rebaselined if his investigations don't turn anything up. Dropping the TODAY label.

Took a look at recent changes but didn't find anything with a noticeable effect on memory usage.  This benchmark is noisy and is primarily intended to detect resources that are being leaked each time the app is suspended and resumed.  That doesn't appear to be happening - if I increase the number of suspend/resume cycles the process' memory consumption does not continue to increase.
I'll go ahead and rebaseline the benchmark.


Changed LoadKeyFromSSLibBytes to Support RSA Keys

Changed LoadKeyFromSSLibBytes to support RSA keys
Seperated LoadRSAPSSKeyFromFile into helper function LoadRSAPSSKeyFromBytes so that data can be passed as a byte array or as a file name.
Made LoadRSAPSSKeyFromBytes into an exported function to use in https://github.com/gittuf/gittuf/pull/174
Added test for RSA key to SSLibKey for LoadKeyFromSSLibbytes

Also, looks like go fmt is needed?


The scroll to the bottom shortcut dissapear when scrolling upwards too much...

Describe the bug
The scroll to the bottom shortcut dissapear when scrolling upwards too much and only reapears if you scroll downwards again, by a little. As on WA, it could be always visible. The current behaviour seems not 100% intuitive, at least my mom couldn't not figure it out without an explanation ;)
To Reproduce
Steps to reproduce the behavior:

scroll upwards by a lot;
the scrolling to the bottom shortcut will disapear;
it will appear again if you start scrolling down again;

It's a matter of personal preferences.


OmniSharp fails to initialize since VSCode updated itself (MacOS High Sierra)

From @attilam on February 1, 2018 10:54
Environment data
dotnet --info output:
.NET Command Line Tools (2.1.3)

Product Information:
 Version:            2.1.3
 Commit SHA-1 hash:  a0ca411ca5

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.13
 OS Platform: Darwin
 RID:         osx.10.12-x64
 Base Path:   /usr/local/share/dotnet/sdk/2.1.3/

Microsoft .NET Core Shared Framework Host

  Version  : 2.0.4
  Build    : 7f262f453d8c8479b9af91d34c013b3aa05bc1ff

VS Code version: Version 1.19.3 (1.19.3) 7c4205b5c6e52a53b81c69d2b2dc8a627abaa0ba
C# Extension version: 1.13.1
Steps to reproduce
[edit] I use VSCode/C# with Unity projects usually, but this gives the same result:
dotnet new console
Start VSCode with generated project.
Expected  behavior
OmniSharp would run and provide intellisense, etc.
Actual behavior
OmniSharp cannot initialize, and no matter what I do (reinstall C# plugin, or even VSCode itself) it won't work anymore.
This is the OmniSharp Log output:
Starting OmniSharp server at 2018-2-1 11:37:39
    Target: /Users/attila/Work/dotnet

OmniSharp server started wth Mono
    Path: /Users/attila/.vscode/extensions/ms-vscode.csharp-1.13.1/.omnisharp/omnisharp/OmniSharp.exe
    PID: 66161

System.TypeInitializationException: The type initializer for 'OmniSharp.Utilities.Platform' threw an exception. ---> System.IO.FileNotFoundException: Could not find file "/etc/os-release"
  at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.Int32 bufferSize, System.Boolean anonymous, System.IO.FileOptions options) [0x0019e] in <4fdc5ed61a074cafb49fa42deb20d521>:0
  at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.Int32 bufferSize, System.IO.FileOptions options, System.String msgPath, System.Boolean bFromProxy, System.Boolean useLongPath, System.Boolean checkHost) [0x00000] in <4fdc5ed61a074cafb49fa42deb20d521>:0
  at (wrapper remoting-invoke-with-check) System.IO.FileStream:.ctor (string,System.IO.FileMode,System.IO.FileAccess,System.IO.FileShare,int,System.IO.FileOptions,string,bool,bool,bool)
  at System.IO.StreamReader..ctor (System.String path, System.Text.Encoding encoding, System.Boolean detectEncodingFromByteOrderMarks, System.Int32 bufferSize, System.Boolean checkHost) [0x00067] in <4fdc5ed61a074cafb49fa42deb20d521>:0
  at System.IO.StreamReader..ctor (System.String path, System.Text.Encoding encoding, System.Boolean detectEncodingFromByteOrderMarks, System.Int32 bufferSize) [0x00000] in <4fdc5ed61a074cafb49fa42deb20d521>:0
  at System.IO.StreamReader..ctor (System.String path, System.Boolean detectEncodingFromByteOrderMarks) [0x0000d] in <4fdc5ed61a074cafb49fa42deb20d521>:0
  at System.IO.StreamReader..ctor (System.String path) [0x00000] in <4fdc5ed61a074cafb49fa42deb20d521>:0
  at (wrapper remoting-invoke-with-check) System.IO.StreamReader:.ctor (string)
  at System.IO.File.OpenText (System.String path) [0x00000] in <4fdc5ed61a074cafb49fa42deb20d521>:0
  at System.IO.File.ReadAllLines (System.String path) [0x00000] in <4fdc5ed61a074cafb49fa42deb20d521>:0
  at OmniSharp.Utilities.Platform.ReadDistroNameAndVersion (System.String& distroName, System.Version& version) [0x00000] in <dcfacdb97fa8408994f131ba70fae790>:0
  at OmniSharp.Utilities.Platform.GetCurrentPlatform () [0x000f2] in <dcfacdb97fa8408994f131ba70fae790>:0
  at OmniSharp.Utilities.Platform..cctor () [0x00000] in <dcfacdb97fa8408994f131ba70fae790>:0
   --- End of inner exception stack trace ---
  at OmniSharp.Stdio.Host..ctor (System.IO.TextReader input, OmniSharp.Stdio.Services.ISharedTextWriter writer, OmniSharp.IOmniSharpEnvironment environment, Microsoft.Extensions.Configuration.IConfiguration configuration, System.IServiceProvider serviceProvider, OmniSharp.CompositionHostBuilder compositionHostBuilder, Microsoft.Extensions.Logging.ILoggerFactory loggerFactory, System.Threading.CancellationTokenSource cancellationTokenSource) [0x0005f] in <8a68ba1f26d14eaf8aead202171d6d0a>:0
  at OmniSharp.Stdio.Program+<>c__DisplayClass0_0.<Main>b__1 () [0x00087] in <8a68ba1f26d14eaf8aead202171d6d0a>:0
  at OmniSharp.CommandLineApplication+<>c__DisplayClass11_0.<OnExecute>b__0 () [0x0000b] in <aec3dd5f118c413cbb04469a3429b570>:0
  at Microsoft.Extensions.CommandLineUtils.CommandLineApplication.Execute (System.String[] args) [0x0035b] in <e56ebbc3ed87488b8e26736bbadaa5d3>:0
  at OmniSharp.CommandLineApplication.Execute (System.Collections.Generic.IEnumerable`1[T] args) [0x00042] in <aec3dd5f118c413cbb04469a3429b570>:0
  at OmniSharp.Stdio.Program+<>c__DisplayClass0_1.<Main>b__0 () [0x00028] in <8a68ba1f26d14eaf8aead202171d6d0a>:0
  at OmniSharp.HostHelpers.Start (System.Func`1[TResult] action) [0x0001c] in <aec3dd5f118c413cbb04469a3429b570>:0

[ERROR] Error: OmniSharp server load timed out. Use the 'omnisharp.projectLoadTimeout' setting to override the default delay (one minute).

Copied from original issue: OmniSharp/omnisharp-vscode#1989

From @attilam on February 1, 2018 15:27
It does seem like some platform sniffing issue, here's the output of uname:
Darwin x86_64
Also, this is what mono --version gives me in terminal, if that helps:
Mono JIT compiler version <IP_ADDRESS> (2017-06/1f4613aa1ac Wed Oct 18 09:31:57 EDT 2017)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
	TLS:           normal
	SIGSEGV:       altstack
	Notification:  kqueue
	Architecture:  amd64
	Disabled:      none
	Misc:          softdebug
	LLVM:          yes(3.6.0svn-mono-master/8b1520c8aae)
	GC:            sgen (concurrent by default)

This issue looks very similar to a problem described in the discussion in #1952. Could you try the following for me?

Open VS Code as you always do.
In the VS Code Terminal pane, run the following:

/bin/bash -c 'uname -s -m'

What is the output?

From @attilam on February 1, 2018 15:48
Similarly to #1952 I've also set a custom value for omnisharp.path in User Settings, and now the extension works again:
"omnisharp.path": "/Users/attila/.vscode/extensions/ms-vscode.csharp-1.13.1/.omnisharp/run"
I guess this is OK as a temporary solution, but the problem will pop up after upgrading again.

Agreed. I'm going to move this over to omnisharp-roslyn, because that's where a fix will need to go.

This seems to be happening in omnisharp-vscode when OmniSharp.exe is run on a globally-installed Mono.

It doesn't repro anymore with the latest mono (we raised the minimum global mono to 6.4.0) so I will close this issue.


Total lag is not showing in consumer group page

Issue submitter TODO list

[X] I've looked up my issue in FAQ
[X] I've searched for an already existing issues here
[X] I've tried running master-labeled docker image and the issue still persists there
[X] I'm running a supported version of the application which is listed here

Describe the bug (actual behavior)
Total lag is not showing in consumer group page
Expected behavior
It should show the total lag of all topics even when the total is 0.
Your installation details
Docke compose
Steps to reproduce
Run a consumer and check the lag
Screenshots
No response
Logs
No response
Additional context
No response

Fixed


Swipe Functions

How can i swipe a container to the left for update or to the right for delete? Is it possible in flet?

https://flet.dev/docs/controls/dismissible ?

Have you checked this yourself? How to add function to the left swipe and to the right swipe?

The function I need is similar to Slidable in Flutter.


Support executors with multiple dependencies

UnsortedGatherExecutor being the guinea pig here.
Gordo internal PR

http://jenkins.arangodb.biz:8080/job/arangodb-matrix-pr/8811/


add pre-commit

Add initial pre-commit configuration files. The pre-commit tool can be
installed using pip:
   pip3 install pre-commit

After that the hook can be activated using:
   pre-commit install

or just run on the modified files
   pre-commit run

ATM the following checks are active:
  - check-yaml
  - end-of-file-fixer
  - trailing-whitespace
  - yamlfmt
  - codespell

Thanks. I don't think the pyproject.toml file is necessary. I pushed an update that moves it to .codespellrc. I hope that's ok.

Thanks. :+1:
I just found out that regex_exclude setting of yamlfmt is somehow not what I want as it excludes the file if the regex matches. Removes all whitespaces (before comments, between key:   value pairs,...) making the yaml files look mostly ugly. :( Since the configuration options are very limited I think we won't use yamlfmt... Maybe yamllint to enable some checks + fix them by hand?

Maybe yamllint to enable some checks + fix them by hand?

Sounds reasonable. I guess we can fix the findings incrementally. The pre-commit thingy seems to support only looking at changed files.

Thanks! I'll try to add the check to the CI build in the next days.


add support for vscode-server

Each of vscode's remote development extensions will create and use a
vscode-server directory in a user's home folder each time a host is
connected to.
Unfortunately, the SSH extension is the only extension that has a
setting that can change the location of vscode-server. Both the
container extension and the WSL extension are missing the option of
changing their vscode-server directories.
Even so, the SSH extension only allows you to define paths for
individual hosts. It doesn't allow for a default path for all ssh
connections.
To edit the host paths, you either need to change your remote SSH
settings in the vscode application itself, or edit your vscode user
settings json file.

Thank you!


Compiler errors due to old 0.9 version of openssl crate.

My system has OpenSSL 1.1.1, and openssl crate 0.9 doesn't work with it. Check out the related issue here:
https://github.com/sfackler/rust-openssl/issues/987
That issue has been linked to several other issues like this one for crates with the same issue as frank_jwt.
Thanks!

While openssl-rust crate dependency has been bumped to 0.10 in b5dc47ed608448c7c6a02f97e7f2140548220336, crates.io index wasn't updated, so cargo thinks there were no updates to this crate. Perhaps a new release is in order?

Just submitted a PR to resolve this. #46

I suspect it's not reflected the change because its a < 1.0 release, therefore Cargo can not assume it's BC safe.
For those looking for a work around, you can use @koala-kalda's branch temporarily by changing you Cargo.lock to use his commit:
frank_jwt = { git = "https://github.com/koala-kalda/frank_jwt", rev = "af4f3480fa26f628519712df719470feb7654896" }

I was blocked, too, with my Rocket project but the workaround helped.
Looking forward to seeing #46 merged, thanks! :+1:

openssl = "0.10.15"


docs: add definition and minimum criteria for samples

Adding guidelines for writing samples which includes definition
and minimum criteria to quality as a sample.
Signed-off-by: Aastha Grover<EMAIL_ADDRESS>

@nashif I have updated this PR with the requested changes. Please re-review and let me know any further feedback. Thanks!!

@microbuilder Thanks for reviewing this PR thoroughly. I have submitted few changes and left comments where needed :) Please re-review and let me know if any further improvisation is required.

We should see about referring to a specific sample in the tree that shows best practices as documented here that people can use to copy from as a starting point.

@galak Have added a sample towards the end to use it as a reference.

@aasthagr I think the overall changes in this PR make for a nice addition to the documentation. Do you think you can address the issues raised in time to get them into v3.2?

@aasthagr I think the overall changes in this PR make for a nice addition to the documentation. Do you think you can address the issues raised in time to get them into v3.2?

@henrikbrixandersen Missed the 3.2 RC1, maybe it can still make it to 3.2 as RC2 is 09/16. I have updated this PR as per the requested changes. You may re-review again now. Thanks!!

@aasthagr I think the overall changes in this PR make for a nice addition to the documentation. Do you think you can address the issues raised in time to get them into v3.2?

@henrikbrixandersen Missed the 3.2 RC1, maybe it can still make it to 3.2 as RC2 is 09/16. I have updated this PR as per the requested changes. You may re-review again now. Thanks!!

This is a documentation change, so I believe it can still be merged in for 3.2


Cover image loading options

BTW, could you consider adding more options for cover image detection ?

For example, maybe an array of "cover soure" like..
Cover Source:

✅ Url
✅ First Image in Page
❌ Property
✅ Preview Text (Like the Feed View - showing some text rather than empty background )

Query from top to bottom, displaying the first valid image.
If users doesn't want to show any image they can just un-check all of the options.  ( but why? 🤔)
And if it can be re-ordered, that would be even better!

Originally posted by @Moyf in https://github.com/decaf-dev/obsidian-vault-explorer/issues/192#issuecomment-2251954136

This could be part of the "fallback" mechanism?

@moyf
Here is a list of URL's that I think a user could load
//External embed with empty alt text
![](https://example.com/file.jpg)

//External embed with alt text
![cover](https://example.com/file.jpg)

//External embed with no file extension. This is common for some image
//hosting sites
![](https://example.com/file?width=200)

//Internal embed
![[file.jpg]]

//Internal embed with parameter
![[file.jpg|200]]

//Wiki Link
[[file.jpg|200]]

//Wiki Link
[[file.jpg|200]]

//Plain url
https://example.com/file.jpg

//Plain url without an image extension
https://example.com/file?width=200

The order that you presented is good, but I think my use case is a little more complex.
- Frontmatter - this is in sequential order
	- Specified property
	- Wiki Link
	- Internal Embed
	- External Embed
	- Plain URL
- Body - this is in sequential order
	- Internal Embed
	- External Embed

Do you think users would want to load a plain url from the body for a cover image?

This could be part of the "fallback" mechanism?

I believe that the selection should be made from top to bottom while respecting the user's preferred order.
For instance, if I configure the settings as "Image Property > URL > First Image in Note" and leave both the image and url fields empty, the system should display the first image.
However, if I fill in the url, the system should display the social media image from that URL, and so on.
Currently, the plugin provides the user with two specific property names: image and url, which I find beneficial:

I believe that these two specific properties are sufficient for my needs. 😊

Do you think users would want to load a plain url from the body for a cover image?

I personally think it's possible, but not necessary.
In my opinion, a certain degree of limitation is necessary, considering that loading all the social media images from links might add extra overhead in terms of performance or storage space...?
But you can always provide this option along with a corresponding checkbox, allowing users who need it to enable it (which may also depend on the development effort required for you to implement this).

It's kind of hard for me to imagine when I would put a plain text image url (http://xxx.jpg) in my note 😂

@Moyf
Okay here's an updated order

Order

Frontmatter property

image property (configured in settings)
url property (configured in settings)

Frontmatter

Loads

Any Wikilink
Any internal embed
Anyexternal embed
Any url

Choose the first one that appears

Body - this is in sequential order

Loads

Any internal embed
Any external embed

Choose the first one that appears

Explanation

Look for an image property in the frontmatter, if configured in the settings
Look for a url property in the frontmatter, if configured in the settings
Look for some type of link in the frontmatter. Choose the link from the first property that appears
Look for some type of link in the body (content minus frontmatter). Choose first link that appears
Otherwise show nothing

So updating the dropdown image that you posted I would think

Image property
Url property
First link in frontmatter
First link in body

And then you could toggle each one on or off, and rearrange.
Does this seem sufficient?

A social media image can be loaded for an external embed or a URL if the link fails to find an image

@Moyf  By the way, where did you get the image of the dropdown? Is that an Obsidian plugin?

Implementing fallback logic would a little complex. I am thinking of the case if you have an order like

Image property
URL
First link in frontmatter
First link in body

Let's say I have an URL and a link in the body
If the URL fails to load, it should load the link the body.
I think for me personally, I wouldn't want that. It would be confusing as to where the link is loading from

@Moyf By the way, where did you get the image of the dropdown? Is that an Obsidian plugin?

Oh yes, it's another plugin called Comonents - which is not available in the community plugin store yet.
You can find the repo in https://github.com/obsidian-components/obsidian-components-release , where you can download and install the plugin.
However, please note that this is a paid, closed-source plugin, and unfortunately, the author has not provided an English introduction or information on purchasing options outside of China. Here is the Chinese documentation
Additionally, there is a public beta version available for use without payment: https://github.com/obsidian-components/obsidian-components-release/releases/tag/20240423.

sufficient

YEEES, completely satisfied! Thank you very much for your effort!

Implementing fallback logic would a little complex. I am thinking of the case if you have an order like

Image property
URL
First link in frontmatter
First link in body

Let's say I have an URL and a link in the body
If the URL fails to load, it should load the link the body.
I think for me personally, I wouldn't want that. It would be confusing as to where the link is loading from

Yay, I share the same opinion.
Personally, I believe that the first image in the text should consist of local images only; I would not expect to display images from "normal URLs" 🤔
The url property in the frontmatter provides more control, allowing me to easily associate the image with its source. However, using links—especially when I have numerous reference links in a note—might not be as intuitive.

@Moyf Added in 1.37.0. Enjoy :)

@Moyf Added in 1.37.0. Enjoy :)

Hooray! You made my day, thank you decaf! 🥳


Fixed module registrations being ignored when that module type is resolved

Whenever Nancy would resolve module types it wouldn't check if the type was already registered in the IOC container or not and would just re-register it. This caused any module type registrations done by the user to be ignored.

Ping @despertargz could you provide some more background on this? Either going to pull it into the pending 1.3 release or close it. Thanks =)

Sure. The bootstrapper provides an IoC container which allows the user to register types. This gives the user complete flexibility over how and what instance gets returned when Nancy needs to resolve the module type.
However, at https://github.com/NancyFx/Nancy/pull/1470/files#diff-54eef3b8fa44e8d7ab316b8da9943c07L127 , Nancy overwrites any registrations the user may have done with its own generic registration.
I think this takes control away from the user being able to specify the instance they want returned when a type is resolved. The IOC container also gives the user an illusion that they have this control.

Ok, now we are getting somewhere. If I understand you correctly, you are manually registering INancyModule instances in the container, which is then overridden by the default registration of modules. Correct? If so then my question to you is:
Why are you manually registering modules in the container, rather than relying on Nancy doing it for you?
So I am much more interested in the why than the how because we've stepped outside of the realm of what Nancy has been designed for. I need to understand the use case / scenario that you are attempting to resolve, so that I can form an opinion on whether or not this is something we should be supporting in the core.
Thanks

Also, as a side note, I want to point out that such a change is not as trivial as it might seem. We would have to update all Nancy.Bootstrapper.* (autofac, unity, structuremap and so on) to provide the same functionality so we ensure that we provide a consistent behavior independent of which bootstrapper that is being used. If that was the case then we would probably have to re-design the bootstrapper API a bit and perhaps provide an abstract method (or similar) to help surface the requirement of having the module registration support not overriding user-registered modules.

@despertargz any more feedback on this? Thanks

Closing this as it requires a bit more though (and response to feedback). Feel free to reopen (or maybe file an issue, so we can discuss it further) :smile:


Demo app error

Hi,
i've downloaded the demo app, but unfortunately it doesn't work

Hi @furiosFast, since you close the issue, is everything working out for you?