Use String.intern() instead of WeakHashMap

Replace occurrence of WeakHashMap for Strings, with String.intern(),
because it is:

less code,
similar performance in JDK8 and later,
less memory overhead,
bins are tunable by the user with JVM flags, and
as of Java 8, no longer is constrained to a fixed size PermGen space

This does not preclude the user from also configuring the G1GC string
deduplication feature, if they choose to do so for the whole JVM.

I looked to see if it made a difference if the intern() call was made upstream of the constructor. In this case I don't think that it does.

In the discussion thread about this, the Guava code was cited since they recommend against String.intern. I looked at the git history, and the documentation of that recommendation is only 4 months old, so I started looking around for any performance comparisons, and I couldn't find anything recent.
I took a gist from a stackoverflow post discussing String.intern performance, and ran it on JDK 11 on a MacBook Pro laptop. While it's probably not the best benchmark, I found that even in JDK 11, String.intern can be significantly slower than a ConcurrentHashMap. It was mostly bad when I tried with a large number of interned strings. For a small number of interned strings, String.intern performance was comparable to ConcurrentHashMap, though lookup (calling intern on an already interned string) was always slower for String.intern. For large numbers of Strings, I was able to get improvements by setting the -XX:StringTableSize argument to the VM to increase the table from its JDK11 default of 65536 (JDK 7-10 defaulted to 60013).
I think if you aren't exceeding the string table size, then String.intern performance is probably comparable enough. But if you exceed, then it starts to degrade quickly. I'm not suggesting to skip this change, but it's just something to think about.

In the discussion thread about this, the Guava code was cited since they recommend against String.intern. I looked at the git history, and the documentation of that recommendation is only 4 months old, so I started looking around for any performance comparisons, and I couldn't find anything recent.

The comment was added in 2020, yes, but based on the git log, it was added as a clarification that it could also be used for String in addition to other immutable types. The note about performance was added based on a comment from 2010, when this functionality was added to Guava back in v3.0. It does not appear to be based on newer performance information, just a note about the original intent.
The more recent performance information (since 2010) I was looking at were at http://java-performance.info/string-intern-in-java-6-7-8/ (and has the same observations as yours below).

I took a gist from a stackoverflow post discussing String.intern performance, and ran it on JDK 11 on a MacBook Pro laptop. While it's probably not the best benchmark, I found that even in JDK 11, String.intern can be significantly slower than a ConcurrentHashMap. It was mostly bad when I tried with a large number of interned strings. For a small number of interned strings, String.intern performance was comparable to ConcurrentHashMap, though lookup (calling intern on an already interned string) was always slower for String.intern. For large numbers of Strings, I was able to get improvements by setting the -XX:StringTableSize argument to the VM to increase the table from its JDK11 default of 65536 (JDK 7-10 defaulted to 60013).

I'm curious how small/large your tests were, and which were more reasonable for Accumulo.

I think if you aren't exceeding the string table size, then String.intern performance is probably comparable enough. But if you exceed, then it starts to degrade quickly. I'm not suggesting to skip this change, but it's just something to think about.

Right. That's the idea. The default value of 65536 for JDK11 seems reasonable to me for our usage. If it needs to be higher, based on the user's operating environment, then they can tune it accordingly.

I did a similar comparison. Here's the results for -XX:StringTableSize=1000003 and 1,000,000 strings:

Implementation
load (ms)
lookup same (ms)
lookup equal (ms)

String.intern()
553
414
400

Our WeakHashMap
633
216
254

ConcurrentHashMap
356
109
101

Based on this, it looks like the ConcurrentHashMap is definitely much faster. However, it doesn't garbage collect. Comparing with our WeakHashMap implementation, the differences with String.intern are much smaller.
Here's a more realistic comparison for our use case with the default StringTableSize for Java 11 of 65536, and 100,000 strings:

Implementation
load (ms)
lookup same (ms)
lookup equal (ms)

String.intern()
88
66
46

Our WeakHashMap
115
58
41

ConcurrentHashMap
104
35
17

Also, after running many times, I also observed String.intern() to have much more consistent performance (about +/- 5%). The two HashMap implementations seemed to have a lot more variance (about +/- 20%), but I didn't compute it precisely. This is only an approximation.

While the ConcurrentHashMap might give the best performance (except for initial load in the realistic case), the WeakHashMap is a better comparison, since that's what we use today. For the realistic case (100,000 strings), the interning is better than our WeakHashMap for loads (probably because of early rehashing as it resizes), and not substantially worse for lookups. Even under the extreme case (1,000,000 strings) interning is only moderately worse than the WeakHashMap after tuning.
Overall, I'm still leaning towards preferring String.intern().

I see this is used for deduping locations and sessions.  The locations and sessions should be relatively low cardinality at a given point in time.  However over longer periods the cardinality could be higher (tservers restarts create a new session and possibly could use different port) for all seen.  But the active set would always be much smaller.  I assume that String.intern() handles strings that are no longer referenced well, but was not quite sure.  Curious if anyone who has done research knows?  Also curious if anyone knows anything about the behavior of String.intern() under concurrent load (like 100 threads all calling String.intern())?    I suspect String.intern() could only be better or the same as the current weak hashmap w.r.t to concurrency.

I assume that String.intern() handles strings that are no longer referenced well, but was not quite sure. Curious if anyone who has done research knows?

The exact behavior is implementation-specific, as it is a native method. However, everything I've found says that yes, manually-interned strings are eligible for garbage collection and removal from the string pool once they are no longer referenced in the program. For string literals, they are eligible for garbage collection once their containing class is unloaded. However, it's hard to find exact documentation on this.

Also curious if anyone knows anything about the behavior of String.intern() under concurrent load (like 100 threads all calling String.intern())? I suspect String.intern() could only be better or the same as the current weak hashmap w.r.t to concurrency.

I have not done any experimentation with concurrent access performance, and I've only found one broken link to a blog discussing it. I know it's thread-safe, but I don't know about performance. I might play around with that a bit to investigate it further.

Realized this change goes from a deduplication mechanism that is localized to a class to one that is JVM wide.  So if a user loads two libraries (Accumulo client and some other lib) that both use String.intern(),  if the non-accumulo library uses String.intern() for a much higher cardinality set of Strings than Accumlo does, then it could possibly impact Accumulo.

Here's some representative numbers for the 1,000,000 strings case with 1000 concurrent threads (and -XX:StringTableSize=1000003):

Implementation
load (ms)
lookup same (ms)
lookup equal (ms)

String.intern()
1902
1274
1166

WeakHashMap
1491
907
743

ConcurrentHashMap
1196
648
387

And here's the more realistic case (100,000 strings, 100 threads, default string table size):

Implementation
load (ms)
lookup same (ms)
lookup equal (ms)

String.intern()
271
150
94

WeakHashMap
256
131
79

ConcurrentHashMap
291
92
92

It looks like intern has slightly more overhead that adds up over time, but for the realistic case, they are all very similar.

Okay, so I'm going to scrap this PR, and just create a simple Interning utility that puts this deduplication code in one place. This is the only time we use it for String types, but we do use it for a few other types elsewhere. Guava's built-in Interner is beta, so I won't use that, but will just move our existing code into a separate util class and add generics, so it can be re-used if we need it.

This is superseded by #1925


feature: --volumes-from as in docker build

This would really improve builds times when we need to use some cache from other image. Not sure if this is valid.

this isn't a feature in docker so can you give more details please.

RUN --mount cache should work. (probably after revendoring bk)

@jessfraz yeh you are right, I got messed up with the --volumes-from command of docker run. (got this idea because kitchen-dokken supported that :D). I will gladly close this RFC.
@AkihiroSuda  can you look at https://github.com/genuinetools/img/issues/136, that's exactly what I did.


SizeT < Int and OffsetT < Int types

I suppose that we must introduce low level platform specific types SizeT and OffsetT based on Int, and update Intrinsics, LibC and other contracts too.
Quote @funny-falcon #4589:

BTW, why Crystal has neither size_t nor intptr_t equivalent? Some places uses UInt64, some uses UInt32 (like this one). It is just... infantile.

For example, Go defines int to be ssize_t and uint to be size_t, and has intptr and uintptr (though, looks like on all supported platforms they have same size to int).
Having platform dependent int has profit in low-level programming, and dis-convenient for application level programming, so it certainly should not be default type.
But still any place that deals with low-level memory handling should operate on platform dependent types/aliases instead of raw UInt32 and/or UInt64. This bogus lines are just tip of iceberg.

#4624 introduces LibC::PtrdiffT, LibC::SizeT already exists.


🐛 Bug Report: Microsoft Auth CORS

📜 Description
Using Microsoft authentication provider for Backstage in deployment running in Kubernetes behind an ingress controller results in a 504 timeout. Further investigation shows the error is a 401 with "Invalid X-Requested-With header" from Backstage.
Followed instructions defined in this document.
👍 Expected behavior
User is authenticated by Azure AD
👎 Actual Behavior with Screenshots
Authentication flow results in this message from the web UI -

Message in Backstage logs -
"GET /api/auth/microsoft/refresh?optional&env=development HTTP/1.1\" 401 - \"-\" \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36\"","service":"backstage","type":"incomingRequest"}

API call from Postman
{
    "error": {
        "name": "Error",
        "message": "Invalid X-Requested-With header"
    },
    "request": {
        "method": "GET",
        "url": "/api/auth/microsoft/refresh?optional&env=development"
    },
    "response": {
        "statusCode": 401
    }
}

👟 Reproduction steps

Set up Microsoft authentication following this document
Deploy Backstage in Kubernetes behind an ingress controller with SSL termination
Attempt to authenticate with Microsoft provider

📃 Provide the context for the Bug.
Authenticating users via Azure AD to customize Backstage based on user roles.
🖥 Your Environment
Chrome Version 114.0.5735.134 (Official Build) (64-bit)
Kubernetes v1.23.17
Nginx Ingress Controller v3.1.1
yarn backstage-cli info
yarn run v1.22.19
$ /tmp/backstage-auth-test/node_modules/.bin/backstage-cli info
OS:   Linux 4.18.0-425.13.1.el8_7.x86_64 - linux/x64
node: v18.14.2
yarn: 1.22.19
cli:  0.22.6 (installed)
backstage:  1.13.0

Dependencies:
  @backstage/app-defaults                          1.3.0
  @backstage/backend-app-api                       0.4.2, 0.4.3
  @backstage/backend-common                        0.18.5, 0.18.4
  @backstage/backend-dev-utils                     0.1.1
  @backstage/backend-plugin-api                    0.5.1, 0.5.2
  @backstage/backend-tasks                         0.5.1, 0.5.2
  @backstage/catalog-client                        1.4.1
  @backstage/catalog-model                         1.3.0
  @backstage/cli-common                            0.1.12
  @backstage/cli-node                              0.1.0
  @backstage/cli                                   0.22.6
  @backstage/config-loader                         1.2.0, 1.3.0
  @backstage/config                                1.0.7
  @backstage/core-app-api                          1.7.0
  @backstage/core-components                       0.13.0
  @backstage/core-plugin-api                       1.5.1
  @backstage/errors                                1.1.5
  @backstage/eslint-plugin                         0.1.3
  @backstage/integration-aws-node                  0.1.2, 0.1.3
  @backstage/integration-react                     1.1.12
  @backstage/integration                           1.4.4, 1.4.5
  @backstage/plugin-api-docs                       0.9.2
  @backstage/plugin-app-backend                    0.3.44
  @backstage/plugin-auth-backend                   0.18.2
  @backstage/plugin-auth-node                      0.2.14, 0.2.13
  @backstage/plugin-catalog-backend                1.9.0
  @backstage/plugin-catalog-common                 1.0.13
  @backstage/plugin-catalog-graph                  0.2.29
  @backstage/plugin-catalog-import                 0.9.7
  @backstage/plugin-catalog-node                   1.3.5
  @backstage/plugin-catalog-react                  1.5.0
  @backstage/plugin-catalog                        1.10.0
  @backstage/plugin-github-actions                 0.5.17
  @backstage/plugin-kubernetes-common              0.6.2
  @backstage/plugin-kubernetes                     0.8.0
  @backstage/plugin-org                            0.6.7
  @backstage/plugin-permission-common              0.7.5
  @backstage/plugin-permission-node                0.7.8, 0.7.7
  @backstage/plugin-permission-react               0.4.12
  @backstage/plugin-proxy-backend                  0.2.38
  @backstage/plugin-scaffolder-backend             1.13.1
  @backstage/plugin-scaffolder-common              1.2.7
  @backstage/plugin-scaffolder-node                0.1.2
  @backstage/plugin-scaffolder-react               1.3.0
  @backstage/plugin-scaffolder                     1.13.0
  @backstage/plugin-search-backend-module-catalog  0.1.0
  @backstage/plugin-search-backend-module-pg       0.5.5
  @backstage/plugin-search-backend-module-techdocs 0.1.0
  @backstage/plugin-search-backend-node            1.2.0
  @backstage/plugin-search-backend                 1.3.0
  @backstage/plugin-search-common                  1.2.3
  @backstage/plugin-search-react                   1.5.2
  @backstage/plugin-search                         1.2.0
  @backstage/plugin-tech-radar                     0.6.3
  @backstage/plugin-techdocs-backend               1.6.1
  @backstage/plugin-techdocs-module-addons-contrib 1.0.12
  @backstage/plugin-techdocs-node                  1.7.0
  @backstage/plugin-techdocs-react                 1.1.5
  @backstage/plugin-techdocs                       1.6.1
  @backstage/plugin-user-settings                  0.7.2
  @backstage/release-manifests                     0.0.9
  @backstage/test-utils                            1.3.0
  @backstage/theme                                 0.2.19
  @backstage/types                                 1.0.2
  @backstage/version-bridge                        1.0.4
Done in 1.49s.

👀 Have you spent some time to check if this bug has been raised before?

[X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

[X] I have read the Code of Conduct

Are you willing to submit PR?
None

It's important that the X-Requested-With header is forwarded to the auth-backend, we use it for CSRF protection. The particular piece of code handling it is over here: https://github.com/backstage/backstage/blob/126e862ece05028ac0de4cff0b2fd9285131621a/plugins/auth-backend/src/lib/flow/authFlowHelpers.ts#L75
It's likely that you gateway does not forward the header


Choose which builds to watch

I'd like the ability to choose which builds to watch. I have about 40 builds on my hudson server, but would only like the plugin to see about 5 of them. Black list or white list works for me.

Just found out, that you can just enter the api url of a view to filter projects. Just enter:
http://hudson.yourdomain.com/view/NameOfView/api/json/
Cheers


Keys do not default if there is no settings.json present

The program fails to detect any keys at all and displays no input when there is no settings.json.
The program should use the default keys when there is no settings.json present, but it current does nothing.

Should be fixed in 4a10dd44806dc851ce9e0ef66a64293cf3711c49, but needs testing.


feat(rbac): add the ability to cache information

Description
As part of our ongoing work, this PR introduces the ability to cache information for the role manager to use. This should help further alleviate the problem of constantly having to rebuild the group hierarchy graph each time there is a request to the role manager.
Fixes

Fixes https://issues.redhat.com/browse/RHIDP-1377

Special note to the reviews
There are a lot of changes to the test files. The reason for this is because I refactored some of the tests to bring out common functionality to a utils file. Now, we have a utils.test.ts file that stores some of the common functions that are spread throughout some of our tests.
I didn't refactor all of the tests, this was more of a starting point. There still needs to be some ongoing effort to

Hello, @PatAKnight. I was pondering the necessity of caching roles. Considering that our API both provides and manages roles, and we have precise control over their application and removal, it seems we may not require an expiration mechanism to update this information. Additionally, roles are already stored in the map RoleList, as illustrated:
https://github.com/janus-idp/backstage-plugins/blob/98fbae66d28da70d7e63500562633e3d4cb02de2/plugins/rbac-backend/src/role-manager/role-manager.ts#L13
This map, though basic, serves as a form of cache itself. Could you kindly elaborate on the necessity for an additional cache such as RoleCache? Thank you for your insights.

Oh, sorry, I see. You are caching roles per user to avoid rebuild group graph each time.

Yep, the idea is to prevent us from rebuilding the graph every single time by caching the roles using RoleCache. The way that I was envisioning it is that the allRoles cache is all of the grouping policies that the RoleManager is aware of, whereas the roleCache are roles that are verified to not have any issues when building the group hierarchy graph. Hopefully, that makes sense.

Closing PR as it is outdated and we discussed that we are going to hold off on this feature for now.


get_label() does not work well for libsvm dataset

I constructed a dataset from libsvm text file:
dvalid = lgb.Dataset('yanan_gbdt_train_libsvm_valid_0713all.libsvm')
In [8]: label=dvalid.get_label()
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
<ipython-input-8-920266127d25> in <module>()
----> 1 label=dvalid.get_label()

/home/sdev/anaconda2/lib/python2.7/site-packages/lightgbm/basic.pyc in get_label(self)
   1161         """
   1162         if self.label is None:
-> 1163             self.label = self.get_field('label')
   1164         return self.label
   1165

/home/sdev/anaconda2/lib/python2.7/site-packages/lightgbm/basic.pyc in get_field(self, field_name)
   1000         """
   1001         if self.handle is None:
-> 1002             raise Exception("Cannot get %s before construct Dataset" % field_name)
   1003         tmp_out_len = ctypes.c_int()
   1004         out_type = ctypes.c_int()

Exception: Cannot get label before construct Dataset

So how can I get the label from the dataset which is constructed from libsvm file ?
Thanks.

See Question 2 here https://lightgbm.readthedocs.io/en/latest/FAQ.html#python-package


Update R script

An extra R script is required: doAzureParallel now requires the rAzureBatch package. If not, the following error occurs:
ERROR: dependency 'rAzureBatch' is not available for package 'doAzureParallel'

@jeroenwo : Thanks for your contribution to the Azure documentation! The author, @jiata, has been notified to review your proposed change.

See also the doAzureParallel documentation: https://github.com/Azure/doAzureParallel/blob/master/README.md#installation

@jeroenwo - Thanks for pointing out the issue, and apologies for any inconvenience. The engineering team is investigating to see if the problem installing the dependent package might be OS-specific. In the meantime, we'll add your script update. Thanks again!
(Pull request reviewers, I'm signing off on behalf of the PM here.)
#sign-off

@dlepow: Only the author, @jiata, can sign off on this pull request.


Finalise new release method

Linked with https://gitlab.internal.sanger.ac.uk/psd/psd-deployment/merge_requests/14

If this passes the tests it should be good to merge in!


Use an enum for the TMaster commands instead of a string

Plus random minor cleanup to TopologyUtils.java and ResourceCompliantRRPacking.java.

👍 - once CI passes.


Feature request: Calendar Vertical Indicator without text

Describe the feature you'd like
As a user, I'd like to have some particular indicator in the agenda with custom color and size. This could allow to displa availabilites, or other indication that do not contain text but could be clicked.
Visual demo
Here is a visual demo taken from amie.so app. The green bar are what am talking about.

That'd be amazing!

I think we will be able to achieve this with a custom EventComponent, stacking #233 (that also allows ignoring overlaps) and custom styles per event #228

That's great. We have to update all the docs accordingly, as there are more and more features not visible in the main README.md.


Notes: Mixtape 06

Title (as of 20210326): I Don't Like Love, Cause I Love You
Versions:
20191027 (New), 20191029 (Repeat), 20191124 (Archive but broken)
Other broadcasts: 20191231, 20200220, 20200312, 20200423, 20200604, 20200813, 20201112

This is from the era where mixtapes were 90 minutes.  I think from my notes I'm seeing an average of 91:30.
I'm pretty sure a majority of this mixtape features in Mixtape 10 (#4), so that cuts down on the amount of sourcing needed.
The archive lists an initial publish date of 20191105, and then an update date of the 24th of that month.  I vaguely remember looking at the archive edition at some point in the past, so maybe that 24th broke it, or maybe it's always been broken.  Either way this presents an interesting dilemma - how do you reconstruct a mixtape that hasn't been played in 4 months, isn't in an archive, and has since been "retired"?

The answer is you cheat, in short.
I had been making some real granpda tech moves by using Google OCR Translate from my phone while aimed at my desktop machine during the stream, and then posting on-phone screenshots of the (understandably wrong) "funny" translations later.  Several people helpfully suggested, if I was interested in more accurate translations, that I should screenshot at maximum resolution on the desktop machine, and then run that through the OCR/translation later.
So I had done that whenever a particularly Japanese-characters-on-screen-heavy mixtape was broadcast.  Because of how frequently this particular mixtape was aired, I had multiple opportunities, and eventually I just started taking screenshots at scene changes too.  This gets tedious while trying to also enjoy the broadcast, and I only needed to do it right once for each mixtape, so it happened sparingly, but luckily this particular mixtape has good coverage.  The timestamps on the screenshots help establish the order, so unless I've missed a particularly fast scene, I should also be able to get the sequencing correct.

Rather than start a new Wiki page and have it sit incomplete (like I did with Mixtape 10) I'm going to keep notes in this Issue.  I've found editing Issues on mobile to be error-prone, so I'll need to be careful with massive edits to the comments.
It makes more sense to get enough of a skeleton of Screenshots and Sequencing done in this Issue before converting to a full Wiki page.
Timing-wise, I don't think this needs to be "done" by the usual deadline, and I can go back and forth with other efforts and this one to gradually fill it out, if need be.

rel time
segment tag

00:00:00.000
EZ MOVIE

00:00:24.400
back coating

00:00:39.200
pc mk ii

00:01:09.166
handicam

00:01:24.100
5 component tape

00:01:38.966
seiko

00:01:54.067
julie alpha

00:02:25.067
OH full

00:02:40.534
HI-TOUCH

00:02:53.467
YOBODASHI

00:03:22.566
VIDEO RELEASING ORGANIZATION

00:04:02.900
CHANGEMAN OP

00:05:23.366
ZAR

00:06:17.200
boy zapped?

00:06:43.233
temptation

00:08:22.800
dairanger Kou

00:10:56.700
ninja empire

00:12:46.534
THE DREAM

00:13:23.833
gozmad changeman

00:15:06.633
breakdance ad

00:15:34.833
kmart ad

00:16:04.500
loughrains ad

00:16:33.933
old vcr

00:17:00.667
sharp

00:17:01.200
michael cera jr ad

00:17:23.533
matchbox santa line

00:17:36.366
leopard drums

00:18:07.700
BeCan lake ad

00:18:36.800
opel ad

00:44:10.833
Lethal Panther

00:45:55.334
Lethal Panther

00:50:11.933
Golden Ninja Warrior

00:52:42.667
Double Dare at Home

00:53:07.067
Self Defense with Master

00:54:39.500
Undefeatable

00:59:54.966
OH

00:59:56.400
The Warrior (Prima)

01:09:04.966
Fist of the North Star Closing

01:10:42.134
Ninja Powerforce

01:14:17.300
DOOT DOOT

01:14:29.400
ZAR Closing

01:15:33.900
Changeman

01:16:38.200
Ninja Force of Assassins

01:22:45.233
ad

01:24:20.766
Ninja Showdown

01:26:22.333
ad

01:27:11.000
Champion on Fire

01:30:16.134
Berl


Add support for defaultValue

When trying to use the defaultValue prop on a <textarea />, the textarea shows up blank. This works with React but not when using a webpack alias to preact-compact. Changing the prop to value instead of defaultValue is a workaround but the defaultValue prop should be supported. Reproducible on<EMAIL_ADDRESS>with<EMAIL_ADDRESS>

Sorry for the no-reply there! We'll have to patch support for that in for sure.

Ahh, I'm having the same problem with select fields right now...

@gholtslander-va defaultValue not being applied?

Yeah, in the case of a select box it just has the top option selected, as though the "value" was empty.

@gholtslander-va good to know, we can prioritize this. I added a fix above but it'd be best to fix it in preact-compat itself.

I have some time right now and was hoping to take a look. Cool.


[BUG:] When switching to new versions of LLamaSharp 0.16.0, there was a slowdown

Description
After upgrading from KM 0.61 and LLamaSharp 0.13 to the latest versions, the processing ImportDocumentAsync/AskAsync speed has significantly decreased, almost 2 times.
All settings and equipment are the same. I see that the GPU is now less loaded, previously CUDA was loaded more than 80%.
Import:

Ask:

How do I get back to my previous speed?
Thanks.
Reproduction Steps

Environment & Configuration
Operating system: Windows 11
.NET runtime version: 8.0.8
LLamaSharp version: 0.16.0
KernelMemory: 0.71...
CUDA version (if you are using cuda backend): 12
CPU & GPU device: CPU or GPU, Intel Core Ultra 9 & RTX 3090
Known Workarounds
No response

Launched Llama

apparently there was a similar problem
https://github.com/SciSharp/LLamaSharp/issues/601

GPU utilization is not more than 30%.
This my test code:
        ModelParams = new(Program.AppSettings.ModelPath)
        {
            Embeddings = false,
            ContextSize = 8192,
            Seed = 1337,
            MainGpu = 0,
            GpuLayerCount = 100,
            SplitMode = GPUSplitMode.Layer,
            BatchSize = 1024,
            UBatchSize = 1024
        };

        Weights = LLamaWeights.LoadFromFile(ModelParams);
        StatelessExecutor executor = new(Weights, ModelParams);

        InferenceParams inferenceParams = new()
        {
            SamplingPipeline = new DefaultSamplingPipeline()
            {
                Temperature = 0.5f,
                RepeatPenalty = 1.0f
            },
            MaxTokens = 2000,
            AntiPrompts = ["\n"]
        };

        StringBuilder sb = new(2048);

        await foreach (string text in executor.InferAsync(value, inferenceParams, cancellationToken))
        {
            sb.Append(text);
        }

        return sb.ToString();

What model are you using, how many tokens are you processing at once, and how are you measuring GPU utilisation?

Model [ggml-model-Q4_K_M.gguf]
https://huggingface.co/ruslandev/llama-3-8b-gpt-4o-ru1.0-gguf/blob/main/ggml-model-Q4_K_M.gguf
In the text above, the value line is: come up with and write a poem about summer.
Measuring GPU utilisation: the graphs are higher and the most important thing is to increase the response time by 2 times, create embeddings

This problem only manifests itself on the GPU

The default values in DefaultSamplingPipeline should be the same as the default sampling values in the llama.cpp main application. Typical P is documented as having a default value of 1.0. So it looks like that's a bug in the default values for DefaultSamplingPipeline..
Would you like to submit a PR changing that default value? If so, and if you want to go put in some extra effort, checking all of the default values against the readme I linked above would be a good idea :)

And why does this parameter have such an effect?

All the parameter is doing is enabling a new sampling stage in llama.cpp (it's disabled entirely when the value is 1.0), so presumably it's just that expensive to calculate typical-p.

These two parameters are too high (should decrease to 512 - this is the optimal value for speed and GPU memory use):
BatchSize = 2048,
UBatchSize = 2048

It is very easy for you to carefully read everything that I have written. I spent a lot of time on this.And there are already a lot of questions. The speed has increased, utilization is 60%, but still not 80% as it should be. I moved all the default settings as they should be. You should install them in the constructor for others. Bath size 512 I can't install because this size is not enough for me. And for some reason, it is checked for the equality of batchsize and ubatchsize.

I use KM. When creating embeddings, a value less than batchsize=2048 does not work, it throws an error. The problem is not even this, but this:
public LLamaEmbedder(LLamaWeights weights, IContextParams @params, ILogger? logger = null)
{
if (@params.UBatchSize != @params.BatchSize)
{
throw new ArgumentException("For non-causal models, batch size must be equal to ubatch size", "params");
}
}

I am using batch size 512 with KM with different models having different vector size and I have never had a problem. 512 is the optimal value for speed and GPU memory use. Your problem will be somewhere else!

I'll check it again. Are you using 512/512?

The issue of GPU utilization is still relevant.

I'll check it again. Are you using 512/512? ContextSize?

Input contains more tokens than configured batch size (Parameter 'batch')

@params.UBatchSize != @params.BatchSize
This is inherited from here.

Unfortunately the embedder can't split input into multiple batches at the moment, so if you are getting embeddings for huge chunks of text you will need an unusually large batch size. Again, this is inherited from that code linked above, which the new embedder is basically ported from.

The issue of GPU utilization is still relevant.

Just a note about this, I think the measurements you're showing might be a little misleading. The typical-p thing gives us a clue - that was adding extra work on the CPU side (sampling is not done on the GPU). So that doesn't lower the actual peak utilisation, but it does increase the amount of idle GPU time between inference (while sampling is running). Slightly different, it's just showing as lower utilisation because the sampling time in task manager is too low.

But I'm comparing Lama native and LLamaSharp graphics on the same machine, in the same configuration.

OK, but why does LLama native show more than 80%?

@params.UBatchSize != @params.BatchSize
This is inherited from here.

Unfortunately the embedder can't split input into multiple batches at the moment, so if you are getting embeddings for huge chunks of text you will need an unusually large batch size. Again, this is inherited from that code linked above, which the new embedder is basically ported from.

It does not have any sense to use huge chunks of text for embeddings. The optimal size of embeddings is between 100-500 words depending on the task (document retrieval is on the lower side and summarization is on the higher side). So, maximum around 500 words. This should fit into a normal batch size of 512. If you decrease these numbers your GPU utilization will increase.

I use a chunk size of 1000 words. Experimentally, this is the best size for me.
The GPU loading problem manifests itself with normal output, not embedding.

Embedding comes first, then a break, then GenerateText:

Embedding and text generation aren't really comparable workloads. Embedding processes one chunk of text all at once (it's similar to prompt processing). Wheras text generation has to constantly go back and forth betwen GPU (generating logits) and CPU (sampling logits to produce tokens). You'd expect GPU load to be lower in that case.

It was just additional information. You persistently ignore the important things that I write :)
I am comparing the utilization when working with Lama native and LLamaSharp when doing the same job.

I'm unlikely to find a reason, because I don't have that deep knowledge of LM and llama native. I thought that my information would help the experts understand where the performance problem might be. Can you compare GPU utilization on your PC with lama native & lama sharp?

I'm unlikely to be able to find time in September for anything more than issue triage and PR reviews unfortunately. I'll be happy to dig into perf stuff in October though (probably after doing a 0.17 update).

Thank you very much for your work! I am ready to participate in improving LLamaSharp as much as my knowledge allows.

We also need to bring everything to the use of SamplingPipeline:
StatelessExecutor.InferAsync()
...
var repeat_last_n = Math.Max(0, inferenceParams.RepeatLastTokensCount < 0 ? _weights.ContextSize : inferenceParams.RepeatLastTokensCount);
...
inferenceParams.RepeatLastTokensCount obsolete !!!

New Version 0.17.0:


Resource Prometheus can be created by Deployment?

What did you do?
I deployed a prometheus cluster by prometheus-operator, and the operator created prometheus
as statefulset.
The prometheus data
# kubectl get sts -n kube-system
NAME                DESIRED   CURRENT   AGE
alertmanager-main   1         1         5d
prometheus-k8s       1         1         5d

And then, I poweroff the master node (I have  3 master nodes) that prometheus pod exists.
What did you expect to see?
The Prometheus pod will run in other master nodes.
What did you see instead? Under which circumstances?
[root@master1 ~]# kubectl get po -n kube-system -o wide | grep prom
prometheus-k8s-0                                       3/3       Unknown             7          5d        <IP_ADDRESS>    master2

The pod can not run on other node.
Environment

Prometheus Operator version:
v0.24.0

Kubernetes version information:

Client Version: version.Info{Major:"1", Minor:"11+", GitVersion:"v1.11.0", GitCommit:"3d2c54994840f73bd2ac99e505a3cad2c70e8161", GitTreeState:"clean", BuildDate:"2018-08-08T01:40:36Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"11+", GitVersion:"v1.11.0", GitCommit:"3d2c54994840f73bd2ac99e505a3cad2c70e8161", GitTreeState:"clean", BuildDate:"2018-10-18T20:21:30Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"}

Does prometheus operator create the prometheus by deployment instead of statefulset?
And I don't want to delete the prometheus pod mannually.

@aixeshunter prometheus operator will use statefulsets as prometheus is a stateful backend. If you power down the node and don't drain it, the k8s scheduler won't reschedule it on a different node, also see https://kubernetes.io/docs/tasks/run-application/force-delete-stateful-set-pod/.

@s-urbaniak Thanks ! I have a question now: why prometheus operator not monitor prometheus pod status?
If operator can delete the prometheus pod with Unknown status automatically, is it better?

Monitoring the prometheus' pod status is in the responsibility domain of k8s, via the statefulset controller in the controller manager. The prometheus operator is supposed to only monitoring the statefulset status.

@s-urbaniak
I am sorry to ask you another question.
I found that alertmanager pod can be rescheduled in another node but prometheus not, and they were all controlled by statefulset.
[root@master1 prometheus-plugin]# kubectl get sts -n monitoring
NAME                DESIRED   CURRENT   AGE
alertmanager-main   1         1         46d
prometheus-k8s      1         1         46d

before the power down:
alertmanager-main-0                    2/2       Running    0          1d       <IP_ADDRESS>    node1.hikcloud
prometheus-k8s-0                       3/3       Running    0           1d       <IP_ADDRESS>    node1.hikcloud

power down the node1 and wait for 5m:
alertmanager-main-0                    2/2       Running    0          1m        <IP_ADDRESS>    master3.hikcloud
prometheus-k8s-0                       3/3       Unknown0           1d       <IP_ADDRESS>    node1.hikcloud

The Alertmanager resource yaml:
apiVersion: monitoring.coreos.com/v1
kind: Alertmanager
metadata:
  labels:
    alertmanager: main
  name: main
  namespace: monitoring
  resourceVersion: "11969572"
  selfLink: /apis/monitoring.coreos.com/v1/namespaces/monitoring/alertmanagers/main
  uid: d0ede83c-db34-11e8-966a-5254008abd3e
spec:
  baseImage: quay.io/prometheus/alertmanager
  externalUrl: http://<IP_ADDRESS>:9093
  replicas: 1
  version: v0.15.2

The Prometheus resource yaml:
apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  labels:
    prometheus: k8s
  name: k8s
  namespace: monitoring
  resourceVersion: "11998660"
  selfLink: /apis/monitoring.coreos.com/v1/namespaces/monitoring/prometheuses/k8s
  uid: f05c4968-db34-11e8-966a-5254008abd3e
spec:
  alerting:
    alertmanagers:
    - name: alertmanager-main
      namespace: monitoring
      port: web
  baseImage: quay.io/prometheus/prometheus
  replicas: 1
  retention: 168h
  ruleSelector:
    matchLabels:
      prometheus: k8s
      role: alert-rules
  serviceAccountName: prometheus-k8s
  serviceMonitorSelector:
    matchExpressions:
    - key: k8s-app
      operator: Exists
  storage:
    volumeClaimTemplate:
      spec:
        resources:
          requests:
            storage: 20Gi
        selector:
          matchLabels:
            app: prometheus-k8s
  version: v2.4.3

@aixeshunter there is no difference between the two, both are statefulsets. Please consider to iterate on the scheduling behavior of k8s with regard to statefulsets. The reason why alertmanager was rescheduled and prometheus can depend on the the specific situation in your cluster.
As per https://kubernetes.io/docs/tasks/run-application/force-delete-stateful-set-pod/#delete-pods:
Kubernetes (versions 1.5 or newer) will not delete Pods just because a Node is unreachable. The Pods running on an unreachable Node enter the ‘Terminating’ or ‘Unknown’ state after a timeout. Pods may also enter these states when the user attempts graceful deletion of a Pod on an unreachable Node. The only ways in which a Pod in such a state can be removed from the apiserver are as follows:

The Node object is deleted (either by you, or by the Node Controller).
The kubelet on the unresponsive Node starts responding, kills the Pod and removes the entry from the apiserver.
Force deletion of the Pod by the user.

The recommended best practice is to use the first or second approach. If a Node is confirmed to be dead (e.g. permanently disconnected from the network, powered down, etc), then delete the Node object. If the Node is suffering from a network partition, then try to resolve this or wait for it to resolve. When the partition heals, the kubelet will complete the deletion of the Pod and free up its name in the apiserver.
Normally, the system completes the deletion once the Pod is no longer running on a Node, or the Node is deleted by an administrator. You may override this by force deleting the Pod.

Hi, i have the same issue.
I use kubeadm to create kubernetes cluster. Maybe operator handler alertmanager lifecycle? I don't know more about it.
Kubernetes version information:
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.0", GitCommit:"ddf47ac13c1a9483ea035a79cd7c10005ff21a6d", GitTreeState:"clean", BuildDate:"2018-12-03T21:04:45Z", GoVersion:"go1.11.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.0", GitCommit:"ddf47ac13c1a9483ea035a79cd7c10005ff21a6d", GitTreeState:"clean", BuildDate:"2018-12-03T20:56:12Z", GoVersion:"go1.11.2", Compiler:"gc", Platform:"linux/amd64"}

We moved everything to use a parallel pod management policy in the statefulset, we've seen that to help with this behavior. Let us know if it helps you too. I'm closing this, here but if you are still seeing behavior that you feel is not appropriate, please feel free to create a new issue! :slightly_smiling_face:


[T3A5][T11-A2]Tan Zhi Jie

Accepting this PR without a review because another of your PRs for this tutorial was reviewed in detail.


updateOrCreate not working?

Laravel Version: 5.4.28
PHP Version: 7.0.20
Database Driver & Version: 10.1.24-MariaDB

Description:
updateOrCreate() for some reason doesn't do the trick for me ...
For some reason it only tries to create a row with the email column and ignores the rest (my strings & uuid)...
Steps To Reproduce:
Schema::create('users', function (Blueprint $table) {
	$table->increments('id');
	$table->uuid('ms_ad_uuid')->unique();
	$table->string('display_name')->nullable();
	$table->string('email')->unique();

	$table->rememberToken();
	$table->timestamps();
});

$currentUser = User::updateOrCreate(
	[
		'ms_ad_uuid' => '2b22985a-af6d-4d21-9c3e-4c3682397b50',
	],
	[
		'display_name' => 'test name',
		'email' =><EMAIL_ADDRESS>	]
);

Screenshot with the error:

it's not related with fillable or something ?

Is it a clean model or have you done some customization to the model?

Oh yeah, good point!
It's the
    protected $fillable = [
        'name', 'email', 'password',
    ];

that was there.
So I guess it's the customatisation to the model that I needed to make, i.e adding all the columns that I want to use to that array!

@iamvitali if this is solved kindly close the issue :)


Radiation blocker values for directional plasma and uranium windows

About the PR

Adds missing radiation blocker values for directional plasma and uranium windows.
Why / Balance

Fixes #31972.
The radiation blocker values were determined by material cost; two glass units are required for a full window, one glass for directional. Thus values are halved.
Technical details

yml changes only

plasma.yml
rplasma.yml
ruranium.yml
uranium.yml

Media

Requirements

[x] I have read and I am following the Pull Request Guidelines. I understand that not doing so may get my pr closed at maintainer’s discretion
[x] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Breaking changes

Changelog

:cl:

fix: added missing missing resistance values for directional plasma and uranium windows

https://github.com/user-attachments/assets/ed011c6c-b1c1-4830-8484-8eed73785ca2

very well emisse, don't care about bugs, just press merg button

id rather have something work and be bugged than something not work and be bugged

fair


Publish always by default sidebar to Laravel project

No need to execute command:
php artisan adminlte:sidebar

Done!


Added in config file loading from multiple sources

Can we fall back to i3 config paths, too?

Ok sure.

Also: this still supports ~/.sway/config?

Yes, that's the first thing tried.


Upgrade alpine base images for serverless components

Description
Changes proposed in this pull request:

upgrade alpines to 15.30
upgrade golang-runtimes to 1.18.0-alpine3.15

/test all

/test all


DrawElements does not work in a safe context

Summary
Using a 0 with draw elements results in nothing drawing, no warnings, no errors. But putting an unsafe void* fixes the problem.
        public void DrawElements()
        {
            gl.DrawElements(PrimitiveType.Triangles, (uint)validIndex, DrawElementsType.UnsignedInt, 0);
        }

        public unsafe void DrawElements()
        {
            gl.DrawElements(PrimitiveType.Triangles, (uint)validIndex, DrawElementsType.UnsignedInt, null);
        }

Steps to reproduce

Platform: Desktop
Framework Version: .NET Core 3
API: OpenGL
API Version: OpenGL 4.5 Core

Setup a VAO with EBO and ABO bound correctly
Call Draw elements with 0
Note that it doesn't work.
Call Draw Elements with unsafe null.
Note that it starts working fine.
Comments
This has been noted by multiple people in discord
If you know how to fix this issue, please submit a pull request instead!
I assume its something in the marshal process?

        gl.BindBuffer(BufferTargetARB.ArrayBuffer, vertexData);
        {
            byte[] bogusBuffer = new byte[dataSize];
            gl.BufferData<byte>(BufferTargetARB.ArrayBuffer, (nuint)dataSize, bogusBuffer, drawType); // reserve space
        }

BufferData seems to have a similar problem when trying to reserve buffer space for BufferSubData but instead it crashes with an access exception, requiring you to provide a throw away buffer that open gl would not normally require.

By passing zero you are asking the library to pass a pointer to zero, rather than a zero pointer.
We had tried to remove these overloads, but guess it’s not working. I’ll have another look at it. However, we most likely won’t be adding a safe overload for this.

Thats very interesting, is that what that nint stuff is? Is it not possible to add a marshal for IntPtr.Zero? Thanks for checking, it certainly threw me off, and if it wasn't for someone else mentioning it, I would have been really stuck trying to figure out what wasn't drawing.

You can use IntPtr without needing the unsafe keyword.
On Tue, Mar 21, 2023 at 2:05 PM Beyley Thomas @.***>
wrote:

Thats very interesting, is that what that nint stuff is? Is it not
possible to add a marshal for IntPtr.Zero? Thanks for checking, it
certainly threw me off, and if it wasn't for someone else mentioning it, I
would have been really stuck trying to figure out what wasn't drawing.
What would be the use of an IntPtr overload when there is a void* one?
—
Reply to this email directly, view it on GitHub
https://github.com/dotnet/Silk.NET/issues/1350#issuecomment-1478578739,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA4KCBSPJHKTEQGUEXKOAYLW5IJ3DANCNFSM6AAAAAAWB7YOLM
.
You are receiving this because you authored the thread.Message ID:
@.***>

Why avoid unsafe? that would just bloat overload counts

Ya, I guess I've just enjoyed OpenTK's ability to leave the Unsafe options off yet still can do all the things I need to do to render things.  But if being an safe compatible API is not a goal (which does not appear to be the case), then just removing the errant behavior would be good.

I have experimented with fixing this and unfortunately I cannot implement a fix that is acceptable to the team.
Please note: even if you're not using unsafe language features, everything you do with Silk.NET (or even OpenTK or any native interop library) is inherently unsafe and avoiding those language features doesn't change that.
Please use null instead of 0. 0 passes an implicit pointer to 0 which is not what you want in most cases. This will require an unsafe context.
Thanks.


Implement a Data Preprocessing Pipeline

Implement a pipeline that performs the following;

Performing normalization, scaling on the data
De-seasonalizing or de-trending data
Smoothing/de-noising the data
Or any additional preprocessing methods

The pipeline should perform some of (possibly all) of the above according to user given specifications, and return the processed data.
This pipeline preferably should be handled by a class, that calls the necessary functions according to given arguments.
!!! This issue is not important for the short term, however you may implement some of the functionalities if you ever want to.

We can slowly start working on this issue now. For some ideas;

Our data transform classes can be used to scale/normalize data
The decomposition classes can be used to smooth out the data and extract related components


[ALLUXIO-2139] Mark BlockLockManagerTest as final class

https://alluxio.atlassian.net/browse/ALLUXIO-2139

Can one of the admins verify this patch?

ok to test

LGTM

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/11325/
Test PASSed.

Jenkins, add to whitelist

LGTM


Consider deprecating base+polyfill due to complexity and misuse

Since releasing the "base+polyfill" version of this library, we've received a number of reports that it's complex and confusing to use (e.g. https://github.com/GoogleChrome/web-vitals/issues/95).
The primary benefit you get from using the "base+polyfill" build is you can collect FID data from all browsers. You also potentially get more accurate visibility reporting, but we have no data on how often that accuracy affects the value of the metrics (i.e. it's more accurate in theory). In other words, if you don't want FID data in other browsers, it's likely not worth using the polyfill build.
Given that the vast, vast majority of sites across the web report good FID scores (more than 95%), given that Firefox now does support FID natively, and given that this library already exposes the INP metric (which was created as a replacement for FID and a more accurate measure of responsiveness), it's worth reconsidering whether or not the additional complexity of including the FID polyfill (both for end users as well as building this library) is worth it.
To answer that question, I wanted to look at how many people were using the polyfill, so I queried the HTTP Archive response_bodies table from June 1, 2022 to look for sites that were using web-vitals at all, and then specifically sites that were shipping code unique to the "base+polyfill" build.
Here were the results:

Segment
Count

All versions
706,302

"base+polyfill" version
22,905

So only about 3% of sites using web-vitals are using the "base+polyfill" version.
However, when I started looking at some of the individual sites that were using the polyfill, I noticed many, many cases where sites were using it incorrectly—including cases where sites were using a third-party service and that service was using the polyfill version incorrectly (raygun and catchpoint analytics stood out, if you work on these products please reach out and I can help with proper implementation).
Examples I found of misuse:

Loading the base bundle without the polyfill script at all (meaning FID won't report).
Loading the base bundle and the polyfill bundle async (the polyfill script needs to be inlined in the <head> or it's not accurate).
Loading the polyfill script as a sync, blocking script in the head (which negatively affects load performance)

I was curious to see if I could query how often sites were using it correctly vs. incorrectly, and this is what I found.

Segment
Count

Correct usage
9,349

Loading the polyfill via an external script (incorrect)
13,556

Loading the polyfill via an external, blocking script (incorrect)
4,813

To summarize, based on my research, it looks like the majority of sites who use the polyfill use it incorrectly (so they get no benefit and likely report incorrect data), and a sizeable portion of sites use it in a way that negatively affects performance (blocking scripts). Only about 1% of sites that use the web-vitals library use the polyfill and use it correctly, and it's unclear how many of them actually look at the polyfill data they get for FID.
Based on all of these reasons, I think it makes sense to deprecate the polyfill in version 3 and then remove it in version 4, but I'd love to hear feedback from anyone who is currently using the polyfill and would not want to see it removed.

I don't think base+polyfill should be removed just warning.

[x] -

What
Get Outlook for iOShttps://aka.ms/o0ukef

From: NavinPa @.>
Sent: Saturday, February 24, 2024 8:05:30 PM
To: GoogleChrome/web-vitals @.>
Cc: ShuaibZaker-Create @.>; Mention @.>
Subject: Re: [GoogleChrome/web-vitals] Consider deprecating base+polyfill due to complexity and misuse (Issue #238)
#95https://github.com/GoogleChrome/web-vitals/issues/95
—
Reply to this email directly, view it on GitHubhttps://github.com/GoogleChrome/web-vitals/issues/238#issuecomment-1962787261, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BGDRJLV25POWN7NHVI2MIN3YVKL6VAVCNFSM53FN2K52U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJWGI3TQNZSGYYQ.
You are receiving this because you were mentioned.Message ID: @.***>


Dropdowns should only toggle one at a time

This fix will prevent all dropdowns from opening at the same time in the event that more than one is put on a page by binding the toggle to the element that is clicked not a page-wide jquery selector.
A few things worth note:
This PR includes manually converted JS -> Coffeescript via js2.coffee as the automatic converter (rake coffee) reverted my changes, but left the js unchanged?
The contribution docs make no note of having to install the npm package js2coffee, which was easy enough to figure out but should probably be included via a package.json.
Finally when running rake coffee several files were changed unrelated to anything I've done (source/javascripts/refills/coffeescript/accordion_tabs.coffee, source/javascripts/refills/coffeescript/accordion_tabs_minimal.coffee, source/javascripts/refills/coffeescript/navigation_centered.coffee).

@jbender thanks for this! Looking at it now.

@jbender very nice... merging this!


Added Khujand Polytechnic Institute of Tajik Technical University

Added Khujand Polytechnic Institute of Tajik Technical University

@abdukhafiz Pull request merged. Thank you!


Create dataset loader for indonglish-dataset

Dataloader name: indonglish/indonglish.py
DataCatalogue: http://seacrowd.github.io/seacrowd-catalogue/card.html?indonglish

Dataset
indonglish

Description
Indonglish-dataset was constructed based on keywords derived from the sociolinguistic phenomenon observed among teenagers in South Jakarta. The dataset was designed to tackle the semantic task of sentiment analysis, incorporating three distinct label categories: positive, negative, and neutral. The annotation of the dataset was carried out by a panel of five annotators, each possessing expertise language and data science

Subsets
-

Languages
ind

Tasks
Sentiment Analysis

License
Unknown (unknown)

Homepage
https://github.com/laksmitawidya/indonglish-dataset

HF URL
https://huggingface.co/TalTechNLP/voxlingua107-epaca-tdnn

Paper URL
https://thesai.org/Publications/ViewPaper?Volume=14&Issue=10&Code=IJACSA&SerialNo=53

#self-assign

@SamuelCahyawijaya I can't find how the dataset in https://huggingface.co/TalTechNLP/voxlingua107-epaca-tdnn is used in this work. I'll implement for the datasets in https://github.com/laksmitawidya/indonglish-dataset?


Consider whether we want to keep ci/templates

https://github.com/googleapis/google-cloud-cpp/tree/master/ci/templates contains scripts and .in files that help us generate ci/kokoro/install/Dockerfile.* and its build.sh script. I believe the intent of ci/templates was to help us managed similar builds across multiple repos. However, with a monorepo we no longer have a problem keeping things in sync.
Therefore, we should consider whether it helps/hurts to keep these templates and generate these files vs just keeping/editing the files directly (like we do in ci/kokoro/docker/).

Quick thoughts:

I think generating ci/kokoro/install/build.sh is no longer a good idea.
I think generating ci/kokoro/install/Dockerfile.* is still useful (lots of repetition across them)
It would be nice if we also generated ci/kokoro/docker/Dockerfile.*

Having said that, if we delete all this code you will not make it terribly said, making the same changes to 13-14 Dockerfiles is relatively easy.


Bump version of syn

Hey! Lovely crate, but in order to make it work:
I had to bump syn to syn = "1.0.72". Then I had to go to bevy and update the syn used there to syn = "1.0.72", and finally the crate would compile with other things I had as well.
The problem is that standard-dist also uses a newer syn crate.

That's weird, I depend on syn = 1.0which if I understand correctly means every dependency semver-compatible to1.0, so every version 1.x.y`. What is the error message you get?

Right, I'm working off main branch of bevy. So this is a future thing.
On Thu, Jun 24, 2021, 19:50 Jakob Hellermann @.***>
wrote:

That's weird, I depend on syn = 1.0which if I understand correctly means
every dependency semver-compatible to1.0, so every version 1.x.y`. What
is the error message you get?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/jakobhellermann/bevycheck/issues/3#issuecomment-867834599,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAIDVSBAPSUKENTEQH5PPM3TUNV7BANCNFSM47FLLD2A
.

While bevycheck depends on bevy = 0.5, I think it should work with bevy from the main branch.
Try adding
[patch.crates-io]
bevy = { git = "https://github.com/bevyengine/bevy", branch = "main" }

to your Cargo.toml, this will override the bevy dependency to use bevy main aswell.


Small UI bug

Win 8.1
Chrome 50

Thanks @ilkome

Do you still view this bug?

Yes, same in latest Firefox DE.
I have 3200*1800 resolution on 15 inc laptop, maybe it's will help.


Send name improvements

For the send name input, the send name should be disabled if a valid address is not input.

I selected to transfer both owner and manager. After the first transaction I closed the modal and clicked send again. The transfer name modal did not update and still shows being able to transfer both the owner and manager.

In wrapped subnames. Send is still visible after sending the name.

When transferring both owner and manager, the transaction screen looks the same. Should add a description item that explains what is being transferred.


PTFE-1959 support deletion of non-existent vsphere vm

Codecov Report
Attention: Patch coverage is 14.28571% with 6 lines in your changes missing coverage. Please review.

Project coverage is 86.30%. Comparing base (2d4bb12) to head (a6819cd).

Files with missing lines
Patch %
Lines

runner_manager/backend/vsphere.py
14.28%
6 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #667      +/-   ##
==========================================
- Coverage   86.66%   86.30%   -0.37%
==========================================
  Files          34       34
  Lines        1425     1431       +6
==========================================
  Hits         1235     1235
- Misses        190      196       +6

Flag
Coverage Δ

api
64.91% <14.28%> (-0.21%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Stop using handler.abort in validators

raise exceptions instead (caught by base.RequestHandler.handle_exception)
partially addresses #66

LGTM, btw.

:+1:

LGTM, merge as desired


imports does not import profiles

What happened?
I'm unsure if this is a bug or a feature request.
I tried to use the new imports feature in v6 to import a profile. I want to be able to standardize some profiles across many different applications, so importing them seemed like the best way to do this ( can't use expressions cause expressions don't work in the profiles section as I understand).
However, when doing this, the profile does not seem to be imported.
What did you expect to happen instead?
I don't see anyting in the documentation that suggests profiles will not be imported. So either I would expect the docs to make it clear what types of things can be imported and which can't, or I would expect profile imports to work the same way other sections work.
How can we reproduce the bug? (as minimally and precisely as possible)
Here's the devspace.yaml that I want the profile imported into. Note the imports statement towards the top:
version: v2beta1
name: email
imports:
  - path: ../../devspace/default-pipelines.yaml
deployments:
  email:
    kubectl:
      manifests:
        - ./redoxlocal/configmap.yaml
        - ./redoxlocal/manifest.yaml
dev:
  email:
    labelSelector:
      service: email
      service-target-name: email-app
      ksync: app
    command:
      - bash
      - -c
      - npm run debug
    args: []
    ports:
      - port: 9229:9229
    sync:
      - $(cat ../../devspace/default-terminal-sync.yaml)

And here's the file that has the "shared" profile and pipelines:
version: v2beta1
name: default-pipelines

imports:
- path: ./common.yaml

pipelines:
  dev:
    run: |-
      run_dependencies --all
      create_deployments --all
      start_dev --all
  deploy:
    run: |-
      run_dependencies --all
      create_deployments --all

profiles:
- name: logs
  patches:
    - op: replace
      path: dev.*.logs
      value:
        enabled: true
        lastLines: 200

When I run devspace dev -p logs I get the following:

fatal Couldn't find profile 'logs'

Local Environment:  MacOS Monterey

DevSpace Version: devspace version 6.0.0-beta.3
Operating System:  mac
ARCH of the OS: AMD64 | ARM64 | i386
Kubernetes Cluster:
Cloud Provider: aws
Kubernetes Version: 1.22

Anything else we need to know?

/kind bug

@tobalsgithub thanks for creating this issue! This is currently expected and currently a limitation of the imports functionality. We can think about adding profiles to the imports functionality, but as of now, this is not working.

Thanks @FabianKramm . Are there other limitations of the imports functionality? It would be great if the documentation made it clear what types of things can be imported vs what types of things cannot.
And for what it's worth, it would be great if imports worked for profiles. Right now, there's no good way that I know of to share profiles across a lot of different projects, because both imports and expressions cannot be used with profiles. So consider this a feature request 😄


BUILD FAILED in 48s 1 actionable task: 1 executed ERROR: Service 'tts' failed to build: The command '/bin/sh -c cat components.json | jq .[].artifact | xargs ./marytts insta

Getting this error:
BUILD FAILED in 48s
1 actionable task: 1 executed
ERROR: Service 'tts' failed to build: The command '/bin/sh -c cat components.json | jq .[].artifact | xargs ./marytts insta

hi, to overcome these build problems, i just published the prebuilt images on dockerhub. pls get the latest version of the repository, the docker-compose file now points to the prebuilt images.

hi, to overcome these build problems, i just published the prebuilt images on dockerhub. pls get the latest version of the repository, the docker-compose file now points to the prebuilt images.

Thanks a ton.

@codeforequity-at it's working now, thanks a lot!


Adds route resources option

This PR adds the options parameter when using Route::resources and Route::apiResources by taking an options array as the second parameter. These are then passed to each of the resource function calls.
An example of the suggested addition

Route::resources([
        'resource-one'      => 'ResourceOneController',
        'resource-two'      => 'ResourceTwoController',
        'resource-three'    => 'ResourceThreeController',
    ], [ 'except' => [ 'create', 'show' ] ] );

Which can replace doing:
Route::resource('resource-one', 'ResourceOneController', ['except' => ['create', 'show']]);
Route::resource('resource-two', 'ResourceTwoController', ['except' => ['create', 'show']]);
Route::resource('resource-three', 'ResourceThreeController', ['except' => ['create', 'show']]);

Which in the end would generate:

POST      | resource-one                          | resouce-one.store      | App\Http\Controllers\ResourceOne@store      | web |
GET|HEAD  | resource-one                          | resouce-one.index      | App\Http\Controllers\ResourceOne@index      | web |
DELETE    | resource-one/{resource_one}           | resouce-one.destroy    | App\Http\Controllers\ResourceOne@destroy    | web |
PUT|PATCH | resource-one/{resource_one}           | resouce-one.update     | App\Http\Controllers\ResourceOne@update     | web |
GET|HEAD  | resource-one/{resource_one}/edit      | resouce-one.edit       | App\Http\Controllers\ResourceOne@edit       | web |

POST      | resource-two                          | resouce-two.store      | App\Http\Controllers\ResourceTwo@store      | web |
GET|HEAD  | resource-two                          | resouce-two.index      | App\Http\Controllers\ResourceTwo@index      | web |
DELETE    | resource-two/{resource_two}           | resouce-two.destroy    | App\Http\Controllers\ResourceTwo@destroy    | web |
PUT|PATCH | resource-two/{resource_two}           | resouce-two.update     | App\Http\Controllers\ResourceTwo@update     | web |
GET|HEAD  | resource-two/{resource_two}/edit      | resouce-two.edit       | App\Http\Controllers\ResourceTwo@edit       | web |

POST      | resource-three                        | resouce-three.store    | App\Http\Controllers\ResourceThree@store    | web |
GET|HEAD  | resource-three                        | resouce-three.index    | App\Http\Controllers\ResourceThree@index    | web |
DELETE    | resource-three/{resource_three}       | resouce-three.destroy  | App\Http\Controllers\ResourceThree@destroy  | web |
PUT|PATCH | resource-three/{resource_three}       | resouce-three.update   | App\Http\Controllers\ResourceThree@update   | web |
GET|HEAD  | resource-three/{resource_three}/edit  | resouce-three.edit     | App\Http\Controllers\ResourceThree@edit     | web |

Since this is a breaking change please target this to 5.7

I don't think this is a breaking change, it should work exactly the same as before?
The current method is:
    public function resources(array $resources)

The proposed method simply adds the options parameter as an optional addition:
    public function resources(array $resources, array $options = [])

So calling the proposed method without an options parameter, should perform exactly the same way as the original method

Changes in method signatures are breaking changes.
Anyone who is extending this class and overriding those methods would need to update their code to include the extra parameter.

Ah right. I'll close this and send a new PR targeting 5.7


Error in Refresh token

Hello all, I'm trying to make a refresh token call using:
oauth2Client.refreshAccessToken(function(err, tokens) {
  // your access_token is now refreshed and stored in oauth2Client
  // store these new tokens in a safe place (e.g. database)
});

But I'm receiving the following error:
{
  error: 'deleted_client',
  error_description: 'The OAuth client was deleted.'
}
code: '401'

Does anybody how to handle this error? The complete code bellow.  Thank you. I'm trying to refresh the refresh token as it expires in 1 hour so I'm trying to refresh it on every call.
The complete code bellow:
const { google } = require('googleapis');
var oauth2Client = google.auth.OAuth2;
var CLIENT_ID = 'xxxxxx';
var CLIENT_SECRET = 'xxxxxx';
var access_token = 'xxxxxx'
var refresh_token ='xxxxx'

var BASE_URL = 'http://localhost:4000';
var REDIRECT_URL = 'http://localhost:3000/oauth2Callback';
var oauth2Client = new google.auth.OAuth2(CLIENT_ID, CLIENT_SECRET,REDIRECT_URL);

const credentials = {access_token: access_token, refresh_token: refresh_token};
oauth2Client.setCredentials(credentials);

oauth2Client.refreshAccessToken(function(err, tokens) {
//console.log(err);
// your access_token is now refreshed and stored in oauth2Client
// store these new tokens in a safe place (e.g. database)
});

Greetings!  May I ask - why are you trying to manually refresh the token?  The auth client takes care of this for you usually :)

So @JustinBeckwith, we don't have to generate new access token when old once expires??

Yes :)  The getRequestMetadata and request methods automagically get an updated access token using the refresh token for you :)


[Crane] Figure out how to pass onClickLabel to Surface

The 1.2.0-alpha06 androidx.compose.material update removed onClickLabel from the Surface API. That means the Crane app's DayContainer no longer has a clean way to specify the onClickLabel (see here).
We need to figure out the best way to still set the onClickLabel to ensure the "Double-click to $onClickLabel" announcements are accurate. The selectable version of Surface uses a selectable modifier, which doesn't have an onClickLabel. The clickable Surface sets a clickable modifier, which could support an onClickLabel, but Surface doesn't expose it. Using the non-interactive Surface and specifying the full clickable modifier ourselves might work but's duplicating quite a few lines and would have to be evaluated to see if there are any problems with the other modifiers that Surface adds.

Fixed!


jsonrpc --pipe non functioning

Bicep version
0.26.54 (Windows)
Describe the bug
When following guidance to create a new bicep jsonrpc connection using named socket, it will not create a new socket that can be used.
Ive tried this on 3 different computer (2 windows 11 machines and a mac) but in none of the cases a new socket is created for consumption.
To Reproduce
Steps to reproduce the behavior:
run command: bicep jsonrpc --pipe \.\pipe\bicep-test-jsonrpc-sock.sock (or equivalent on mac)
run command (pwsh): get-childitem \.\pipe\ -Filter 'bicep' -Force
nothing shows up.
when trying to add "jsonrpc"=true to bicep config this also does not seem to do anything
Additional context
even if i use System.IO.Pipes.NamedPipeClientStream it stalls on connect() that hangs my current session.
i thought something was wrong and mabye pipes where blocked by fw of policy so i added bicep specifically to allow any actions through the computer firewall as well as wrote a small script to create a server using System.IO.Pipes.NamedPipeServerStream. it does seem to work, when testing that pipes are allowed to exist.
TRACE: Bicep version: 0.26.54+5e20b29b58, CLI arguments: "jsonrpc --pipe \\.\pipe\bicep-test-jsonrpc-sock.sock"
The 'jsonrpc' CLI command group is an experimental feature. Experimental features should be enabled for testing purposes only, as there are no guarantees about the quality or stability of these features. Do not enable these settings for any production usage, or your production environment may be subject to breaking.

The --pipe flag instructs the CLI to connect to an existing JSONRPC server as a client via a named pipe, so in your example, you will have to pre-create the named pipe server and start listening on it, before invoking the command containing the name of the pipe.
Here's an example of what this looks like in C#, and here's an example in Node.
Apologies, this is not at all clear from the documentation - I will update it to clarify!

Thank you for claryfying! I saw in the source that you used a client as part of the jsonrpc handler, but I thought it was some self tests to ensure connection.
In the edited documentation, it still says that you can use it as a server (line 2), is this still the case, and if so, how?

Thanks for pointing out the additional documentation issues! I've submitted a PR here to address them: #13895


Updates and potential syntax adjustments.

Hi @gfx
Thanks for doing all the heavy lifting on this here json parser!  I noticed the library hadn't been updated in a while, and needed some cleanup to be compatible with current Swift.  Along the way, I made some contributions that hopefully you'll find useful, totally open to discussion.  Some of the things I changed were:

Updating to latest Swift syntax
Moving from Result<T> to try-catch for synchronous operations.
Cohesive naming scheme serialize / deserialize
Access control keeping serialization / deserialization associated w/ singular Json object
Remove C style for loops / while loops (these are unsupported in future Swift versions)
Simplifications regarding buffers

Essentially, front loading some of the buffer parsing instead of doing it in parallel.  This adds slight overhead to unsuccessful operations, since there is some unnecessarily collected data.  I didn't notice a performance change in performance tests.

My main motivation for these changes is adding completely foundation independent support for my existing library: Genome.  Unfortunately, even though I removed all Foundation references, there's still an inability to cast value types to AnyObject which was happening implicitly w/o foundation references and I didn't realize it.
As I said earlier, totally open to discussing the implications of some of these changes, and if necessary, I'm happy to maintain my changes in a separate fork.
Going Forward
There's a few other things I'd like to get setup.

[ ] Cocoapods support

I'm happy to do the podspec setup etc., but if you want to maintain ownership over the pod, you should be the one to push it to Trunk

[ ] Package Support

Once this library is setup for integration, I'd like to investigate supporting Swift Packages so that our libraries can be bundled that way.

Thanks again for getting this library going,
Logan

Basically agreed with this patch. Wait a minute for reviewing.

The code looks good.
Do you have any idea to pass tests for CI?

@gfx
Awesome, glad that this can be helpful.  I think I have one more small commit for a couple of things at some point, then we can talk about merging.
Travis:
I'm not all to familiar with Travis, but looking through the build logs, there's two major problems I see.
iOS 7.0
I had dropped 7.0 in favor of 8.0 support since iOS 9 has been released and this is generally an acceptable support range.  I don't think the library is too reliant on iOS 8+ capabilities, so I'll toy w/ dropping requirement to 7 in project to see if anything is affected.
Xcode 6.1
The build server is using version 6.1, when it should be on at least 7.0 to take advantage of the latest syntax, etc.  I think 7.2 would be best.

@LoganWright Xcode 7.2 seems available on Travis. Can you try this? https://github.com/gfx/Swift-JsonSerializer/pull/5/files
I agree with dropping iOS7.0. I'd do so if you didn't.

@gfx
Ok, I'll leave the 8.0+ requirement for now then.  The PR you posted looks fine, but I won't be able to try it without you merging since I'd have to setup a separate travis instance.  If you merge in yours, I can push something to make it re run checks on this branch.

@gfx
Not sure why, but Travis doesn't seem to have upgraded, it's still trying to compile on 6.1:
https://travis-ci.org/gfx/Swift-JsonSerializer/builds/100691539#L101

@LoganWright Oops. Can you apply #5 to this branch by yourself?

@gfx - I didn't realize I could have configured it from this branch!  Learning about travis a little bit through this.  It looks like we've finally reached a passing stage :)

Great :clap:
I'm merging this branch. And now you are a collaborator of this project ;)
Because the main branch of this project is protected, you can't push commits to the master directly. You have to make a pull request for each change. However, you can merge any pull-requests without my review.
Thanks again!


ADP content update bumps the date for the CVE record

From the 2023-05-03 SPWG meeting, confirm this behavior: changes to an ADP container bump the date on the container, and also bump the date on the CVE record.
If an ADP adds or updates an ADP container frequently, consumers will see lots of updated records. dateUpdated is defined as "The date/time the record was last updated." We probably don't want to redefine this. Semantically, the ADP container is part of the the record, data was updated, so consumers should be made aware by a new date on the record.
Significantly more frequent updates may surprise consumers.
From https://github.com/CVEProject/automation-working-group/issues/117

Art: It appears that dateUpdated is bumped within each container and also at the CVE Record level. I took a look at https://cveawg.mitre.org/api/cve/CVE-2022-3205 as an example. You can see different dateUpdated values for the CNA and the CVE Program container in the example. The most recent date of update from the CVE Program container is reflected at the CVE Record level as i would expect, since that is the more recent of the two. I think this is correct, but let me know if i am missing something.


relative map and perception

message LaneMarker {
optional apollo.hdmap.LaneBoundaryType.Type lane_type = 1;
optional double quality = 2; // range = [0,1]; 1 = the best quality
optional int32 model_degree = 3;
// equation X = c3 * Z^3 + c2 * Z^2 + c1 * Z + c0
optional double c0_position = 4;
optional double c1_heading_angle = 5;
optional double c2_curvature = 6;
optional double c3_curvature_derivative = 7;
optional double view_range = 8;
optional double longitude_start = 9;
optional double longitude_end = 10;
}
I learn the relative map recently and i find the navigation path generated from lane marker information needs the c0~c3.Can anyone tell me the meaning of c0~c3 and how to acquire these information?I will really appreciate you!

For lane marker, Apollo 3.0 has cubic parameters. Please check any branch around Aug 27, 2018.

@techoe What is the specific meaning of c0-c3?

coefficient of cubic curve.
c3 * x^3 + c2 * x^2 + c1 * x + c0 = 0

Closing this issue as it appears to be resolved. Feel free to reopen if you have additional questions!


Add support for serialization

The generated entity classes should support being used with a serialization package like dart_mappable or json_serializable.

Added in 36fb7fb and a5bb918


How to log in with Facebook ??

How to log in with Facebook ??
already try with facebook-login-for-robots
and it throws me errors, why do not you create a tutorial on how to do this? there are many people who have the same problem I see

I would love to write a tutorial as soon as I succeed running the app. 😎

Referencing issue #880 since it appears to be the furthest along.


NoMethodError (undefined method login' for #ActionDispatch::Request...

rails 4.1, devise 3.4
I can create a user fine. The user just can't sign in (ie problem in POST users/sign_in, Devise::SessionsController#create). Here are some key variables from config/initializers/devise.rb:
config.authentication_keys = [ :login ]
config.request_keys = [ :login ]
config.http_authenticatable = [false]

Here is the form from controllers/sessions/new.html.erb
<%= form_for(resource, as: resource_name, url: session_path(resource_name)) do |f| %>

<%= f.label :login %>

  <%= f.email_field :login, autofocus: true %>

<%= f.label :password %>

  <%= f.password_field :password, autocomplete: "off" %>

  <% if devise_mapping.rememberable? -%>

<%= f.check_box :remember_me %> <%= f.label :remember_me %>

 <% end -%>

<%= f.submit "Log in" %>

 <% end %>

config/routes.rb
  devise_for :users, controllers: {
     sessions: 'users/sessions'
  }

models/User.rb
 class User < ActiveRecord::Base
   # Include default devise modules. Others available are:
   #  :confirmable, :lockable, :timeoutable and :omniauthable
   devise :database_authenticatable, :registerable,
         :recoverable, :rememberable, :trackable, :validatable,
         :authentication_keys => [:login]

   attr_accessor :login
....
end

Here are the parameters submitted:
{"utf8"=>"✓",
    "authenticity_token"=>"yada",
<EMAIL_ADDRESS>
      "password"=>"[FILTERED]",
      "remember_me"=>"0"},
    "commit"=>"Log in"}

I ended up solving my issue by changing lib/devise/strategies/authenticable.rb#148 to:
      def request_values
        keys = request_keys.respond_to?(:keys) ? request_keys.keys : request_keys
        values = keys.map { |k| self.request[self.scope][k] }
        Hash[keys.zip(values)]
      end

You can view the full stacktrace on Stackoverflow.
Is this something I should create a pull request for??

Closing this one since it isn't a real issue with Devise and this is the expected behavior.


[Timespinner] Updates

Increase roll chance as people complained there where not enough seeds
Updated settings with latest available settings

It was already doubled here: https://github.com/ArchipelagoMW/AsyncTools/commit/cdeb139ef0c9f4a3ee205e4358a22460d1a8bbad#diff-d50b2f8ca243dffb8d9fa62f77a7aaf6818d789534d1a469cd763cfa0a389f7eR11


Errors with no call site in newest flow

I just updated to flow 0.59.0, and the result was a bunch of the following errors:

I have no clue what is causing them, where they are originating from, or what I can do to fix this. I've been going around in my code trying to type things that use the function it's complaining about, and this does not help.
The function in question is defined as following:
function object<S: { +[name: string]: ArbGen<*> }>(
  map: S
): ArbGen<$ObjMap<S, <V>(v: ArbGen<V>) => V>> { /* implementation */ }

I'm unable to produce a minimal example that shows this problem, because I have no clue what's causing it. I just know this used to work in flow 0.56.0.
[Edit]
I just tested on older flow versions, and this broke in 0.57.1. There is a very good chance that this is an issue in my code, I just don't know how to find it, nor fix it, cause I'm getting next to no error information.

Maybe related to https://github.com/facebook/flow/issues/2674#issuecomment-344714624 ?

I remember looking at that issue before I started writing this library at all, and I'm pretty sure that had already been fixed, but I might remember wrong.

I managed to make a somewhat minimal reproduction. Maybe someone can tell me what is wrong here:
// @flow

export type ArbGen<+T> = {
  getSingle(): T,
};

declare function zipObject(o: any, props: $ReadOnlyArray<string>): any;

declare export function combinations(...arbs: $ReadOnlyArray<ArbGen<*>>): ArbGen<$ReadOnlyArray<any>>;

declare export function object<S: { +[name: string]: ArbGen<*> }>(map: S): ArbGen<$ObjMap<S, <V>(v: ArbGen<V>) => V>>;

declare var age: ArbGen<number>;
declare var name: ArbGen<string>;

type Person = { age: number; name: string };
const p: Person = object({
  age,
  name
}).getSingle();

I was able to work around this by replacing * with mixed. The behavior of * can be pretty surprising, and while I'm not sure exactly what is happening here, I think the solution is just to avoid * here.

I still don't think this issue should be closed, because the bigger issue is the fact that the error does not provide any information about what is actually wrong, nor how to fix it. And this clearly seems like a bug, no?


feat (apps/pano): use dialog for post deletion

Adds the modal required for post deletion using the dialog component
closes #569

there are still some things needs to be done so I'm converting it to draft.

Backend is missing
Variable namings are not fixed
Custom "username : cancan" is still there 🤣

there are still some things needs to be done so I'm converting it to draft.

Backend is missing
Variable namings are not fixed
Custom "username : cancan" is still there 🤣
rebase branch from develop please

Thanks so much for converting it to draft! We'll be integrating backend when it's ready. This is kind of a placeholder implementation to make layout ready for the logic.


k8s.gcr.io Image Registry Will Be Frozen From the 3rd of April 2023

Per post k8s.gcr.io will be frozen.
Any impact on nebula-operator?

There should be no impact now as the nebula-related containers are all docker-hub-based image delivery. @MegaByte875 correct me if wrong.

Please look at the Nebula-operator helm charts
Scheduler pulling the image

https://github.com/vesoft-inc/nebula-operator/blob/367c1336afa4f6e1610d6a419f4a784ce631378c/charts/nebula-operator/values.yaml#L9


Test Flake (general): Prometheus pod is failing to start

https://circleci.com/gh/istio/istio/318829?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link
The failure is in e2e-galley test, but it is caused by Prometheus pod not starting.
We have seen this failure a couple of times when investigating some flakes.

@bianpengyuan can you take a look?  this seems like it might be related to the recent changes to Prom readiness ?

BTW, I'm seeing this failure to initialize in a number of PRs and not limited strictly to the Prometheus pod.

I don't think prom-init container is the culprit. Its behavior is deterministic (check existence of certs file) and should not introduce any flake. In fact, while investigating the issue, I found a issue that prom-init container will always pass even when secret is not ready. So I suspect this is more of a general test-infra issue.

Happened for me this morning on https://circleci.com/gh/istio/istio/321930
"Failed to complete Init. Error context deadline exceeded" caused by Prometheus getting stuck in 0/1       Init:0/1   0          16m.  The build attempts Running command kubectl logs prometheus-68896c5775-5rtm6 -n istio-system -c prometheus --kubeconfig= which fails with Command error: exit status 1 and of course there are no logs in Init:0/1.
I propose that FetchAndSaveClusterLogs() should do kubectl describe in additional to kubectl logs.  Perhaps for every pod but definitely for the pods that failed kubectl logs.

Happened for me this morning in https://circleci.com/gh/istio/istio/326914 .  @GregHanson are you the test-watcher this month?  How about the idea I proposed above, having FetchAndSaveClusterLogs() record pod status so we can see why Prometheus is getting stuck?

@ozevren Please let's merge https://github.com/istio/istio/pull/11904 so we can get descriptions of why the non-starting pods aren't starting.

Since https://github.com/istio/istio/pull/13571, I have not seen evidence that this issue persists.
Last report was 2 months ago.
Closing. Please re-open if issue persists.


Traefik SSL is not working with and without Cloudflare

When i try to launch traefik with and without cloudflare.
version: '3'

networks:
  proxy:
    external: true

services:
  traefik:
    image: traefik:v2.2
    container_name: traefik
    logging:
      options:
        max-size: 10m
    command:

      - "--api.dashboard=true" #Enable Traefik Dashboard
      - "--providers.docker=true"
      - "--providers.docker.endpoint=tcp://socket-proxy:2375" #Connect to socket-proxy container that monitors Docker Unix Sock
      - "--providers.docker.watch=true"
      - "--providers.docker.exposedbydefault=false"
      - "--entrypoints.http.address=:80"
      - "--entrypoints.https.address=:443"
      #- "--serversTransport.insecureSkipVerify=true" #Disables SSL Certificate Verification between Traefik and endpoint
      #- "--entryPoints.http.forwardedHeaders.trustedIPs=<IP_ADDRESS>/32, 192.168.xxx.xxx/24" #Uncomment and set IP 192.168.xxx.xxx if you have upstream proxy like Nginx in order to get correct X-Forwarded-for IP header
      #- "--entryPoints.https.forwardedHeaders.trustedIPs=<IP_ADDRESS>/32, 192.168.xxx.xxx/24" #Uncomment and set IP 192.168.xxx.xxx if you have upstream proxy like Nginx in order to get correct X-Forwarded-for IP header
      - "--log=true"
      - "--log.level=DEBUG" # (Default: error) DEBUG, INFO, WARN, ERROR, FATAL, PANIC
      - "--accessLog=true"
      - "--accessLog.filePath=/var/log/docker/traefik.log"
      - "--accessLog.bufferingSize=100" # Configuring a buffer of 100 lines
#     - "--accessLog.filters.statusCodes=400-499"
#      - "--certificatesResolvers.le.acme.caServer=https://acme-staging-v02.api.letsencrypt.org/directory" # uncomment when testing

       - "--certificatesResolvers.le.acme.dnsChallenge.provider=cloudflare"
       - "--certificatesResolvers.le.acme.dnsChallenge.resolvers=<IP_ADDRESS>:53,<IP_ADDRESS>:53" #ACME dnsChallenge uncomment when using ACME dnsChallenge
#      - "--certificatesResolvers.le.acme.dnsChallenge.delayBeforeCheck=60" #ACME dnsChallenge uncomment when using ACME dnsChallenge
      - "--certificatesResolvers.le.acme.httpChallenge.entryPoint=http" #ACME httpChallange
#      - "--certificatesresolvers.le.acme.tlschallenge=true" #ACME TLS Challenge
#      -<EMAIL_ADDRESS>      - "--certificatesresolvers.le.acme.storage=/etc/traefik/acme.json"
      - "--providers.file.filename=/traefik_conf.yaml" #Qualsys TLS Settings
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.traefik.entrypoints=http"
      - "traefik.http.routers.traefik.rule=Host(`traefik.domain.in`)"
      - "traefik.http.middlewares.traefik-auth.basicauth.users=ghost:$$2y$$05$$sM6kwkUEK5Af08PHGW9uQO3IMoTzSjhropxMSrB5qeCe9XnqPVt0S"
      - "traefik.http.middlewares.traefik-https-redirect.redirectscheme.scheme=https"
      - "traefik.http.routers.traefik.middlewares=traefik-https-redirect"
      - "traefik.http.routers.traefik-secure.entrypoints=https"
      - "traefik.http.routers.traefik-secure.rule=Host(`traefik.domain.in`)"
      - "traefik.http.routers.traefik-secure.middlewares=traefik-auth"
      - "traefik.http.routers.traefik-secure.tls=true"
#      - "traefik.http.routers.traefik-secure.tls.certresolver=le"
      - "traefik.http.routers.traefik-secure.service=api@internal"
#Qualsys TLS Settings
      - "traefik.http.middlewares.securedheaders.headers.forcestsheader=true"
      - "traefik.http.middlewares.securedheaders.headers.sslRedirect=true"
      - "traefik.http.middlewares.securedheaders.headers.STSPreload=true"
      - "traefik.http.middlewares.securedheaders.headers.ContentTypeNosniff=true"
      - "traefik.http.middlewares.securedheaders.headers.BrowserXssFilter=true"
      - "traefik.http.middlewares.securedheaders.headers.STSIncludeSubdomains=true"
      - "traefik.http.middlewares.securedheaders.headers.STSSeconds=315360000"

    ports:
      - "80:80"
      - "443:443"
    networks:
      - proxy
    volumes:
      - ./data/acme.json:/etc/traefik/acme.json:rw
      #- /var/run/docker.sock:/var/run/docker.sock:ro
      - ./logs/traefik.log:/var/log/docker/traefik.log
      - ./data/traefik_conf.yaml:/traefik_conf.yaml
    environment:
      -<EMAIL_ADDRESS>      - CF_API_KEY=mycfkey

    restart: unless-stopped

  socket-proxy:
    image: tecnativa/docker-socket-proxy
    container_name: socket-proxy
    restart: unless-stopped
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      CONTAINERS: 1
    networks:
      - proxy

What am i missing here? Plz help :disappointed:

Hi,
It looks like you did not use the script to launch traefik. There is a script called ubuntu_1804_install_traefik.sh under deeztek-docker/Linux/traefik as outlined in the README.md.
The script does a lot of things for you including creating necessary directories and files, see below. The acme.json file get created before launching the container because if it's not there, docker will simply create a directory like you are experiencing now.
At least that's what you originally posted. Not sure what problem you are experiencing now because you edited your post.
echo "Creating pre-requisite directories files and setting permissions"
#Create necessary directories and files
/bin/mkdir -p /opt/traefik/data
/usr/bin/touch /opt/traefik/data/acme.json
/bin/chmod 600 /opt/traefik/data/acme.json
/bin/mkdir -p /opt/traefik/logs
/usr/bin/touch /opt/traefik/logs/traefik.log

Hi,
I have used script for creating traefik docker file and it did worked for letsencrypt but what i'm not able to work is with Cloudflare. When letsencrypt worked that's why i edited my post.
I get an error when i use Cloudflare SSL and shows this error in browser.
ERR_SSL_VERSION_OR_CIPHER_MISMATCH

I don't use cloudflare so I can't offer much help. Have you read through the docs?
https://go-acme.github.io/lego/dns/cloudflare/

I don't use cloudflare so I can't offer much help. Have you read through the docs?
https://go-acme.github.io/lego/dns/cloudflare/

The problem was in my Cloudflare SSL settings.
I'm able to find the solution by a Reddit user helped me but Thank you for your reply and i love all your scripts.


Added a chair intolerance to :massbring.

You can now only bring people. You can no longer bring their chairs.

Did not test these changes. Shit could quite possibly hit the fan. Regardless, there's no reason it should not work.

Two people have given a nod to a wall of fan-shit mayhem.

Let the chaos begin, Lord Bushman.

Looks like it should work so good enough for me but do please test in the nightly

This change broke the :ban command on Adonis. As such, I'm now marking this pull request as null and void. Please contact me thru the Adonis WeChat Group for more information.


Reduce noisy scheduler events

If we can't schedule pods because of resource requirements (i.e we've exceeded a limit range), the scheduler currently spews forth events on each iteration, for each pod, about these failures. This event includes all nodes, which in a large cluster leads to a lot of POST load and a lot of noise in the events.
Creating pods in the default ns with the e2e cluster we'll hit this case with > 10 pods per node. Modifed to (open to suggestions):
Events:
  FirstSeen				LastSeen			Count	From		SubobjectPath	Reason			Message
  Fri, 05 Jun 2015 14:40:06 -0700	Fri, 05 Jun 2015 14:40:37 -0700	6	{scheduler }			failedScheduling	Error scheduling: Pod foo failed fitness predicates PodFitsResources. See logs for details.

Splendid! LGTM!  cc @davidopp @lavalamp in case they want to review.

Yeah I'm not sure who uses this particular event except someone who works on scheduler (and I don't use it), @lavalamp do you think it adds value/suggest modification?

Reworded according to suggestion, should be good when shippable is green i guess

LGTM


ELK - Kibana: 502 Bad Gateway from Mantl UI

ansible 1.9.6
Python 2.7.5
git status# HEAD detached at 1.1.0
Cloud Environament: Openstack on Prem (Juno, via Fuel Mirantis 6.1.1)
Terraform v0.6.14
Kibana as part of Mantl add on elk does not launch. In Messos shows it fails multiple times.
Running kibana from external docker connects to Eleastic search and see logs.
Assumption: Launch script pointing to incorrect elastic search location on port 4000 missing mantl as not found in consul without '-mantl':
--elasticsearch.http=http://elasticsearch-executor.service.consul:4000 , should be elasticsearch-executor-mantl.service.consul
Attached stderr and stdout from Mesos sandbox.
stderr.txt
stdout.txt
[root@mantl-control-01 ~]# ping elasticsearch-executor.service.consul
ping: unknown host elasticsearch-executor.service.consul
[root@mantl-control-01 ~]# ping elasticsearch-executor.service.consul
ping: unknown host elasticsearch-executor.service.consul
[root@mantl-control-01 ~]# ping elasticsearch-executor-mantl.service.consul
PING elasticsearch-executor-mantl.service.consul (10.XX.XX.XX) 56(84) bytes of data.
64 bytes from mantlhgl4-worker-003.novalocal (10.XX.XX.XX): icmp_seq=1 ttl=64 time=1

@ygoodmn thanks for the report. Can you share the command you used to install the addon? Did you have to do any kind of uninstall / reinstall process?
The default configuration  of the Kibana addon doesn't actually use the elasticsearch.http setting. Instead, it uses the value of the ELASTICSEARCH_SERVICE to discover the correct host and port from the service registered consul. See
https://github.com/CiscoCloud/mantl/blob/1.1.0/roles/elk/templates/kibana-mesos.json.j2#L16
https://github.com/CiscoCloud/mantl/blob/1.1.0/roles/elk/defaults/main.yml#L18 + line 42
It is confusing that elasticsearch.http is set to a different value so I created an issue to fix that (#1545). But, I'm not sure it is the problem in this case. In the stderr logs, I see
I0613 07:56:10.746350    36 sched.cpp:1024] Got error 'Framework has been removed'
I0613 07:56:10.746384    36 sched.cpp:1805] Asked to abort the driver
I0613 07:56:10.747728    36 sched.cpp:1070] Aborting framework '28a367d2-6391-47d7-989a-34a67fb8b19d-0003'

It almost looks like there is a conflict with the kibana framework within mesos (this is why I am wondering if you had to uninstall / reinstall - perhaps something didn't get cleaned up correctly). Are you able to uninstall the addon on your cluster (https://github.com/CiscoCloud/mantl/blob/1.1.0/roles/elk/README.rst#uninstalling-the-elk-addon)? If so, after you uninstall and before reinstalling, can you verify that the kibana framework is no longer registered in Mesos (you can check the frameworks tab in the Mesos UI)?

Used the addon command as document, and did not re-install. This happened for 2 separate mantl installs, so as it was reproducible I entered it here.

I can confirm this issue, happened to me too. Many times.

It resolves for me in version 1.2.0 of Mantl.


engines field at package.json

Right now it is set to ^5.0.0, is there any concerns why it might not work on other versions?

This can probably be lowered to 4.0 if we take care that there is no destructuring happening anywhere.

ha-ha, I mean the opposite :), right now there is couple projects working with 6.x & 7.x

Oh! Right. Yeah, I don't see any problems changing it to >= or whatever!

SGTM

fyi, because of that we can't use yarn (there is probably some workaround though) it seems it's stricter then npm

Fixed in 7faa863a3e19cbebf46b0415dc6e81f40a2c4a25