Error only when load HLS

Have you read the FAQ and checked for duplicate open issues?
Yes
What version of Shaka Player are you using?
3.3.2
Can you reproduce the issue with our latest release version?
yes
Can you reproduce the issue with the latest code from main?
yes
Are you using the demo app or your own custom app?
both
If custom app, can you reproduce the issue using our demo app?
yes
What browser and OS are you using?
windows, macOs, chrome, firefox & edge
What did you do?

Load HLS manifest
What did you expect to happen?
Shaka should load and play HLS manifest
What actually happened?
Error Code : 1002
Error message: "TypeError: Failed to fetch\n    at $jscomp.generator.Engine_.program_ (https://*/dist/shaka-player-).
If I load HLS manifest directly from video player such as VLC it works fine but not worked when load from Shaka player.
DASH and CMAF not effected. the problem only happen on HLS.

Looks like a CORS error.  Are you loading content from another domain/origin, other than where your web player is hosted?  If so, the server must explicitly allow access.  See https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS for details.
If it is in fact a CORS error, you will see logs in the JS console of the browser indicating this.  These are generated by the browser and aren't visible in JavaScript, so Shaka Player can't directly detect CORS issues and explicitly name them in an error code.

Thank you for response @joeyparrish .
I've had fixing CORS problem and revalidate the caching and now have new HEAD failed request.
There is a console message : Unable to find byte-order-mark, making an educated guess.
error.code : 1001
error.data[1] : 404
I've also done HLS endpoint testing with VLC player and it's works

Looks like an HTTP 404 error (not found).  Are you sure there isn't a typo?
Does your server support HEAD requests?

Looks like an HTTP 404 error (not found). Are you sure there isn't a typo?
Does your server support HEAD requests?

Yes, the server support HEAD request.
I've had test it with another player to confirm and it works.
also HLS endpoint located on same server where CMAF dan DASH endpoint reside so it's have an identical server configuration

@bram-abe Can you check with 4.0.0? I think the problem could have been solved.


Update MD file on new branch 2017-27-01_07-12-04

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

split/updateTestCon.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


It is not clear what happens after Managed Identities are enabled on a Cluster

Will the Identity icon appear in the Cluster's properties in Azure Portal?  If it appears, can the associated Managed Identities then be used by all the Services and Applications running on the Cluster?  I am brand new to this topic and do see that App Services, Azure Functions, etc. display the Identity icon and I can associate Managed Identities there with service instances.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 59a2784c-c4b4-f341-80f5-5d7746a93aa2
Version Independent ID: fe703183-80c9-36a7-25f8-cf2ab03bd45e
Content: Azure Service Fabric - Configure an existing Azure Service Fabric cluster to enable managed identity support
Content Source: articles/service-fabric/configure-existing-cluster-enable-managed-identity-token-service.md
Service: service-fabric
GitHub Login: @athinanthny
Microsoft Alias: atsenthi

Thanks for the feedback! We are currently investigating and will update you shortly.

@eugen-nw there is not portal support for Managed Identifies with Service Fabric. All operations need to be performed by updating via template. When it comes to Service Fabric the simplest and most effective way to manage a cluster both on prem and in Azure is via template. The portal has limited options on what can be achieved.

ARM templates are one way to go about it.  Accidentally, we've never had the need to use one to deploy a SF cluster and we've been in this business for years.  We configure Clusters using the Portal and deploy Applications and Services using Azure DevOps release pipelines.  ARM templates could be simplest when you have in house knowledge and experience using them.  For me so far most samples I've encountered were incomplete, so I was wasting time fixing them instead of getting my work done.

DevOps piplines are another good way to achieve deployment too. It is similar to the template style. I am not overly skilled in DevOps but many users find that way of deploying to be ideal.
Deploying a cluster via the portal is a fine way to. But as far as configuring and updating the cluster you are quite limited if using only portal

Fair enough. I would suggest filing a feature request here:
https://feedback.azure.com/forums/293901-service-fabric
That way others can view it and upvote if they think it would be useful and the product team can prioritize the ask.

Thank you.  There's already one:
https://feedback.azure.com/forums/293901-service-fabric/suggestions/38639077-don-t-require-arm-deployment-to-use-managed-identi


Cannot import/construct normal Java object

new java.lang.StringBuffer()
                          ^~~~ Parse Error

new java.util.HashMap<Int, Int>()
                     ^~~~ Parse Error. Does not support generic in Java style.

And there is no way to import Java packages.

In the first example, to workaround, try
new java.lang.StringBuffer( )

This is a known issue #97
We won't support generic Java interoperation natively. One way to achieve the similar effect is to wrap it into non-generic method call in your own Java file, and let f2j compiler know it.

@bixuanzju the workaround seems magic hack... could it be fixed?

@zonyitoo That's because we also have unit as (), the parser will get confused.
I suggest you to use this hack for now, when it get fixed you could simply replace all ( ) with () in your code.

@lihuanglx ah, i see. thanks.


Error in IDialog.Wait

Hi,
please see the below code:
`private async Task MessageReceivedAsync(IDialogContext context, IAwaitable result)
{
var message = await result;
        if (message.Text.ToLower() == "hi")
        {
            context.Wait(AfterHiAsync);
        }
        else if (message.Text.ToLower() == "reset")
        {
            context.Wait(AfterResetAsync(context,result));
        }
    }`

I get an error in the line:                  context.Wait(AfterResetAsync(context,result));
The type arguments for method 'IDialogStack.Wait(ResumeAfter)' cannot be inferred from the usage. Try specifying the arguments explicitly.
Please help what am I doing wrong?

The second line should be something like this:
context.Wait(AfterResetAsync);
You should pass the Wait method a delegate representing the next method you want to execute after the user's message, not a result from calling that method.


Files freezes when right-clicking files/folders

Describe the bug
Files freeze when right-clicking files/folders and sometimes it crashes. And if this happens, closing and reopen Files will showing the Home tab instead of continuing where you left off.
To Reproduce
Steps to reproduce the behavior:

Open Files
Right click on a folder/files
Files freeze/crashes (it's more frequent to freeze rather than crashes)

Expected behavior
Files doesn't freeze or crash
Screenshots
https://user-images.githubusercontent.com/61906684/110193845-05dd7f00-7e69-11eb-9fff-0a1271564aad.mp4
Desktop (please complete the following information):

OS Version: Microsoft Windows 10 Home Single Language 10.0.18363 N/A Build 18363
App version: <IP_ADDRESS>

Additional context
Add any other context about the problem here. Does this problem occur again after restarting the app? Yes
Log file
debug.txt
debug_fulltrust.txt

Is there any difference if you wait 1 sseconds before right clicking?

Is there any difference if you wait 10 seconds before right clicking? (until a few more items appear on the sidebar)

No, no difference at all.
https://user-images.githubusercontent.com/61906684/110195919-77222f80-7e73-11eb-8243-546aec18e2f6.mp4

same, first time using the app

@kimlimjustin could you try the following?

create an empty folder
set Files to open this folder on startup
check if right-click is working

check if right-click is working (in the folder empty space)

No, it's not working and resulting Files freeze.

do shortcut files (.lnk) appear for you in Files?

Sometimes I need to wait a long time to let it appears in Files, but right-clicking also not working too after it appears. But sometimes although I waited for a long time, it still doesn't appear, IDK why tho.

have you had this issue with previous versions of Files?

I remember version 1.1.0 had no this bug, but I'm not really sure (I know it's 1.1.0 because I opened a bug soon after trying Files and I check back that version). The thing I can sure of is the bug occurs recently.

@kimlimjustin could you try the following?

create an empty folder
open Files
navigate to the folder using the navigationbar
check if right-click is working (in the folder empty space)

Also:

do shortcut files (.lnk) appear for you in Files?
have you had this issue with previous versions of Files?

1.context does not work on empty folders either
2.i created a shortcut for a folder(quick access then changed it to wt.exe) and when entering the folder with "files" it gets stuck on loading as shown:

I have this issue too. Different folders have different right click menu appear times. The time to wait is also consistent for each folder.

Horribly slow right click menu times for me as well.

I have this issue too. Different folders have different right click menu appear times. The time to wait is also consistent for each folder.

Horribly slow right click menu times for me as well.

It's more frequent to be freeze in my case which is even worse xd.

I also have this issue

I also have the same issue. Left-clicking navigates properly, but as soon as I right-click it just gets stuck. First time user of the app btw.

Reporting back here after #4033 has been merged, and the right-click issue is fixed with me. Thanks for the quick fix!

It's fixed on mine since version <IP_ADDRESS> released, so I'm going to close this issue, thanks for the quick fix!

I'm still getting the exact same issue, with version <IP_ADDRESS>, windows 20H2

This issue should be resolved with #4625.


Set up Oauth Authentication with BitBucket

I am running minnaker in AWs EC2.
Basically i have followed steps from https://www.spinnaker.io/setup/security/authentication/oauth/#bring-your-own-provider
In the step to "manually set the redirect_uri for Gate"
hal config security authn oauth2 edit --pre-established-redirect-uri https://my-real-gate-address.com:8084/login
is my-real-gate-address.com my public IP??
Here is detail to my configs:
hal config
name: default
version: 1.18.7
providers:
  appengine:
    enabled: false
    accounts: []
  aws:
    enabled: true
    accounts:
    - name: deref-master-managing
      requiredGroupMembership: []
      providerVersion: V1
      permissions: {}
      accountId: 'SECRET'
      regions:
      - name: us-east-1
      - name: ap-southeast-1
      - name: ap-southeast-2
      assumeRole: role/SpinnakerManagedRoleAccount
      lifecycleHooks: []
    - name: deref-systest
      requiredGroupMembership: []
      providerVersion: V1
      permissions: {}
      accountId: 'SECRET'
      regions:
      - name: us-east-1
      - name: ap-southeast-1
      - name: ap-southeast-2
      assumeRole: role/SpinnakerManagedRoleAccount
      lifecycleHooks: []
    - name: deref-sectest
      requiredGroupMembership: []
      providerVersion: V1
      permissions: {}
      accountId: 'SECRET'
      regions:
      - name: us-east-1
      - name: ap-southeast-1
      - name: ap-southeast-2
      assumeRole: role/SpinnakerManagedRoleAccount
      lifecycleHooks: []
    primaryAccount: deref-master-managing
    bakeryDefaults:
      baseImages: []
    accessKeyId: SECRET
    secretAccessKey: SECRET/SECRET+SECRET/SECRET
    defaultKeyPairTemplate: '{{name}}-keypair'
    defaultRegions:
    - name: us-west-2
    defaults:
      iamRole: BaseIAMRole
  ecs:
    enabled: false
    accounts: []
  azure:
    enabled: false
    accounts: []
    bakeryDefaults:
      templateFile: azure-linux.json
      baseImages: []
  dcos:
    enabled: false
    accounts: []
    clusters: []
  dockerRegistry:
    enabled: false
    accounts: []
  google:
    enabled: false
    accounts: []
    bakeryDefaults:
      templateFile: gce.json
      baseImages: []
      zone: us-central1-f
      network: default
      useInternalIp: false
  huaweicloud:
    enabled: false
    accounts: []
    bakeryDefaults:
      baseImages: []
  kubernetes:
    enabled: true
    accounts:
    - name: spinnaker
      requiredGroupMembership: []
      providerVersion: V2
      permissions: {}
      dockerRegistries: []
      serviceAccount: true
      cacheThreads: 1
      namespaces: []
      omitNamespaces: []
      kinds: []
      omitKinds: []
      customResources: []
      cachingPolicies: []
      onlySpinnakerManaged: true
    primaryAccount: spinnaker
  oracle:
    enabled: false
    accounts: []
    bakeryDefaults:
      templateFile: oci.json
      baseImages: []
  cloudfoundry:
    enabled: false
    accounts: []
deploymentEnvironment:
  size: SMALL
  type: Distributed
  accountName: spinnaker
  imageVariant: SLIM
  updateVersions: true
  consul:
    enabled: false
  vault:
    enabled: false
  location: spinnaker
  customSizing: {}
  sidecars: {}
  initContainers: {}
  hostAliases: {}
  affinity: {}
  tolerations: {}
  nodeSelectors: {}
  gitConfig:
    upstreamUser: spinnaker
  livenessProbeConfig:
    enabled: false
  haServices:
    clouddriver:
      enabled: false
      disableClouddriverRoDeck: false
    echo:
      enabled: false
persistentStorage:
  persistentStoreType: s3
  azs: {}
  gcs:
    rootFolder: front50
  redis: {}
  s3:
    bucket: spinnaker
    rootFolder: front50
    pathStyleAccess: true
    endpoint: http://minio.spinnaker:9000
    accessKeyId: minio
    secretAccessKey: SECRET+SECRET/SECRET
  oracle: {}
features:
  auth: false
  fiat: false
  chaos: false
  entityTags: false
  artifacts: true
metricStores:
  datadog:
    enabled: false
    tags: []
  prometheus:
    enabled: false
    add_source_metalabels: true
  stackdriver:
    enabled: false
  newrelic:
    enabled: false
    tags: []
  period: 30
  enabled: false
notifications:
  slack:
    enabled: false
  twilio:
    enabled: false
    baseUrl: https://api.twilio.com/
  github-status:
    enabled: false
timezone: America/Los_Angeles
ci:
  jenkins:
    enabled: false
    masters: []
  travis:
    enabled: false
    masters: []
  wercker:
    enabled: false
    masters: []
  concourse:
    enabled: false
    masters: []
  gcb:
    enabled: false
    accounts: []
repository:
  artifactory:
    enabled: false
    searches: []
security:
  apiSecurity:
    ssl:
      enabled: false
    overrideBaseUrl: https://SECRETPUBLIC_IP/api/v1
  uiSecurity:
    ssl:
      enabled: false
    overrideBaseUrl: https://SECRETPUBLIC_IP
  authn:
    oauth2:
      enabled: true
      client:
        clientId: SECRET
        clientSecret: SECRET
        useCurrentUri: false
      resource: {}
      userInfoMapping: {}
    saml:
      enabled: false
      userAttributeMapping: {}
    ldap:
      enabled: false
    x509:
      enabled: false
    iap:
      enabled: false
    enabled: true
  authz:
    groupMembership:
      service: EXTERNAL
      google:
        roleProviderType: GOOGLE
      github:
        roleProviderType: GITHUB
      file:
        roleProviderType: FILE
      ldap:
        roleProviderType: LDAP
    enabled: false
artifacts:
  bitbucket:
    enabled: false
    accounts: []
  gcs:
    enabled: false
    accounts: []
  oracle:
    enabled: false
    accounts: []
  github:
    enabled: false
    accounts: []
  gitlab:
    enabled: false
    accounts: []
  gitrepo:
    enabled: false
    accounts: []
  http:
    enabled: true
    accounts: []
  helm:
    enabled: false
    accounts: []
  s3:
    enabled: false
    accounts: []
  maven:
    enabled: false
    accounts: []
  templates: []
pubsub:
  enabled: false
  google:
    enabled: false
    pubsubType: GOOGLE
    subscriptions: []
    publishers: []
canary:
  enabled: false
  serviceIntegrations:
  - name: google
    enabled: false
    accounts: []
    gcsEnabled: false
    stackdriverEnabled: false
  - name: prometheus
    enabled: false
    accounts: []
  - name: datadog
    enabled: false
    accounts: []
  - name: signalfx
    enabled: false
    accounts: []
  - name: aws
    enabled: false
    accounts: []
    s3Enabled: false
  - name: newrelic
    enabled: false
    accounts: []
  reduxLoggerEnabled: true
  defaultJudge: NetflixACAJudge-v1.0
  stagesEnabled: true
  templatesEnabled: true
  showAllConfigsEnabled: true
plugins:
  plugins: []
  enabled: false
  downloadingEnabled: false
  pluginConfigurations:
    plugins: {}
webhook:
  trust:
    enabled: false
telemetry:
  enabled: true
  endpoint: https://stats.spinnaker.io
  instanceId: SECRET
  connectionTimeoutMillis: 3000
  readTimeoutMillis: 5000

~/.hal/default/profiles/settings-local.js
window.spinnakerSettings.feature.artifactsRewrite = true;

~/.hal/default/profiles/gate-local.yml
server:
  servlet:
    context-path: /api/v1
  tomcat:
    protocolHeader: X-Forwarded-Proto
    remoteIpHeader: X-Forwarded-For
    internalProxies: .*
    httpsServerPort: X-Forwarded-Port

security:
  basicform:
    enabled: false
  oauth2:
    client:
      clientId:
      clientSecret:
      userAuthorizationUri: https://bitbucket.org/site/oauth2/authorize
      accessTokenUri: https://bitbucket.org/site/oauth2/access_token
      scope: ""
    resource:
      userInfoUri: https://api.bitbucket.org/2.0/user
    userInfoMapping: # Used to map the userInfo response to our User
      email: email
      username: username

But interestingly enough take a look at the gate.yml inside the Gate Pod
It has BASIC security still enabled. how do i disable it?
/opt/spinnaker/config/gate.yml
## WARNING
## This file was autogenerated, and _will_ be overwritten by Halyard.
## Any edits you make here _will_ be lost.

spectator:
  applicationName: ${spring.application.name}
  webEndpoint:
    enabled: false

server:
  ssl:
    enabled: false
  port: '8084'
  address: <IP_ADDRESS>
security:
  basic:
    enabled: true
  user: {}
  oauth2:
    enabled: true
    client:
      clientId: SECRET
      clientSecret: SECRET
      useCurrentUri: false
    resource: {}
    userInfoMapping: {}
cors: {}
google: {}

integrations:
  gremlin:
    enabled: false
    baseUrl: https://api.gremlin.com/v1

# halconfig

redis:
  connection: ${services.redis.baseUrl:redis://localhost:6379}

In Spinnaker UI it always shows as anonymous! Please help

I have an update on this:
Thank God Bitbucket Oauth is working now in my 2 Minnaker instances. Here is what i did:

Created a new EC2 instance in Sydney c5.xlarge and deployed minnaker latest from github. I notice the new minnaker has a mariadb pod
checked the EC2 instance type and system is using only 44% memory which is alright by me
set up Oauth
Disabled basic auth with user name and pass

I found 2 distinct set of files Ubuntu system spinnaker config files and Halyard Spinnaker config files I modified both sets of files whenever required
For Ubuntu system spinnaker config files
1. vim /etc/spinnaker/templates/profiles/front50-local.yml
spinnaker.s3.versioning: false
2. vim /etc/spinnaker/templates/profiles/settings-local.js
window.spinnakerSettings.feature.kustomizeEnabled = true;
3. vim /etc/spinnaker/templates/profiles/gate-local.yml
server:
  servlet:
    context-path: /api/v1
  tomcat:
    protocolHeader: X-Forwarded-Proto
    remoteIpHeader: X-Forwarded-For
    internalProxies: .*
    httpsServerPort: X-Forwarded-Port

security:
  oauth2:
    client:
      userAuthorizationUri: https://bitbucket.org/site/oauth2/authorize
      accessTokenUri: https://bitbucket.org/site/oauth2/access_token
      scope: ""
    resource:
      userInfoUri: https://api.bitbucket.org/2.0/user
    userInfoMapping: # Used to map the userInfo response to our User
      email: email
      username: username

4. vim /etc/spinnaker/templates/service-settings/gate.yml
healthEndpoint: /api/v1/health
For Halyard Spinnaker config files (inside Halyard Pod)
1. vi ~/.hal/default/profiles/gate-local.yml
server:
  servlet:
    context-path: /api/v1
  tomcat:
    protocolHeader: X-Forwarded-Proto
    remoteIpHeader: X-Forwarded-For
    internalProxies: .*
    httpsServerPort: X-Forwarded-Port
security:
  oauth2:
    client:
      userAuthorizationUri: https://bitbucket.org/site/oauth2/authorize
      accessTokenUri: https://bitbucket.org/site/oauth2/access_token
      scope: ""
    resource:
      userInfoUri: https://api.bitbucket.org/2.0/user
    userInfoMapping: # Used to map the userInfo response to our User
      email: email
      username: username

The issue has been resolved in the OSS project. Refer to my comment above for more details.

The issue has been resolved in the OSS project. Refer to my comment above for more details.


Alter body retrieval to '=== undefined' for response-provider checks

This is a proposed resolution to:
https://github.com/RuntimeTools/chai-openapi-response-validator/issues/49
The (this.res.body) falsey check is failing for potentially valid falsey values (false, null), returning 'undefined' for non-axios request providers.

Reviewing now :)


PERF-#5247: Decrease memory consumption for MultiIndex

Signed-off-by: Dmitry Chigarev<EMAIL_ADDRESS>
What do these changes do?
This change should decrease memory consumption for dataframes that use MultiIndex (see this comment).

[x] first commit message and PR title follow format outlined here

NOTE:  If you edit the PR title to match this format, you need to add another commit (even if it's empty) or amend your last commit for the CI job that checks the PR title to pick up the new PR title.

 What I was measuring is the time for splitting a MultiIndex into pieces and then restoring an original index from these pieces with and without the .remove_unused_levels method applied.
I was measuring with and without putting the data into Ray. When Ray was not involved I've seen performance degradation only at the stage of splitting the index (because an extra step of removing unused levels was applied), however, there were zero differences in concatenating pieces with and without unused levels being dropped.
When measuring with Ray involved there was also a slight slowdown at the splitting stage that was although eliminated with increasing the lengths of the values stored in the index (str_len):

just in case you're interested, here's the table that includes testing without ray:
 complete table with the results

script I used to measure all this
import pandas as pd
import numpy as np
from timeit import default_timer as timer

from modin.core.execution.ray.common.utils import initialize_ray

import ray
initialize_ray()

import uuid

def get_random_str(length):
    res = str(uuid.uuid1())
    return res[:length] if len(res) <= length else res * (length // len(res))

def get_level_values(level_len, str_len):
    return [get_random_str(str_len) for _ in range(level_len)]

def get_mi(idx_len, nlevels, str_len, is_highly_unique):
    if is_highly_unique:
        levels = [get_level_values(idx_len, str_len) for _ in range(nlevels)]
        return pd.MultiIndex.from_arrays(levels)
    else:
        unique_rate = 0.2
        levels = [
            np.random.choice(get_level_values(int(idx_len * unique_rate), str_len), idx_len)
            for _ in range(nlevels)
        ]
        return pd.MultiIndex.from_arrays(levels)

def split_and_restore(mi, nparts=112, drop_levels=True, use_ray=False):
    print(f"running with {drop_levels=}")
    one_part = len(mi) // nparts
    refs = [None] * nparts

    t1 = timer()
    for idx in range(nparts):
        split = mi[idx * one_part : (idx + 1) * one_part]
        if drop_levels:
            split = split.remove_unused_levels()

        if use_ray:
            refs[idx] = ray.put(split)
        else:
            refs[idx] = split

    splitting_time = timer() - t1
    print("\tsplitting took", splitting_time)

    t1 = timer()
    if use_ray:
        refs = ray.get(refs)
    refs[0].append(refs[1:])
    restoring_time = timer() - t1
    print("\trestoring took", restoring_time)
    return splitting_time, restoring_time

idx_lens = [
    100_000,
    1_000_000,
    10_000_000
]
nlevels = [3]
str_lens = [10, 100]

result = pd.DataFrame(
    index=np.array([(f"{x} split", f"{x} concat") for x in idx_lens]).flatten(),
    columns=pd.MultiIndex.from_product(
        [[0, 1], ["sparsed", "dense"], str_lens, [0, 1]],
        names=["use ray", "data density", "str_len", "remove unused levels"]
    )
)

for use_ray in [False, True]:
    print(f"========== USE RAY={use_ray} ===========")
    for idx_len in idx_lens:
        for nlevel in nlevels:
            for str_len in str_lens:
                print(f"==== {idx_len=} {nlevel=} {str_len=} sparsed data ====")
                key = (idx_len, nlevel, str_len, "sparsed")
                mi = get_mi(idx_len, nlevels=nlevel, str_len=str_len, is_highly_unique=False)

                print("index done")
                s_time, c_time = split_and_restore(mi, use_ray=use_ray)
                result.loc[f"{idx_len} split", (int(use_ray), "sparsed", str_len, 1)] = s_time
                result.loc[f"{idx_len} concat", (int(use_ray), "sparsed", str_len, 1)] = c_time

                s_time, c_time = split_and_restore(mi, drop_levels=False, use_ray=use_ray)
                result.loc[f"{idx_len} split", (int(use_ray), "sparsed", str_len, 0)] = s_time
                result.loc[f"{idx_len} concat", (int(use_ray), "sparsed", str_len, 0)] = c_time

                print(f"==== {idx_len=} {nlevel=} {str_len=} higly unique data ====")
                key = (idx_len, nlevel, str_len, "dense")
                mi = get_mi(idx_len, nlevels=nlevel, str_len=str_len, is_highly_unique=True)

                print("index done")
                s_time, c_time = split_and_restore(mi, use_ray=use_ray)
                result.loc[f"{idx_len} split", (int(use_ray), "dense", str_len, 1)] = s_time
                result.loc[f"{idx_len} concat", (int(use_ray), "dense", str_len, 1)] = c_time

                s_time, c_time = split_and_restore(mi, drop_levels=False, use_ray=use_ray)
                result.loc[f"{idx_len} split", (int(use_ray), "dense", str_len, 0)] = s_time
                result.loc[f"{idx_len} concat", (int(use_ray), "dense", str_len, 0)] = c_time

                result.to_excel("mi_res.xlsx") # store the result in case OOM occurs at further iterations

@dchigarev Great results, I see no reason not to add this change. However, for the sake of completeness, I suggest making the same table for nparts=16 (closer situation for the user) if you don't mind.

I suggest making the same table for nparts=16 (closer situation for the user)

good point.
Here are the measurements for the case of the lower number of splits, overall the trend stayed the same:
time for num_splits=16

I've also run memory consumption tests, measuring ray memory before and after putting the index pieces into Ray storage:
memory consumption tests

It showed that removing unused levels has only a noticeable benefit in cases of large strings, otherwise, memory consumption reduction is not significant but we still lose time in doing this extra step of removing levels.
So I guess we should need some kind of a switch between removing or not removing these unused levels, although I don't know yet what the condition could be.

It showed that removing unused levels has only a noticeable benefit in cases of large strings, otherwise, memory consumption reduction is not significant but we still lose time in doing this extra step of removing levels.
So I guess we should need some kind of a switch between removing or not removing these unused levels, although I don't know yet what the condition could be.

The biggest slowdown is about 0.2 seconds (when using also serialization, this is our main case for the pandas backend), while we save thousands of megabytes and avoid out of memory cases, which can happen very often when using a multi-index on machines where there is RAM<100GB. Considering also that Modin aims to run workloads with 100 GB and larger datasets, this looks like a great deal.
P.S. it is also possible that there is room for performance improvement in remove_unused_levels function itself.
+1 to this change as it is


fix k8saudit rule EphemeralContainers Created

Signed-off-by: Hi120ki<EMAIL_ADDRESS>
What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area plugins

/area registry

/area build

/area documentation

What this PR does / why we need it:
I'm using kubernetes on GKE, and deploy k8saudit based on admission controller.
In k8saudit detection, a rule EphemeralContainers Created is not works because of kmodify condition.
kmodify condition is
- macro: kmodify
  condition: (ka.verb in (create,update,patch))

But when creating ephemeral container by kubectl debug command, ka.verb is null.

When create Pod, kubernetes audit log records methodName: "io.k8s.core.v1.pods.create" and generate ka.verb: create.
But when creating ephemeral container by kubectl debug command, kubernetes audit log records methodName: "io.k8s.core.v1.pods.ephemeralcontainers.patch" and ka.verb is null.

So, I delete condition kmodify and it works well.
I'm waiting for your comments on my proposed rules.
Thank you.
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: hi120ki
Once this PR has been reviewed and has the lgtm label, please assign leogr for approval by writing /assign @leogr in a comment. For more information see:The Kubernetes Code Review Process.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Hi @hi120ki. Thanks for your PR.
I'm waiting for a falcosecurity member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
Once the patch is verified, the new status will be reflected by the ok-to-test label.
I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/ok-to-test

cc @darryk10

Hi @hi120ki
Have you tried with self-managed clusters? this might be GKE dealing with k8audit logs in a different way.
Could you please test this in different environments?
Thanks

Hello!
I tested the EphemeralContainers created rule in a couple of K8s clusters deployed with kOps or kubeadm. The rule triggered as expected employing the kmodify macro.
So as @darryk10 said, the problem might be related to the fact that GKE audit logs format is different.

I see. Thank you for inspection. I understand this issue is related to the GKE audit logs format.
Can I close this issue (no merge) because of gke specification?

Hi @hi120ki, since the plugin is related to k8saudit i think we should keep the rule as it is.
In case there would be a specific plugin for GKE this change would come useful so thanks for your contribution :)

Can I close this PR (no merge) because of gke specification?

:+1:
We will probably need a dedicated flavor of the k8saudit plugin for GKE (ie. k8saudit-gke)

Thank you for your great support.
I would be happy to contribute when k8saudit-gke is released!


Website Down !!! Please fix the Server Issue

Description

Documentation version

Hello @katareayush, website is working for me as usual.
Can you provide more information about issues you are encountering? Do you see any console errors? What's your browser is rendering on visit attempt?

[image: image.png]
This is the scenario from my side , I am unable to access it from 2 days.
On Sun, Jun 30, 2024 at 5:14 PM Bartosz Kaszubowski <
@.***> wrote:

Hello @katareayush https://github.com/katareayush, website is working
for me as usual.
Can you provide more information about issues you are encountering? Do you
see any console errors? What's your browser is rendering on visit attempt?
—
Reply to this email directly, view it on GitHub
https://github.com/facebook/react-native-website/issues/4153#issuecomment-2198533661,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BCQZNB2WOA6CKWGCQ5GM4P3ZJ7VR3AVCNFSM6AAAAABKEADJZ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJYGUZTGNRWGE
.
You are receiving this because you were mentioned.Message ID:
@.***>


test-branch/execute-job/quoted-authorization/2267747d-157f-47f8-9447-9b976e535883

PR for testing a7ed86d17d3ce4f29819e9a52edf1863b42da5a2

/canonical/self-hosted-runners/run-workflows a7ed86d17d3ce4f29819e9a52edf1863b42da5a2


Association one-to-many does not force class of the link

Association works, but the Linked_Class is not defined.
As discussed with @dmarcelino on Gitter, a test is written for the one-to-one relation, but not for the one-to-many.
I will do a PR soon.

Uh, somehow, it worked ... I suspect the console to give me hard times while web interface just works (deleting the class by console might not have really dropped it or something).
Anyway, after playing with tests, I can't find something wrong.

I see, if you've got a working test would you mind doing a PR? It will be good too prevent regressions in the future.

just did, you were quicker than me :)

Thanks!


Addition of message in user management

Describe the task
In the current implementation, there is no message displayed to the user in the following scenarios

User role is updated
User is removed
Permission related user is updated

Please refer screenshot for user management
User role is updated by Manage user roles
User is removed by Remove user
User permissions are updated in the Roles tab

Expected behavior
Toast messages should be available to the user in the following scenarios

User role is updated
Message: User role updated successfully
User is removed
Message: User removed successfully
Permission  related to the user is updated
Message: User  permissions updated successfully

Please refer implementation of the toast message done while adding a new client and updating the client details.
Refer to screenshot of toast message available on adding a new client

This issue PR is merged. Closing it.


refactor: replace for loop with iter

Hey,
A simple (and quite unexpected) perf improvement, just replacing a for loop with iter
before
- 5       | Simple source test with resampling (Mono)                                | 77            | 1558.4x               | 120
- 6       | Simple source test with resampling (Stereo)                              | 102           | 1176.5x               | 120
- 7       | Simple source test with resampling (Stereo and positionnal)              | 148           | 810.8x                | 120

after
- 5       | Simple source test with resampling (Mono)                                | 63.559        | 1888.0x               | 120
- 6       | Simple source test with resampling (Stereo)                              | 85.878        | 1397.3x               | 120
- 7       | Simple source test with resampling (Stereo and positionnal)              | 134.003       | 895.5x                | 120

Hey, I cannot reproduce the performance impact. Are you sure about your numbers?
Also your main branch may be out of date because it does not show the sub-millisecond numbers.
Alternatively, you could add a benchmark for the iai benchmark suite to get the absolute numbers.
We really ought to start with #188 ... using criterion for the local benchmarks will reduce the noise

Hey, yup I was out of date indeed. Just merged everything, re-run the benches a few time, and yup on my side it's still quite better, particularly visible on Simple source test with resampling (Mono) e.g.:
before
- 5       | Simple source test with resampling (Mono)                                | 72.425        | 1656.9x               | 120

after
- 5       | Simple source test with resampling (Mono)                                | 63.129        | 1900.9x               | 120

Interesting. I will try to put this specific case in criterion and measure again

Hey, just tested locally compared to main... and indeed there are no differences... so weird...
Maybe let's just keep the criterion stuff instead of iai, so at least we can use it locally?

Hey sorry I have left this in a hurry. I'm actually looking to support both. Iai on CI and local criterion with the 'official' benchmarks. Will take a bit of time though

oh ok, sounds good. Let's close that then this is too blurry


feat add fastp

PR checklist

[x] Added fastp to perform trimming
[ ] If you've fixed a bug or added code that should be tested, add tests!
[ ] If you've added a new tool - have you followed the pipeline conventions in the contribution docs
[ ] If necessary, also make a PR on the nf-core/raredisease branch on the nf-core/test-datasets repository.
[ ] Make sure your code lints (nf-core lint).
[ ] Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
[ ] Ensure the test suite passes (nextflow run . -profile test_one_sample,docker --outdir <OUTDIR>).
[ ] Usage Documentation in docs/usage.md is updated.
[ ] Output Documentation in docs/output.md is updated.
[ ] CHANGELOG.md is updated.
[ ] README.md is updated (including new tool citations and authors/contributors).

According to what it says here the polyG should happen automatically:
# polyG tail trimming, useful for NextSeq/NovaSeq data   -g, --trim_poly_g                  force polyG tail trimming, by default trimming is automatically enabled for Illumina NextSeq/NovaSeq data