Update New-EmailAddressPolicy.md

Docs Build status updates of commit 299a9da:
:clock10: Pending: waiting for processors (13 builds ahead of you)
:warning: Docs Build is busy, currently there are 13 builds ahead of this one, for more information you can view the Build queue graph on the Docs Portal.

Docs Build status updates of commit 299a9da:
:clock10: Full build: the average full build time is 17 min(s) 54 sec(s), this is based on the last 1000 build(s) for this repository.

Docs Build status updates of commit 299a9da:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/New-EmailAddressPolicy.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel


Support Accelerate load_in_8bit

Hello! Love what you are doing here with dynamic batching, streaming, and tensor parallelism you have a ton of great functionality!
One thing I am wondering about is whether or not you support loading large (10+B) models using 8bit precision via accelerate? This is basically the only way I've been able to run these models on A10 GPUs and is basically the only barrier to me adopting this for my projects!
Would love to chat more to see if I could contribute here.

Nevermind, I am blind. I see that is what the quantize flag does! Can't wait to see the new flash llama using quantization! Is that coming soon?

Yeah is there a way to turn off the flash model? I can run llama 13b on an A10 if I use 8bit quantization and would love to get the dynamic batching from this system until you support quantization for the flash model so if there is a way to just use the normal LlamaForCausalLM so I can enable 8bit that would be awesome is this possible right now @OlivierDehaene


Removed some unneeded generated headers from the editor directory SCsub.

These headers are currently built in (and included from) the core directory.
Removing the editor directory versions to avoid confusion.

That was likely wrongly rebased in #17595 after the merge of #18899.


[hotfix] add coordinatorState into hashCode() to comply with equals().

What is the purpose of the change
add coordinatorState into hashCode() to comply with equals().
Brief change log
add coordinatorState into hashCode() to comply with equals().
Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? no
If yes, how is the feature documented? no

This fix is probably fine. I think it is okay as it was before, because the contract with equals is not wrong, just hashCode is less good in spreading values over a HashMap.
Not sure if there are any implications for this, because as far as I know equals() and hashCode() are not very well defined here and are mainly used for tests.

It is ok to leave it as before if the method mainly used for tests and feel free to close this pr then.


Server hardware implementation

Description
Implementation for the Server hardware resource
Issues Resolved

#31

Check List

[x] New functionality includes testing.

[x] All tests pass ($ rake test).
[x] New resources and/or actions have are included in the matchers.rb and matchers_spec.

[x] New functionality has been documented in the README if applicable.

[x] New functionality has been thoroughly documented in examples (please include helpful comments).

[x] Changes documented in the CHANGELOG.

:+1:  I'm good with this PR, but I think there is 1 thing that needs to be changed in the SDK: being able to retrieve and check for existence based on the hostname, not just name and URI. If you set the name of a server_hardware resource to something other than the hostname, it won't be able to retrieve it, so on every run, it'll try to re-add it (resulting in an error). Again, I think this is an SDK issue, not an issue with this cookbook.

Also, the Ruby 2.1 build is unhappy again because of another gem requiring Ruby 2.2. Not our problem, so I don't think we need to hold off on any merging because of it.


README  request

I think the README needs to add a reference to the movie data being used in this journey. In the intro, the first mention of it is in the phrase 'ratings data', which no one would associate with movies.
Can you expand paragraph 3 to include a mention of the movielens ratings data and what it contains? Maybe include a link to their website?

Yes good idea!


Can you please support mongodb?

Can you please add support for MongoDB as well ?

Hello @rekaparathalwar
mongoDB plugin is developed #1415
and being refactored for performance issues. #4884

Hi @RoySRose  Can you please let me know when it will be available for us to use please ?

Hello, @rekaparathalwar
Well. #4884 PR review is almost done and I think any further fix is necessary in close future.
So, after that. You can clone the master and use it.


Query: Include annotation is not ignored for group by aggregate causing client eval

Running a simple group by expression to compute a sum of a value is failing to translate and as a result, running the group by locally.
Query:
  var summedCharges = await dataContext.Charges
                .Include(i => i.BillingItem)
                .Where(i => !i.Deleted &&
                            i.UserId == userId &&
                            i.SchoolId == schoolId &&
                            i.BillingItem.SchoolYearId == schoolYearId &&
                            i.IncludeInTrp &&
                            i.BillingItem.TypeCode != trpBillingItemTypeCode

                )
                .GroupBy(i => i.UserId)
                .Select(g => g.Sum(i => i.Amount))
                .OrderBy(i=>i.Value)
                .FirstOrDefaultAsync();

Warning Messages:
[12:53:20 WRN] The LINQ expression 'GroupBy([i].UserId, _Include(queryContext, [i], new [] {[i.BillingItem]}, (queryContext, entity, included) => { ... }))' could not be translated and will be evaluated locally.
[12:53:20 WRN] The LINQ expression 'Sum()' could not be translated and will be evaluated locally.
[12:53:20 WRN] The LINQ expression 'orderby Convert({from ChargeEntity i in [g] select [i].Amount => Sum()}, Decimal) asc' could not be translated and will be evaluated locally.
[12:53:20 WRN] The LINQ expression 'Sum()' could not be translated and will be evaluated locally.
[12:53:20 WRN] The LINQ expression 'Sum()' could not be translated and will be evaluated locally.
[12:53:20 WRN] The LINQ expression 'Sum()' could not be translated and will be evaluated locally.
[12:53:20 WRN] The LINQ expression 'FirstOrDefault()' could not be translated and will be evaluated locally.

Further technical details
EF Core version: 2.1.0-rc1-final
Database Provider:  Microsoft.EntityFrameworkCore.SqlServer
Operating system:  Win 10
IDE: (e.g. Visual Studio 2017 15.4)

You are using Include which means you are asking for some entity hence SQL group by cannot be used. What is result you are trying to compute with the query? Rewriting the query differently could achieve server eval using SQL group by.

so I don't need to Include the child objects unless I want them in the projection?  I thought it would be required since I'm filtering on a child object's property (billingitem.schoolyearid in this example).

@cdarrigo - Include is only required when you are projecting out an entity and you want navigation on that entity object to be populated. If you are not projecting out entity then include will be ignored. All navigation access inside the query doesn't require any Include. We add appropriate joins automatically to compute results.
@maumar @anpete - Ideally this Include should be ignored because we are not projecting out entity - GroupBy Aggregate operation case. I believe our logic around GroupBy + Include is weak that we are adding Include here which causes client eval instead of server eval.

@smitpatel Sounds like it.

@smitpatel That was it. I removed the unnecessary include and its grouping on the server.  Thanks!

@smitpatel Is this a regression from 2.0?

@ajcvickers - Not a regression. We fail to detect that Include is unnecessary and put that in our QueryModel which does streaming group by instead of SQL group by translation.

fixed in 66d135fb3e2e8bb550bfad8119bb5a75cf70f04e


DATAJDBC-181 - Use NamingStrategy for instantiating entities.

The ResultSetParameterValueProvider uses the relevant property for a requested parameter in order to obtain the proper column name to use.
Improved mocking of ResultSets in tests to actually fail when a non-existing column gets requested instead of just returning null.

Resolved via https://github.com/spring-projects/spring-data-jdbc/commit/18bf6ca77c91335baf7be234d5516b8a5654c26d and https://github.com/spring-projects/spring-data-jdbc/commit/b49b7674f24d3e019b682fdaac18ed9d5cdd0a56


Extensions break without HOME environment variable

Describe the bug
When running without a set HOME environment variable (e.g in an Azure DevOps agent), the behaviour of az extension add -n azure-cli-ml and az ml in inconsistent. az extension add -n azure-cli-ml still manages to install the extension in the home directory, but az ml can't find it.
This issue is not easy to debug for the user!
Strangely, AZURE_EXTENSION_DIR, when set, is honoured by "az extension add" but not by "az ml".
Command Name
az ml
Errors:
az: 'ml' is not in the 'az' command group. See 'az --help'.

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

unset HOME (to simulate the environment of Azure DevOps agents)
az extension add -n azure-cli-ml (succeeds)
az ml (fails - UNEXPECTED)

az: 'ml' is not in the 'az' command group. See 'az --help'.

AZURE_EXTENSION_DIR=. az ml  (fails - UNEXPECTED)

az: 'ml' is not in the 'az' command group. See 'az --help'.

HOME=. az ml (succeeds - expected)

az ml: error: the following arguments are required: _subcommand
usage: az ml [-h]
             {computetarget,service,model,dataset,datastore,environment,experiment,folder,pipeline,run,workspace}

with --debug:
  File "/root/.azure/cliextensions/azure-cli-ml/azext_ml/__init__.py", line 29, in <module>
    load_common_help()
  File "/root/.azure/cliextensions/azure-cli-ml/azext_ml/__init__.py", line 24, in load_common_help
    from azureml._cli.aml_cli import load_azurecli_help
  File "/root/.azure/cliextensions/azure-cli-ml/azureml/_cli/aml_cli.py", line 11, in <module>
    from azureml._cli import abstract_subgroup
  File "/root/.azure/cliextensions/azure-cli-ml/azureml/_cli/abstract_subgroup.py", line 12, in <module>
    from azureml._cli.cli_command import CliCommand
  File "/root/.azure/cliextensions/azure-cli-ml/azureml/_cli/cli_command.py", line 8, in <module>
    from azureml._base_sdk_common.common import set_correlation_id
  File "/root/.azure/cliextensions/azure-cli-ml/azureml/_base_sdk_common/common.py", line 93, in <module>
    USER_PATH = os.path.join(os.getenv('HOME'), '.config')
  File "/opt/az/lib/python3.6/posixpath.py", line 80, in join
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

Expected Behavior
az ml should work. If this is not possible, then at least the az extension add command should fail with a clear error message that HOME is not set.
Environment Summary
Linux-4.15.0-1063-azure-x86_64-with-debian-stretch-sid
Python 3.6.5
Shell: bash

azure-cli 2.0.76
azure-cli-ml 1.0.74

Extensions:
azure-cli-ml 1.0.74

Additional Context

Why unset HOME ?
azure-cli put extensions installed by az extension add on a certain dir which based on $HOME

Thanks for the feedback, we will look into it soon.

It would be useful to have clear instructions on how to pre-provision extensions, for example when spinning Azure DevOps build agents.

Met the same problem in build
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.6.9/x64/bin/azdev", line 8, in <module>
    sys.exit(main())
  File "/opt/hostedtoolcache/Python/3.6.9/x64/lib/python3.6/site-packages/azdev/__main__.py", line 40, in main
    config_dir=get_azdev_config_dir())
  File "/opt/hostedtoolcache/Python/3.6.9/x64/lib/python3.6/site-packages/azdev/utilities/config.py", line 24, in get_azdev_config_dir
    _, env_name = os.path.splitdrive(get_env_path())
  File "/opt/hostedtoolcache/Python/3.6.9/x64/lib/python3.6/posixpath.py", line 138, in splitdrive
    p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType

Intuitively, it's a bug that caused by azdev.
As a workaround, you could use a virtual Python environment to make it work like what CI does now.
I will fix it with the latest azdev release.

Sorry. It's not an extension of azdev. But share the common behavior when HOME is unset.

azure-cli put configuraiton files in ~/.azure by default. Which is set via:
https://github.com/Azure/azure-cli/blob/13eff82d94b9f59f8206f0424b5bbb0c18055522/src/azure-cli-core/azure/cli/core/_environment.py#L7-L9
The triky part is here, it use tilde ~ instead of HOME environment variable.
According to the doc from GNU 3.5.2 Tilde Expansion

If a word begins with an unquoted tilde character (‘~’), all of the characters up to the first unquoted slash (or all characters, if there is no unquoted slash) are considered a tilde-prefix.
If none of the characters in the tilde-prefix are quoted, the characters in the tilde-prefix following the tilde are treated as a possible login name.
If this login name is the null string, the tilde is replaced with the value of the HOME shell variable.
If HOME is unset, the home directory of the user executing the shell is substituted instead.
Otherwise, the tilde-prefix is replaced with the home directory associated with the specified login name.

To conclude, ~ still have meaningful value even when HOME is unset. Which means ~/.azure/ is still a value path.
azure-cli-ml use the HOME environment variable so the behavior is inconsisnt with az extension add -n azure-cli-ml

I can confirm this behavior is an issue for much more than just ml. When HOME is not set, this is the output of the help text of az:
Group
    az

Subgroups:
    account                      : Manage Azure subscription information.
    ad                           : Manage Azure Active Directory Graph entities needed for Role
                                   Based Access Control.
    appconfig                    : Manage App Configurations.
    appservice                   : Manage App Service plans.
    cache              [Preview] : Commands to manage CLI objects cached using the
                                   `--defer` argument.
    cloud                        : Manage registered Azure clouds.
    deployment                   : Manage Azure Resource Manager template deployment at subscription
                                   scope.
    disk                         : Manage Azure Managed Disks.
    eventhubs                    : Manage Azure Event Hubs namespaces, eventhubs, consumergroups and
                                   geo recovery configurations - Alias.
    extension                    : Manage and update CLI extensions.
    functionapp                  : Manage function apps. To install the Azure Functions Core tools
                                   see https://github.com/Azure/azure-functions-core-tools.
    group                        : Manage resource groups and template deployments.
    hdinsight                    : Manage HDInsight resources.
    image                        : Manage custom virtual machine images.
    iot                          : Manage Internet of Things (IoT) assets.
    keyvault                     : Manage KeyVault keys, secrets, and certificates.
    local-context [Experimental] : Manage Local Context.
    lock                         : Manage Azure locks.
    managedapp                   : Manage template solutions provided and maintained by Independent
                                   Software Vendors (ISVs).
    network                      : Manage Azure Network resources.
    policy                       : Manage resource policies.
    provider                     : Manage resource providers.
    resource                     : Manage Azure resources.
    role                         : Manage user roles for access control with Azure Active Directory
                                   and service principals.
    security           [Preview] : Manage your security posture with Azure Security
                                   Center.
    snapshot                     : Manage point-in-time copies of managed disks, native blobs, or
                                   other snapshots.
    storage                      : Manage Azure Cloud Storage resources.
    tag                          : Manage resource tags.
    vm                           : Manage Linux or Windows virtual machines.
    vmss                         : Manage groupings of virtual machines in an Azure Virtual Machine
                                   Scale Set (VMSS).
    webapp                       : Manage web apps.

Commands:
    configure                    : Manage Azure CLI configuration. This command is interactive.
    feedback                     : Send feedback to the Azure CLI Team!
    find                         : I'm an AI robot, my advice is based on our Azure documentation as
                                   well as the usage patterns of Azure CLI and Azure ARM users.
                                   Using me improves Azure products and documentation.
    interactive        [Preview] : Start interactive mode. Installs the Interactive
                                   extension if not installed already.
    login                        : Log in to Azure.
    logout                       : Log out to remove access to Azure subscriptions.
    rest                         : Invoke a custom request.
    version            [Preview] : Show the versions of Azure CLI modules and extensions
                                   in JSON format by default or format configured by --output.

The issue also occurs when running as root. In such a case, HOME is set to /root in ubuntu 16.04 LTS. Setting HOME=. seems to cause all the subgroups to be resolved again.

@haroldrandom considering the broad scope of this issue across many services, do you still think this should not be fixed centrally?

@DavidParks8 Thanks for the remind, I will think about how to deal with it in CLI.

@DavidParks8 Could you give a list of the missing command groups?

@algattik are you still facing this issue?

Since there hasn't been a response to this issue in the last several months, we assume it was resolved and will close it. Should there still be a question, please open a new issue and we will respond on a timely basis. Thank you.


nicotine-plus: Add version 3.1.1

A "graphical client for the Soulseek file sharing network": https://github.com/nicotine-plus/nicotine-plus

/verify


[Function Request] Table 901 - Assembly Line - GetSkipVerificationsThatChangeDatabase

Dear ALAppExtensions team,
Through this issue we would like to request a new external function that serves as a 'getter' for global variable SkipVerificationsThatChangeDatabase, i.e.:
[External] GetSkipVerificationsThatChangeDatabase() : Boolean
EXIT(SkipVerificationsThatChangeDatabase);

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.


Upgrade Electron to Version 11

Upgrade the version of Electron that Electron.NET is using from 9 to 11.

I started a POC here: https://github.com/scottkuhl/Electron.NET/tree/poc/upgrade-to-electron-11
That repo updates all NPM packages to the latest along with Electron to v11.  It looks like there is one breaking change that will affect Electron.NET: https://github.com/electron/electron/pull/23578  The id property has been removed from BrowserView.
I don't see anywhere this code is used on the web app for testing.  And since there are no automated tests, how do you normally go about testing these kinds of changes?

Our next update will support Electron 11 + .NET 5.

@GregorBiswanger
I have a PR in for .NET 5 and I'm working on the Electron upgrade to 11 in the linked repo above.  Should I just hold off and wait?

Oh no, that´s perfect! I will definitely prefer your PR... because we only have time for the big update at the beginning of December...


Fix further reading link

Fixed the further reading link of Synapse from the (mistaken) ADB to the (correct) Synapse one.

@AbeOmor

Can you review this PR?

IMPORTANT: When this content is ready to merge, you must add #sign-off in a comment or the approval may get overlooked.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#sign-off


showLineNumbers occurs error in React

Using next-mdx-remote, then use some plugins:
  const content = await serialize(post.content, {
    mdxOptions: {
      remarkPlugins: [
        remarkHint,
        remarkBreaks,
      ],
      rehypePlugins: [
        rehypePrismPlus, // import rehypePrismPlus from "rehype-prism-plus";
        rehypeSlug,
      ]
    },
  })

then gets:
Warning: React does not recognize the `showLineNumbers` prop on a DOM element. If you intentionally want it to appear in the DOM as a custom attribute, spell it as lowercase `showlinenumbers` instead. If you accidentally passed it from a parent component, remove it from the DOM element.

with:
```jsx showLineNumbers
const Component: VFC<Props> = ({ children }) => {
  return (
    <Hoge>
      <Foo />
    </Hoge>
  )
}

Tried showlinenumbers, showlinenumbers={true} and showLineNumbers but those all are throwing error (not exact same error)
Do I missed something or misconfigurated?
Thanks!

It has to be camelCase i.e. showLineNumbers, but the example you provided should work. Does the code block render without error if you exclude the showLineNumbers option?

@timlrx Hello, thank you for responding!
Answer is yes, excluding, means only these code:
```jsx showLineNumbers
const Component: VFC<Props> = ({ children }) => {
  return (
    <Hoge>
      <Foo />
    </Hoge>
  )
}

works perfectly as I expected.

Also, additionally, jsx {1} is not seems to working too.

It works using the xdm / mdx v2 compiler so there's probably some difference in how the blocks are parsed. next-remote-mdx still uses mdx v1. Do you have a repo which I can replicate the error?
Also pinging @wooorm since we just discussed the issue on twitter. Would appreciate the help if you know what might be the difference, thanks.

My guess is that MDX 1 never really supported userland properties well. There have been other issues in the past about casing being weird. That’s solved in xdm and MDX 2.
The Hashi folks are working on a next-mdx-remote version with support for the MDX 2 RC btw.

Thanks, in that case I will just support XDM or MDX v2 for simplicity.
@rikusen0335 maybe you could try mdx-bundler instead? I use that myself and it works with the plugin.

@timlrx Of course! I would try replacing to mdx-bundler then once if it's not worked, I would reproduce it on Codesandbox or another repo.

So after replaced next-mdx-remote to mdx-bundler and applied same theme as demo pics used, it seems working perfectly!
I appreciate it.
![image](https://user-images.githubusercontent.com/19174234/141120308-0f829a54-7113-46d5-88c4-8bb430482d01.png


UI sends wrong request when creating new metadata field without qualifier

Describe the bug
Use the UI to create a new metadata field. (It does not matter which metadata schema is used.) Leave the qualifier form input blank.
A POST request is sent to <url>/api/core/metadatafields?schemaId=<schema id> with the following body:
{"element":"mytest1","qualifier":"","scopeNote":"created via GUI"}

Note that qualifier contains an empty string (instead of null). The request is able to create the metadata field, but an empty string is stored in the database, instead of the customary NULL value. (That in turn leads to the metadata field not being retrievable by some parts of DSpace.)
To Reproduce
Steps to reproduce the behavior:

Use the UI to create a new metadata field. Leave the qualifier blank.
Observe that an empty string is sent to the backend
Observe that an empty string is saved to the database

Expected behavior
When the qualifier input is left blank, the request should be:
{"element":"mytest1","qualifier":null,"scopeNote":"created via GUI"}

or even omit the qualifier:
{"element":"mytest1","scopeNote":"created via GUI"}

Additionally the backend may want to reject empty values with 422 Unprocessable Entity, or convert them to null automatically.
When reproducing the bug locally, I used the latest dspace-angular and the latest backend (at time of writing).

Estimated by @benbosman as 4 hours total (2 hours REST, 2 hours Angular).  Assigning to @benbosman  to try to fit in prior to Testathon.


Update Issues Template to be Relevant to Repo

Issue Description
Update Issues Template to be Relevant to Repo

Resolved in this commit: https://github.com/cerner/eslint-config-terra/commit/6f292cb37eb971818ee8364eb38de3c5e652b9ed


Tuple fix

Removed all tuple args from functions and replaced with separate args

Added import fix from https://github.com/pysamp/PySAMP/pull/116

Thanks a lot! Merging.


Consistently use String.format instead of concatenation in FsUtils

Would be nice to have lazy eval of log strings so
this work doesn’t occur for disabled log levels.

I meant to send this to a fork. Please ignore!


Materio Free Bootstrap Admin Template

Materio Free Bootstrap 5 HTML Admin Template:
It is an Open-source & easy to use Free Bootstrap 5 Admin Template with elegant design & unique layouts. Materio admin offers an Attractive Dashboard, enhanced bootstrap 5 components, AUTH Pages, Tables, Ready to use cards & form layouts.
Besides, the highest industry standards are considered to bring you the very best bootstrap admin template that is not just fast and easy to use, but highly scalable. Offering ultimate convenience and flexibility, you’ll be able to build whatever application you want without any hassle.
Furthermore, you can use this one of the best innovative free bootstrap admin templates to create eye-catching, high-quality, and high-performing single-page applications. Besides, your apps will be completely responsive, ensuring they’ll look stunning and function flawlessly on desktops, tablets, and mobile devices.
For instance, you can create:

SaaS platforms
Project management apps
Ecommerce backends
CRM systems
Analytics apps

Features:

Based on Bootstrap 5
Basic Table
Basic Cards
Forms
1 Dashboard

Demo: https://www.themeselection.com/demo/materio-bootstrap-html-admin-template-free/html/
Download: https://themeselection.com/item/materio-free-bootstrap-html-admin-template/
GitHub Repo: https://github.com/themeselection/materio-bootstrap-html-admin-template-free

Looks dope :) just merged it

Hey,
Thanks a lot for adding it.
I have submitted the image as well, but it is not showing on the repo page.
Here is the image:
https://github.com/awesome-bootstrap-org/awesome-bootstrap/pull/42/commits/db13d68cb831f509b78b1dd5976def88e32c22a6


Build fails when using Realm and CocoaPods

I am trying to run Infer on my project that includes a bunch of Pods. One of them is Realm.
Running xcodebuild without infer the project builds without any issue.
Running with infer it fails with the following message:
** BUILD FAILED **

The following build commands failed:
	CompileC /Users/tveloso/Library/Developer/Xcode/DerivedData/ABC-dbvlmexxbrkpqubvzgqzuzdlxptp/Build/Intermediates/Pods.build/Debug-iphoneos/Pods-ABC-Realm.build/Objects-normal/armv7/external_commit_helper.o Realm/Realm/ObjectStore/impl/apple/external_commit_helper.cpp normal armv7 c++ com.apple.compilers.llvm.clang.1_0.compiler
(1 failure)
None

I am using the latest xcodebuild and infer.

Hello @TiagoVeloso,
I will investigate this issue and give you updates!
In the meanwhile, could you please send me a paste of the real error? What you provided is just a summary of the errors that happened during the build. The real message should be a bit above that line.
Thanks!

This was the first occurrence I found. But there are a couple of errors like this one:
While building module 'CoreFoundation' imported from /Users/tveloso/Developer/abc/Pods/Realm/include/realm/external_commit_helper.hpp:22:
While building module 'Darwin' imported from /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator9.2.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/CoreFoundation.h:11:
While building module 'std' imported from /Users/tveloso/Developer/infer-osx-v0.5.0/facebook-clang-plugins/clang/bin/../include/c++/v1/math.h:309:
In file included from <module-includes>:1:
In file included from /Users/tveloso/Developer/infer-osx-v0.5.0/facebook-clang-plugins/clang/bin/../include/c++/v1/algorithm:624:
In file included from /Users/tveloso/Developer/infer-osx-v0.5.0/facebook-clang-plugins/clang/bin/../include/c++/v1/initializer_list:47:
In file included from /Users/tveloso/Developer/infer-osx-v0.5.0/facebook-clang-plugins/clang/bin/../include/c++/v1/cstddef:43:
/Users/tveloso/Developer/infer-osx-v0.5.0/facebook-clang-plugins/clang/bin/../include/c++/v1/stddef.h:46:15: fatal error: cyclic dependency in module 'Darwin': Darwin -> std -> Darwin
#include_next <stddef.h>
              ^

if you need more of my build out type let me know

Could you please provide the command you used to build?
Thanks.

Something like this:
infer -- xcodebuild -workspace ABC.xcworkspace -scheme Debug -sdk iphonesimulator -destination 'platform=iOS Simulator,OS=9.2,name=iPhone 6s' clean build

The error paste you provided is not enough, please make sure you see the error:  message of clang.
Does it compile if you just run the build command?
xcodebuild -workspace ABC.xcworkspace -scheme Debug -sdk iphonesimulator -destination 'platform=iOS Simulator,OS=9.2,name=iPhone 6s'

Yes without infer it builds without an issue.
In attachment is the complete log file of the build process
out.txt

Ok seems like this is a known problem in clang and for now what I recommend to do is to append
CLANG_ENABLE_MODULES=NO

to your build command, so for example in your case it will become:
infer -- xcodebuild -workspace ABC.xcworkspace -scheme Debug -sdk iphonesimulator -destination 'platform=iOS Simulator,OS=9.2,name=iPhone 6s' CLANG_ENABLE_MODULES=NO

Please let me know if that helps!

Is there any workaround besides using CLANG_ENABLE_MODULES=NO. The reason why I can't use is it is my project requires modules.

I am in the same situation as @joshluongo I wasn't able to compile with that flag as well.

I found a simple solution to this problem if your app uses modules, Simply switch off Enable Modules (C and Objective-C) in the target for 'Realm'. http://imgur.com/PSfw7ks
If you do that then you don't have to add CLANG_ENABLE_MODULES=NO to your command.

Should be fixed in latest master (see #335).


Add a command that makes the app request storage permissions

Hello,
I'd like to know if there's a way to access my files from within Nix-on-Droid.
I've tried to cd to /sdcard or /storage/emulated/0 but I can't do anything in them, ls gives ls: .: Permission denied for both of them.
I've tried from the normal & failsafe app.

Is this maybe what you need? https://github.com/t184256/nix-on-droid/issues/85

Yeah, you should try the storage toggle in the app settings inside Android settings.

Thanks @Gerschtli I saw that issue but couldn't find that toggle. It was actually under "Permission".
Indeed that solves the problem, sorry for the noise I should have tried better.
Would it be possible to show a popup when trying to read these directories, or add a command to unlock them (that would show a popup for asking permission) ?

Would it be possible to show a popup when trying to read these directories

Everything's possible in theory, but that'd be really hard to implement and would likely incur a performance cost.

or add a command to unlock them (that would show a popup for asking permission)?

Yes, if someone knowing Android programming understands and revives/recreates what Termux did to achieve that.

Linking the Termux relevant wiki page: https://wiki.termux.com/wiki/Internal_and_external_storage


Consider replacing src/util/simdutf8check.h

The code in src/util/simdutf8check.h is suboptimal. It could be replaced either by...

simdutf which provides full support for Unicode (transcoding, validation, and so forth).
is_utf8 which provides just the check for UTF-8 validity.

Both of these libraries have runtime dispatching, support for NEON, AVX-512 and so forth, with extensive testing.
Switching to simdutf would improve the performance.

Thanks for advice. We would benchmark it later

I got some compiling error with simdutf, @lemire

Error: Error: no such instruction: vpcompressb %zmm0,%zmm1{%k6}'`
GCC 10.3.0

CPU:
➤ lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                104
On-line CPU(s) list:   0-103
Thread(s) per core:    2
Core(s) per socket:    26
Socket(s):             2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 85
Model name:            Intel(R) Xeon(R) Platinum 8269CY CPU @ 2.50GHz
Stepping:              7

Note that simdutf is part of Node.js which is built and used on millions of systems.
If you cannot build simdutf, the most likely cause is a bad toolchain.

I got some compiling error with simdutf, @lemire

Error: Error: no such instruction: ·vpcompressb %zmm0,%zmm1{%k6}'
GCC 10.3.0

CPU:
➤ lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                104
On-line CPU(s) list:   0-103
Thread(s) per core:    2
Core(s) per socket:    26
Socket(s):             2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 85
Model name:            Intel(R) Xeon(R) Platinum 8269CY CPU @ 2.50GHz
Stepping:              7

@mofeiatwork
saw the similar error on our centos7 toolchain, it works on our ubuntu22 toolchains, maybe it's time to upgrade our centos7 toolchain to gcc11 ?

It is almost certainly the case that you are mixing an old code generator with a more recent compiler. Linux distributions upgrade the two in sync, but you can design systems that have old code generators and very recent compiler. Check your gas/nasm version.
The problem often is that the compiler does not know about the code generator, so it asks the code generator to produce instructions it does know about.
Quoting from the simdutf documentation:

AVX-512 support require a processor with AVX512-VBMI2 (Ice Lake or better) and a recent compiler (GCC 8 or better, Visual Studio 2019 or better, LLVM clang 6 or better). You need a correspondingly recent assembler such as gas (2.30+) or nasm (2.14+): recent compilers usually come with recent assemblers. If you mix a recent compiler with an incompatible/old assembler (e.g., when using a recent compiler with an old Linux distribution), you may get errors at build time because the compiler produces instructions that the assembler does not recognize: you should update your assembler to match your compiler (e.g., upgrade binutils to version 2.30 or better under Linux) or use an older compiler matching the capabilities of your assembler.

it could be, we are using centos7 image, with default binutils and manual compiled gcc10, it doesn't work when use gcc10 + default binutils (include the as), but works when upgrades to gcc11 + default binutils.
still can't understand what's the cause.

plan to upgrade gcc in the toolchain from gcc10.3 to gcc11.3 #29552

it could be, we are using centos7 image, with default binutils and manual compiled gcc10. It doesn't work when use gcc10 + default binutils (include the as), but works when upgrades to gcc11 + default binutils.
still can't understand what's the cause.

appears that the default as in centos7 is too old (version 2.27), need upgrade the binutils as well.

with #29813 change, the simduft project can be built successful with no error.
$ docker run -it --rm -v `pwd`/simdutf:/root/simdutf starrocks/dev-env-centos7:latest bash
[root@c0315012778a ~]# cd /root/simdutf/
[root@c0315012778a simdutf]# cmake -B build
-- The CXX compiler identification is GNU 10.3.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/rh/gcc-toolset-10/root/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- No build type selected, default to Release
-- Found Python3: /usr/bin/python3.6 (found version "3.6.8") found components: Interpreter
-- Python found, we are going to amalgamate.py.
-- The tests are enabled.
-- Performing Test Iconv_IS_BUILT_IN
-- Performing Test Iconv_IS_BUILT_IN - Success
-- Found Iconv: built in to C library
-- Iconv was found!
-- Iconv is part of the C library.
-- looking for static C++ library in /opt/rh/gcc-toolset-10/root/usr/lib/gcc/x86_64-pc-linux-gnu/10.3.0
-- libstdc++.a not found
-- The benchmarks can be disabled by setting SIMDUTF_BENCHMARKS, e.g., -D SIMDUTF_BENCHMARKS=OFF.
-- Found the following ICU libraries:
--   uc (required)
-- Failed to find all ICU components (missing: ICU_INCLUDE_DIR ICU_LIBRARY)
-- We rely on the system's ICU. It was not found!
-- Iconv was found!
-- Iconv is part of the C library.
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Compiling using the C++ standard:11
-- Configuring done
-- Generating done
-- Build files have been written to: /root/simdutf/build
[root@c0315012778a simdutf]# cd build
[root@c0315012778a build]# make -j 10
[  1%] Building CXX object src/CMakeFiles/simdutf.dir/simdutf.cpp.o
[  1%] Linking CXX static library libsimdutf.a
[  1%] Built target simdutf
[  2%] Generating simdutf.cpp, simdutf.h, amalgamation_demo.cpp, README.md
[  3%] Building CXX object tests/reference/CMakeFiles/simdutf_tests_reference.dir/encode_utf8.cpp.o
[  4%] Building CXX object tools/CMakeFiles/sutf.dir/sutf.cpp.o
[  5%] Building CXX object tests/reference/CMakeFiles/simdutf_tests_reference.dir/encode_utf16.cpp.o
[  5%] Building CXX object tests/reference/CMakeFiles/simdutf_tests_reference.dir/encode_utf32.cpp.o
[  6%] Building CXX object benchmarks/CMakeFiles/stream.dir/stream.cpp.o
[  7%] Building CXX object tests/reference/CMakeFiles/simdutf_tests_reference.dir/encode_latin1.cpp.o
[  7%] Building CXX object tests/reference/CMakeFiles/simdutf_tests_reference.dir/validate_utf8_to_latin1.cpp.o
[  7%] Building CXX object benchmarks/CMakeFiles/alignment.dir/alignment.cpp.o
[  8%] Building CXX object benchmarks/CMakeFiles/threaded.dir/threaded.cpp.o
[  9%] Building CXX object tests/reference/CMakeFiles/simdutf_tests_reference.dir/validate_utf16_to_latin1.cpp.o
[ 10%] Building CXX object tests/reference/CMakeFiles/simdutf_tests_reference.dir/validate_utf32_to_latin1.cpp.o
[ 10%] Building CXX object tests/reference/CMakeFiles/simdutf_tests_reference.dir/validate_utf8.cpp.o
[ 11%] Building CXX object tests/reference/CMakeFiles/simdutf_tests_reference.dir/validate_utf16.cpp.o
[ 11%] Building CXX object tests/reference/CMakeFiles/simdutf_tests_reference.dir/validate_utf32.cpp.o
SCRIPTPATH=/root/simdutf/singleheader PROJECTPATH=/root/simdutf
We are about to amalgamate all simdutf files into one source file.
See https://www.sqlite.org/amalgamation.html and https://en.wikipedia.org/wiki/Single_Compilation_Unit for rationale.
[ 12%] Building CXX object tests/reference/CMakeFiles/simdutf_tests_reference.dir/validate_latin1.cpp.o
timestamp is 2023-08-11 14:01:05 -0400
Creating /root/simdutf/build/singleheader/simdutf.h

Creating /root/simdutf/build/singleheader/simdutf.cpp
[ 12%] Linking CXX static library libsimdutf_tests_reference.a
[ 12%] Built target simdutf_tests_reference
[ 13%] Building CXX object tests/helpers/CMakeFiles/simdutf_tests_helpers.dir/test.cpp.o
[ 14%] Building CXX object tests/helpers/CMakeFiles/simdutf_tests_helpers.dir/transcode_test_base.cpp.o
[ 14%] Building CXX object tests/helpers/CMakeFiles/simdutf_tests_helpers.dir/random_int.cpp.o
[ 15%] Building CXX object tests/helpers/CMakeFiles/simdutf_tests_helpers.dir/random_utf8.cpp.o
[ 15%] Building CXX object tests/helpers/CMakeFiles/simdutf_tests_helpers.dir/random_utf16.cpp.o
Done with all files generation.
Files have been written to directory: /root/simdutf/build/singleheader/
Done with all files generation.

Giving final instructions:

While in the singleheader directory under a linux or macOS system with an install toolchain, try:

c++ -o amalgamation_demo amalgamation_demo.cpp -std=c++17 && ./amalgamation_demo

[ 15%] Built target singleheader-files
[ 16%] Building CXX object tests/helpers/CMakeFiles/simdutf_tests_helpers.dir/random_utf32.cpp.o
[ 17%] Building CXX object singleheader/CMakeFiles/amalgamation_demo.dir/amalgamation_demo.cpp.o
[ 17%] Linking CXX executable stream
[ 18%] Linking CXX executable threaded
[ 18%] Built target stream
[ 19%] Linking CXX executable alignment
[ 19%] Built target threaded
[ 19%] Built target alignment
[ 19%] Linking CXX static library libsimdutf_tests_helpers.a
[ 19%] Built target simdutf_tests_helpers
[ 19%] Building CXX object tests/CMakeFiles/random_fuzzer.dir/random_fuzzer.cpp.o
[ 20%] Building CXX object tests/CMakeFiles/special_tests.dir/special_tests.cpp.o
[ 20%] Building CXX object tests/CMakeFiles/validate_ascii_basic_tests.dir/validate_ascii_basic_tests.cpp.o
[ 20%] Building CXX object tests/CMakeFiles/validate_ascii_with_errors_tests.dir/validate_ascii_with_errors_tests.cpp.o
[ 20%] Building CXX object tests/CMakeFiles/bele_tests.dir/bele_tests.cpp.o
[ 20%] Building CXX object tests/CMakeFiles/validate_utf8_basic_tests.dir/validate_utf8_basic_tests.cpp.o
[ 20%] Building CXX object tests/CMakeFiles/select_implementation.dir/select_implementation.cpp.o
[ 21%] Building CXX object tests/CMakeFiles/validate_utf8_brute_force_tests.dir/validate_utf8_brute_force_tests.cpp.o
[ 21%] Linking CXX executable sutf
[ 22%] Linking CXX executable select_implementation
[ 23%] Linking CXX executable validate_ascii_basic_tests
[ 23%] Built target sutf
[ 24%] Building CXX object tests/CMakeFiles/validate_utf8_puzzler_tests.dir/validate_utf8_puzzler_tests.cpp.o
[ 25%] Linking CXX executable validate_utf8_basic_tests
[ 25%] Built target select_implementation
[ 26%] Building CXX object tests/CMakeFiles/validate_utf8_with_errors_tests.dir/validate_utf8_with_errors_tests.cpp.o
[ 26%] Built target validate_ascii_basic_tests
[ 26%] Built target validate_utf8_basic_tests
[ 27%] Building CXX object tests/CMakeFiles/validate_utf16le_basic_tests.dir/validate_utf16le_basic_tests.cpp.o
[ 28%] Building CXX object tests/CMakeFiles/validate_utf16be_basic_tests.dir/validate_utf16be_basic_tests.cpp.o
[ 28%] Linking CXX executable validate_utf8_brute_force_tests
[ 29%] Linking CXX executable validate_ascii_with_errors_tests
[ 29%] Built target validate_utf8_brute_force_tests
[ 30%] Building CXX object tests/CMakeFiles/validate_utf16le_with_errors_tests.dir/validate_utf16le_with_errors_tests.cpp.o
[ 30%] Built target validate_ascii_with_errors_tests
[ 31%] Building CXX object tests/CMakeFiles/validate_utf16be_with_errors_tests.dir/validate_utf16be_with_errors_tests.cpp.o
[ 32%] Linking CXX executable bele_tests
[ 33%] Linking CXX executable random_fuzzer
[ 33%] Built target bele_tests
[ 33%] Building CXX object tests/CMakeFiles/validate_utf32_basic_tests.dir/validate_utf32_basic_tests.cpp.o
[ 33%] Built target random_fuzzer
[ 33%] Linking CXX executable special_tests
[ 33%] Building CXX object tests/CMakeFiles/validate_utf32_with_errors_tests.dir/validate_utf32_with_errors_tests.cpp.o
[ 33%] Built target special_tests
[ 34%] Building CXX object tests/CMakeFiles/convert_latin1_to_utf8_tests.dir/convert_latin1_to_utf8_tests.cpp.o
[ 34%] Linking CXX executable validate_utf8_puzzler_tests
[ 34%] Built target validate_utf8_puzzler_tests
[ 35%] Building CXX object tests/CMakeFiles/convert_latin1_to_utf16le_tests.dir/convert_latin1_to_utf16le_tests.cpp.o
[ 36%] Linking CXX executable validate_utf8_with_errors_tests
[ 36%] Built target validate_utf8_with_errors_tests
[ 36%] Building CXX object tests/CMakeFiles/convert_latin1_to_utf16be_tests.dir/convert_latin1_to_utf16be_tests.cpp.o
[ 36%] Linking CXX executable validate_utf16be_basic_tests
[ 36%] Built target validate_utf16be_basic_tests
[ 36%] Linking CXX executable convert_latin1_to_utf8_tests
[ 37%] Building CXX object tests/CMakeFiles/convert_latin1_to_utf32_tests.dir/convert_latin1_to_utf32_tests.cpp.o
[ 37%] Linking CXX executable validate_utf16le_basic_tests
[ 38%] Linking CXX executable validate_utf32_basic_tests
[ 38%] Built target convert_latin1_to_utf8_tests
[ 38%] Building CXX object tests/CMakeFiles/convert_utf8_to_latin1_tests.dir/convert_utf8_to_latin1_tests.cpp.o
[ 38%] Built target validate_utf32_basic_tests
[ 38%] Building CXX object tests/CMakeFiles/convert_utf8_to_latin1_with_errors_tests.dir/convert_utf8_to_latin1_with_errors_tests.cpp.o
[ 38%] Linking CXX executable validate_utf16be_with_errors_tests
[ 39%] Linking CXX executable validate_utf16le_with_errors_tests
[ 40%] Linking CXX executable validate_utf32_with_errors_tests
[ 40%] Built target validate_utf16be_with_errors_tests
[ 41%] Building CXX object tests/CMakeFiles/convert_valid_utf8_to_latin1_tests.dir/convert_valid_utf8_to_latin1_tests.cpp.o
[ 41%] Built target validate_utf16le_with_errors_tests
[ 41%] Built target validate_utf32_with_errors_tests
[ 41%] Linking CXX executable convert_latin1_to_utf16le_tests
[ 41%] Building CXX object tests/CMakeFiles/convert_valid_utf8_to_utf16le_tests.dir/convert_valid_utf8_to_utf16le_tests.cpp.o
[ 41%] Building CXX object tests/CMakeFiles/convert_valid_utf8_to_utf16be_tests.dir/convert_valid_utf8_to_utf16be_tests.cpp.o
[ 41%] Built target validate_utf16le_basic_tests
[ 41%] Building CXX object tests/CMakeFiles/convert_valid_utf8_to_utf32_tests.dir/convert_valid_utf8_to_utf32_tests.cpp.o
[ 41%] Built target convert_latin1_to_utf16le_tests
[ 42%] Building CXX object tests/CMakeFiles/convert_utf8_to_utf16le_tests.dir/convert_utf8_to_utf16le_tests.cpp.o
[ 43%] Linking CXX executable convert_latin1_to_utf16be_tests
[ 43%] Built target convert_latin1_to_utf16be_tests
[ 43%] Building CXX object tests/CMakeFiles/convert_utf8_to_utf16be_tests.dir/convert_utf8_to_utf16be_tests.cpp.o
[ 43%] Linking CXX executable convert_latin1_to_utf32_tests
[ 43%] Built target convert_latin1_to_utf32_tests
[ 44%] Building CXX object tests/CMakeFiles/convert_utf8_to_utf16le_with_errors_tests.dir/convert_utf8_to_utf16le_with_errors_tests.cpp.o
[ 45%] Linking CXX executable convert_utf8_to_latin1_tests
[ 46%] Linking CXX executable convert_valid_utf8_to_latin1_tests
[ 46%] Built target convert_utf8_to_latin1_tests
[ 47%] Building CXX object tests/CMakeFiles/convert_utf8_to_utf16be_with_errors_tests.dir/convert_utf8_to_utf16be_with_errors_tests.cpp.o
[ 47%] Built target convert_valid_utf8_to_latin1_tests
[ 47%] Building CXX object tests/CMakeFiles/convert_utf8_to_utf32_tests.dir/convert_utf8_to_utf32_tests.cpp.o
[ 48%] Linking CXX executable convert_utf8_to_latin1_with_errors_tests
[ 49%] Linking CXX executable convert_valid_utf8_to_utf16be_tests
[ 49%] Built target convert_utf8_to_latin1_with_errors_tests
[ 49%] Building CXX object tests/CMakeFiles/convert_utf8_to_utf32_with_errors_tests.dir/convert_utf8_to_utf32_with_errors_tests.cpp.o
[ 49%] Built target convert_valid_utf8_to_utf16be_tests
[ 50%] Building CXX object tests/CMakeFiles/convert_utf16le_to_latin1_tests.dir/convert_utf16le_to_latin1_tests.cpp.o
[ 51%] Linking CXX executable convert_valid_utf8_to_utf32_tests
[ 52%] Linking CXX executable convert_valid_utf8_to_utf16le_tests
[ 52%] Built target convert_valid_utf8_to_utf32_tests
[ 52%] Built target convert_valid_utf8_to_utf16le_tests
[ 53%] Building CXX object tests/CMakeFiles/convert_utf16be_to_latin1_tests.dir/convert_utf16be_to_latin1_tests.cpp.o
[ 53%] Building CXX object tests/CMakeFiles/convert_utf16le_to_latin1_tests_with_errors.dir/convert_utf16le_to_latin1_tests_with_errors.cpp.o
[ 53%] Linking CXX executable convert_utf8_to_utf16le_tests
[ 54%] Linking CXX executable convert_utf8_to_utf16be_tests
[ 54%] Built target convert_utf8_to_utf16le_tests
[ 54%] Built target convert_utf8_to_utf16be_tests
[ 54%] Building CXX object tests/CMakeFiles/convert_utf16be_to_latin1_tests_with_errors.dir/convert_utf16be_to_latin1_tests_with_errors.cpp.o
[ 55%] Building CXX object tests/CMakeFiles/convert_valid_utf16le_to_latin1_tests.dir/convert_valid_utf16le_to_latin1_tests.cpp.o
[ 56%] Linking CXX executable convert_utf8_to_utf32_tests
[ 56%] Built target convert_utf8_to_utf32_tests
[ 56%] Building CXX object tests/CMakeFiles/convert_valid_utf16be_to_latin1_tests.dir/convert_valid_utf16be_to_latin1_tests.cpp.o
[ 57%] Linking CXX executable convert_utf8_to_utf16le_with_errors_tests
[ 57%] Built target convert_utf8_to_utf16le_with_errors_tests
[ 58%] Building CXX object tests/CMakeFiles/convert_utf16le_to_utf8_tests.dir/convert_utf16le_to_utf8_tests.cpp.o
[ 59%] Linking CXX executable convert_utf16le_to_latin1_tests
[ 60%] Linking CXX executable convert_utf16le_to_latin1_tests_with_errors
[ 60%] Built target convert_utf16le_to_latin1_tests
[ 61%] Building CXX object tests/CMakeFiles/convert_utf16be_to_utf8_tests.dir/convert_utf16be_to_utf8_tests.cpp.o
[ 62%] Linking CXX executable convert_utf16be_to_latin1_tests
[ 63%] Linking CXX executable convert_utf8_to_utf32_with_errors_tests
[ 63%] Linking CXX executable convert_utf8_to_utf16be_with_errors_tests
[ 63%] Linking CXX executable convert_valid_utf16le_to_latin1_tests
[ 63%] Built target convert_utf16le_to_latin1_tests_with_errors
[ 64%] Building CXX object tests/CMakeFiles/convert_utf16le_to_utf8_with_errors_tests.dir/convert_utf16le_to_utf8_with_errors_tests.cpp.o
[ 64%] Built target convert_utf16be_to_latin1_tests
[ 65%] Building CXX object tests/CMakeFiles/convert_utf16be_to_utf8_with_errors_tests.dir/convert_utf16be_to_utf8_with_errors_tests.cpp.o
[ 65%] Built target convert_utf8_to_utf32_with_errors_tests
[ 65%] Built target convert_utf8_to_utf16be_with_errors_tests
[ 66%] Building CXX object tests/CMakeFiles/convert_utf32_to_latin1_tests.dir/convert_utf32_to_latin1_tests.cpp.o
[ 66%] Built target convert_valid_utf16le_to_latin1_tests
[ 67%] Linking CXX executable convert_utf16be_to_latin1_tests_with_errors
[ 67%] Building CXX object tests/CMakeFiles/convert_valid_utf32_to_latin1_tests.dir/convert_valid_utf32_to_latin1_tests.cpp.o
[ 67%] Building CXX object tests/CMakeFiles/convert_utf32_to_latin1_with_errors_tests.dir/convert_utf32_to_latin1_with_errors_tests.cpp.o
[ 67%] Built target convert_utf16be_to_latin1_tests_with_errors
[ 68%] Building CXX object tests/CMakeFiles/convert_utf32_to_utf8_tests.dir/convert_utf32_to_utf8_tests.cpp.o
[ 69%] Linking CXX executable convert_valid_utf16be_to_latin1_tests
[ 69%] Built target convert_valid_utf16be_to_latin1_tests
[ 70%] Building CXX object tests/CMakeFiles/convert_utf32_to_utf8_with_errors_tests.dir/convert_utf32_to_utf8_with_errors_tests.cpp.o
[ 71%] Linking CXX executable convert_valid_utf32_to_latin1_tests
[ 72%] Linking CXX executable convert_utf32_to_latin1_with_errors_tests
[ 72%] Built target convert_valid_utf32_to_latin1_tests
[ 73%] Linking CXX executable convert_utf32_to_latin1_tests
[ 74%] Building CXX object tests/CMakeFiles/convert_utf32_to_utf16le_tests.dir/convert_utf32_to_utf16le_tests.cpp.o
[ 74%] Built target convert_utf32_to_latin1_with_errors_tests
[ 74%] Building CXX object tests/CMakeFiles/convert_utf32_to_utf16be_tests.dir/convert_utf32_to_utf16be_tests.cpp.o
[ 74%] Built target convert_utf32_to_latin1_tests
[ 75%] Building CXX object tests/CMakeFiles/convert_utf32_to_utf16le_with_errors_tests.dir/convert_utf32_to_utf16le_with_errors_tests.cpp.o
[ 75%] Linking CXX executable convert_utf16le_to_utf8_with_errors_tests
[ 75%] Linking CXX executable convert_utf16le_to_utf8_tests
[ 75%] Linking CXX executable convert_utf32_to_utf8_tests
[ 75%] Built target convert_utf16le_to_utf8_with_errors_tests
[ 75%] Building CXX object tests/CMakeFiles/convert_utf32_to_utf16be_with_errors_tests.dir/convert_utf32_to_utf16be_with_errors_tests.cpp.o
[ 75%] Built target convert_utf16le_to_utf8_tests
[ 75%] Building CXX object tests/CMakeFiles/convert_valid_utf16le_to_utf8_tests.dir/convert_valid_utf16le_to_utf8_tests.cpp.o
[ 75%] Built target convert_utf32_to_utf8_tests
[ 75%] Linking CXX executable convert_utf16be_to_utf8_with_errors_tests
[ 76%] Building CXX object tests/CMakeFiles/convert_valid_utf16be_to_utf8_tests.dir/convert_valid_utf16be_to_utf8_tests.cpp.o
[ 76%] Built target convert_utf16be_to_utf8_with_errors_tests
[ 76%] Linking CXX executable convert_utf16be_to_utf8_tests
[ 77%] Building CXX object tests/CMakeFiles/convert_valid_utf32_to_utf8_tests.dir/convert_valid_utf32_to_utf8_tests.cpp.o
[ 77%] Built target convert_utf16be_to_utf8_tests
[ 78%] Building CXX object tests/CMakeFiles/convert_valid_utf32_to_utf16le_tests.dir/convert_valid_utf32_to_utf16le_tests.cpp.o
[ 79%] Linking CXX executable convert_utf32_to_utf8_with_errors_tests
[ 79%] Built target convert_utf32_to_utf8_with_errors_tests
[ 80%] Building CXX object tests/CMakeFiles/convert_valid_utf32_to_utf16be_tests.dir/convert_valid_utf32_to_utf16be_tests.cpp.o
[ 80%] Linking CXX executable convert_utf32_to_utf16le_tests
[ 81%] Linking CXX executable convert_utf32_to_utf16be_tests
[ 81%] Linking CXX executable convert_utf32_to_utf16le_with_errors_tests
[ 81%] Built target convert_utf32_to_utf16le_tests
[ 81%] Building CXX object tests/CMakeFiles/convert_utf16le_to_utf32_tests.dir/convert_utf16le_to_utf32_tests.cpp.o
[ 81%] Built target convert_utf32_to_utf16be_tests
[ 81%] Built target convert_utf32_to_utf16le_with_errors_tests
[ 81%] Building CXX object tests/CMakeFiles/convert_utf16be_to_utf32_tests.dir/convert_utf16be_to_utf32_tests.cpp.o
[ 81%] Building CXX object tests/CMakeFiles/convert_utf16le_to_utf32_with_errors_tests.dir/convert_utf16le_to_utf32_with_errors_tests.cpp.o
[ 82%] Linking CXX executable convert_utf32_to_utf16be_with_errors_tests
[ 82%] Linking CXX executable convert_valid_utf32_to_utf8_tests
[ 82%] Linking CXX executable convert_valid_utf32_to_utf16le_tests
[ 82%] Built target convert_utf32_to_utf16be_with_errors_tests
[ 83%] Building CXX object tests/CMakeFiles/convert_utf16be_to_utf32_with_errors_tests.dir/convert_utf16be_to_utf32_with_errors_tests.cpp.o
[ 83%] Built target convert_valid_utf32_to_utf8_tests
[ 84%] Building CXX object tests/CMakeFiles/convert_valid_utf16le_to_utf32_tests.dir/convert_valid_utf16le_to_utf32_tests.cpp.o
[ 84%] Built target convert_valid_utf32_to_utf16le_tests
[ 85%] Building CXX object tests/CMakeFiles/convert_valid_utf16be_to_utf32_tests.dir/convert_valid_utf16be_to_utf32_tests.cpp.o
[ 86%] Linking CXX executable convert_valid_utf16le_to_utf8_tests
[ 86%] Linking CXX executable convert_valid_utf16be_to_utf8_tests
[ 86%] Built target convert_valid_utf16le_to_utf8_tests
[ 87%] Building CXX object tests/CMakeFiles/count_utf8.dir/count_utf8.cpp.o
[ 87%] Linking CXX executable convert_valid_utf32_to_utf16be_tests
[ 87%] Built target convert_valid_utf16be_to_utf8_tests
[ 88%] Building CXX object tests/CMakeFiles/count_utf16le.dir/count_utf16le.cpp.o
[ 88%] Built target convert_valid_utf32_to_utf16be_tests
[ 89%] Building CXX object tests/CMakeFiles/count_utf16be.dir/count_utf16be.cpp.o
[ 90%] Linking CXX executable convert_utf16le_to_utf32_with_errors_tests
[ 90%] Built target convert_utf16le_to_utf32_with_errors_tests
[ 90%] Building CXX object tests/CMakeFiles/detect_encodings_tests.dir/detect_encodings_tests.cpp.o
[ 91%] Linking CXX executable convert_utf16le_to_utf32_tests
[ 91%] Built target convert_utf16le_to_utf32_tests
[ 92%] Building CXX object tests/CMakeFiles/basic_fuzzer.dir/basic_fuzzer.cpp.o
[ 93%] Linking CXX executable count_utf8
[ 94%] Linking CXX executable convert_utf16be_to_utf32_tests
[ 94%] Built target count_utf8
[ 95%] Building CXX object benchmarks/src/CMakeFiles/simdutf_benchmarks_benchmark.dir/benchmark_base.cpp.o
[ 95%] Linking CXX executable count_utf16le
[ 96%] Linking CXX executable convert_valid_utf16le_to_utf32_tests
[ 96%] Linking CXX executable convert_utf16be_to_utf32_with_errors_tests
[ 96%] Built target convert_utf16be_to_utf32_tests
[ 96%] Building CXX object benchmarks/src/CMakeFiles/simdutf_benchmarks_benchmark.dir/cmdline.cpp.o
[ 96%] Linking CXX executable convert_valid_utf16be_to_utf32_tests
[ 96%] Built target count_utf16le
[ 96%] Built target convert_valid_utf16le_to_utf32_tests
[ 97%] Building CXX object benchmarks/src/CMakeFiles/simdutf_benchmarks_benchmark.dir/benchmark.cpp.o
[ 97%] Built target convert_utf16be_to_utf32_with_errors_tests
[ 97%] Built target convert_valid_utf16be_to_utf32_tests
[ 97%] Linking CXX executable count_utf16be
[ 97%] Built target count_utf16be
[ 98%] Linking CXX executable detect_encodings_tests
[ 98%] Built target detect_encodings_tests
[ 99%] Linking CXX executable basic_fuzzer
[ 99%] Built target basic_fuzzer
[ 99%] Linking CXX executable amalgamation_demo
[ 99%] Built target amalgamation_demo
[ 99%] Linking CXX static library libsimdutf_benchmarks_benchmark.a
[ 99%] Built target simdutf_benchmarks_benchmark
[ 99%] Building CXX object benchmarks/CMakeFiles/benchmark.dir/benchmark.cpp.o
[100%] Linking CXX executable benchmark
[100%] Built target benchmark
[root@c0315012778a build]#

Note that simdutf is used by Node.js and Bun, and various other systems, in production... and it has been used in production for several years.
We also have fast base64 decoding functions (they are what Node.js uses to decode base64) as part of simdutf now!!!

validate_utf8+haswell, input size: 4979846, iterations: 100, dataset: all-in1.txt
   8.765 GB/s (0.8 %)    7.307 Gc/s     1.20 byte/char
validate_utf8_sr+haswell, input size: 4979846, iterations: 100, dataset: all-in1.txt
   1.069 GB/s (1.3 %)    0.891 Gc/s     1.20 byte/char

It's around 8x times boost compared to current implementation in SR.

It's around 7x~8x times boost compared to current implementation in SR.

Interesting.

It's around 7x~8x times boost compared to current implementation in SR.

Interesting.

@lemire does it match your expectation?

@kevincai  Indeed. It is within my expectations... meaning that I expect that your results are correct. Yet these are good results.


feat: add haveyoudebuggedit/gotestfmt

haveyoudebuggedit/gotestfmt: go test output for humans
$ aqua g -i haveyoudebuggedit/gotestfmt

Thank you for your contribution!

Published the prerelease https://github.com/aquaproj/aqua-registry/releases/tag/v3.55.0-0
We'll release v3.55.0 this night.

Released. https://github.com/aquaproj/aqua-registry/releases/tag/v3.55.0


chore: Deletes remaining sample files ending with postgre.

Last task of #2139
Documentation changes merged in https://critique.corp.google.com/cl/524816665

Reference #2139

Failure is unrealted and being tracked in #1623.


00log.txt should not be seen as an extension by Validate Database.

The Create Database tool writes a log text file to the gdb folder that records the version of the tool and the date of creation but the file is seen as a non-GeMS specified extension by the Validate Database. In fact, the report should probably be changed to display the contents of that file if it has any (some report the presence of an empty log file ?) and ignored otherwise.

Great, thanks Evan!


Status bar doesn't use SPAA

Testing #84715

I cannot reproduce when disabling custom title bar so I suspect something in the title bar makes the status bar go greyscale...

This is so random that I will not invest further, seems to work fine in some cases:

had the same experience, when testing with the custom menus enabled.


redis-ha-4.27.0 - split brain

Describe the bug
I deployed the chart with default values. During its explatation we met condition when redis-0 and redis-2 are replicas of redis-1, and redis-1 is replica of redis-0. The split-brain-fix container wasn`t able to fix the problem.
<IP_ADDRESS> -   redis-0
<IP_ADDRESS> - redis-1
<IP_ADDRESS> -   redis-2
redis-0:
  |   | 2024-06-18 18:23:36.849 | 1:S 18 Jun 2024 15:23:36.849 * Connecting to MASTER <IP_ADDRESS>:6379 |
  |   | 2024-06-18 18:23:36.849 | 1:S 18 Jun 2024 15:23:36.849 * MASTER <-> REPLICA sync started |
  |   | 2024-06-18 18:23:36.850 | 1:S 18 Jun 2024 15:23:36.850 # Error condition on socket for SYNC: Connection refused |
  |   | 2024-06-18 18:23:37.852 | 1:S 18 Jun 2024 15:23:37.852 * Connecting to MASTER <IP_ADDRESS>:6379 |
  |   | 2024-06-18 18:23:37.852 | 1:S 18 Jun 2024 15:23:37.852 * MASTER <-> REPLICA sync started

redis-1 (sentinel tries to restart it):
  |   | 2024-06-18 18:26:55.109 | 1:S 18 Jun 2024 15:26:55.109 * Ready to accept connections tcp |
  |   | 2024-06-18 18:26:55.109 | 1:S 18 Jun 2024 15:26:55.109 * Connecting to MASTER <IP_ADDRESS>:6379 |
  |   | 2024-06-18 18:26:55.110 | 1:S 18 Jun 2024 15:26:55.109 * MASTER <-> REPLICA sync started |
  |   | 2024-06-18 18:26:55.110 | 1:S 18 Jun 2024 15:26:55.110 * Non blocking connect for SYNC fired the event. |
  |   | 2024-06-18 18:26:55.111 | 1:S 18 Jun 2024 15:26:55.111 * Master replied to PING, replication can continue... |
  |   | 2024-06-18 18:26:55.113 | 1:S 18 Jun 2024 15:26:55.112 * Trying a partial resynchronization (request 8605e4e1a74e2a74a8ad3742efb5784ad4b0ce41:1). |
  |   | 2024-06-18 18:26:55.113 | 1:S 18 Jun 2024 15:26:55.113 * Master is currently unable to PSYNC but should be in the future: -NOMASTERLINK Can't SYNC while not connected with my master |
  |   | 2024-06-18 18:26:56.114 | 1:S 18 Jun 2024 15:26:56.113 * Connecting to MASTER <IP_ADDRESS>:6379 |
  |   | 2024-06-18 18:26:56.114 | 1:S 18 Jun 2024 15:26:56.114 * MASTER <-> REPLICA sync started

sentinel-1 (leader)
  |   | 2024-06-18 18:26:55.883 | 1:X 18 Jun 2024 15:26:55.883 * +reboot master mymaster <IP_ADDRESS> 6379 |
  |   | 2024-06-18 18:28:09.960 | 1:X 18 Jun 2024 15:28:09.960 # +new-epoch 21 |
  |   | 2024-06-18 18:28:09.960 | 1:X 18 Jun 2024 15:28:09.960 # +try-failover master mymaster <IP_ADDRESS> 6379 |
  |   | 2024-06-18 18:28:09.963 | 1:X 18 Jun 2024 15:28:09.963 * Sentinel new configuration saved on disk |
  |   | 2024-06-18 18:28:09.963 | 1:X 18 Jun 2024 15:28:09.963 # +vote-for-leader aa33680947f52ae19df761ea8f26a4285d4910c1 21 |
  |   | 2024-06-18 18:28:09.969 | 1:X 18 Jun 2024 15:28:09.969 * d4ca60ac0fa2353d3c6a5684df1401f8faccf6ef voted for aa33680947f52ae19df761ea8f26a4285d4910c1 21 |
  |   | 2024-06-18 18:28:09.969 | 1:X 18 Jun 2024 15:28:09.969 * d21ee95d5d45a94a9deb59bd2b2797a4bddedf53 voted for aa33680947f52ae19df761ea8f26a4285d4910c1 21 |
  |   | 2024-06-18 18:28:10.039 | 1:X 18 Jun 2024 15:28:10.039 # +elected-leader master mymaster <IP_ADDRESS> 6379 |
  |   | 2024-06-18 18:28:10.039 | 1:X 18 Jun 2024 15:28:10.039 # +failover-state-select-slave master mymaster <IP_ADDRESS> 6379 |
  |   | 2024-06-18 18:28:10.116 | 1:X 18 Jun 2024 15:28:10.115 # -failover-abort-no-good-slave master mymaster <IP_ADDRESS> 6379 |   |   | 2024-06-18 18:28:10.187 | 1:X 18 Jun 2024 15:28:10.187 * Next failover delay: I will not start a failover before Tue Jun 18 15:34:10 2024 |
  |   | 2024-06-18 18:32:53.938 | 1:X 18 Jun 2024 15:32:53.936 * +reboot master mymaster <IP_ADDRESS> 6379

split-brain-fix-1
  |   | 2024-06-18 18:20:30.025 | Could not connect to Redis at <IP_ADDRESS>:6379: Connection refused |
  |   | 2024-06-18 18:20:30.025 | Could not connect to Redis at <IP_ADDRESS>:6379: Connection refused |
  |   | 2024-06-18 18:21:30.027 | Identifying redis master (get-master-addr-by-name).. |
  |   | 2024-06-18 18:21:30.027 | using sentinel (hewi-redis-ha), sentinel group name (mymaster) |
  |   | 2024-06-18 18:21:30.043 | Tue Jun 18 15:21:30 UTC 2024 Found redis master (<IP_ADDRESS>) |
  |   | 2024-06-18 18:21:30.046 | Could not connect to Redis at <IP_ADDRESS>:6379: Connection refused |
  |   | 2024-06-18 18:21:30.049 | Tue Jun 18 15:21:30 UTC 2024 Start... |
  |   | 2024-06-18 18:21:30.057 | Initializing config.. |
  |   | 2024-06-18 18:21:30.057 | Copying default redis config.. |
  |   | 2024-06-18 18:21:30.057 | to '/data/conf/redis.conf' |
  |   | 2024-06-18 18:21:30.061 | Copying default sentinel config.. |
  |   | 2024-06-18 18:21:30.061 | to '/data/conf/sentinel.conf' |
  |   | 2024-06-18 18:21:30.063 | Identifying redis master (get-master-addr-by-name).. |
  |   | 2024-06-18 18:21:30.063 | using sentinel (hewi-redis-ha), sentinel group name (mymaster) |
  |   | 2024-06-18 18:21:30.083 | Tue Jun 18 15:21:30 UTC 2024 Found redis master (<IP_ADDRESS>) |
  |   | 2024-06-18 18:21:30.083 | Identify announce ip for this pod.. |
  |   | 2024-06-18 18:21:30.083 | using (hewi-redis-ha-announce-1) or (hewi-redis-ha-server-1) |
  |   | 2024-06-18 18:21:30.088 | identified announce (<IP_ADDRESS>) |
  |   | 2024-06-18 18:21:30.088 | Verifying redis master.. |
  |   | 2024-06-18 18:21:30.088 | ping (<IP_ADDRESS>:6379) |
  |   | 2024-06-18 18:21:30.091 | Could not connect to Redis at <IP_ADDRESS>:6379: Connection refused |
  |   | 2024-06-18 18:21:34.102 | Could not connect to Redis at <IP_ADDRESS>:6379: Connection refused |
  |   | 2024-06-18 18:21:39.125 | Could not connect to Redis at <IP_ADDRESS>:6379: Connection refused |
  |   | 2024-06-18 18:21:45.137 | Tue Jun 18 15:21:45 UTC 2024 Can't ping redis master (<IP_ADDRESS>) |
  |   | 2024-06-18 18:21:45.137 | Attempting to force failover (sentinel failover).. |
  |   | 2024-06-18 18:21:45.137 | on sentinel (hewi-redis-ha:26379), sentinel grp (mymaster) |
  |   | 2024-06-18 18:21:45.144 | Tue Jun 18 15:21:45 UTC 2024 Failover returned with 'NOGOODSLAVE' |
  |   | 2024-06-18 18:21:45.144 | Setting defaults for this pod.. |
  |   | 2024-06-18 18:21:45.144 | Setting up defaults.. |
  |   | 2024-06-18 18:21:45.144 | using statefulset index (1) |
  |   | 2024-06-18 18:21:45.144 | Getting redis master ip.. |
  |   | 2024-06-18 18:21:45.144 | blindly assuming (hewi-redis-ha-announce-0) or (hewi-redis-ha-server-0) are master |
  |   | 2024-06-18 18:21:45.161 | identified redis (may be redis master) ip (<IP_ADDRESS>) |
  |   | 2024-06-18 18:21:45.161 | Setting default slave config for redis and sentinel.. |
  |   | 2024-06-18 18:21:45.161 | using master ip (<IP_ADDRESS>) |
  |   | 2024-06-18 18:21:45.161 | Updating redis config.. |
  |   | 2024-06-18 18:21:45.162 | we are slave of redis master (<IP_ADDRESS>:6379) |
  |   | 2024-06-18 18:21:45.162 | Updating sentinel config.. |
  |   | 2024-06-18 18:21:45.162 | evaluating sentinel id (${SENTINEL_ID_1}) |
  |   | 2024-06-18 18:21:45.162 | sentinel id (aa33680947f52ae19df761ea8f26a4285d4910c1), sentinel grp (mymaster), quorum (2) |
  |   | 2024-06-18 18:21:45.163 | redis master (<IP_ADDRESS>:6379) |
  |   | 2024-06-18 18:21:45.164 | announce (<IP_ADDRESS>:26379) |
  |   | 2024-06-18 18:21:45.165 | Tue Jun 18 15:21:45 UTC 2024 Ready...

split-brain-fix-0
  |   | 2024-06-18 18:21:56.044 | using sentinel (hewi-redis-ha), sentinel group name (mymaster) |
  |   | 2024-06-18 18:21:56.052 | Tue Jun 18 15:21:56 UTC 2024 Found redis master (<IP_ADDRESS>) |
  |   | 2024-06-18 18:22:56.056 | Identifying redis master (get-master-addr-by-name).. |
  |   | 2024-06-18 18:22:56.056 | using sentinel (hewi-redis-ha), sentinel group name (mymaster) |
  |   | 2024-06-18 18:22:56.063 | Tue Jun 18 15:22:56 UTC 2024 Found redis master (<IP_ADDRESS>) |
  |   | 2024-06-18 18:23:56.067 | Identifying redis master (get-master-addr-by-name)..

To Reproduce
I tried node/pod deletion and redis-cli replicaof with no success to reproduce this bug
Expected behavior
split-brain-fix container should fix even this rare case
Additional context
The scripts logic was broken by inability of sentinel to failover. Maybe script should have additional condition to check the role of potential default master. I will be very apreatiate for any help with this. Please let me know if you need some additional logs/checks

+1

I've had this too.
I've found that when we've added a descheduler to the stack (https://github.com/kubernetes-sigs/descheduler) to balance nodes automatically, this kind of issue will disable the redis service frequently.
Can the master allocation be done with kubernetes lease locks? https://kubernetes.io/docs/concepts/architecture/leases/

@DandyDeveloper
Hi
I'm having problem when my network becomes a bit unstable (for example pods are not able to each other for a sec.) and my redis pods can't see each other

@tschirmer I'm trying to work out why this would happen unless the podManagementPolicy of the STS is set to Parallel?
Is this happening in either of your cases? @tschirmer ??
Because in theory, on first rollout, the first pod should start up and become master, way before -1/-2 start.

Haven't set it to Parallel. I suspect it would be something like, pod when evicted isn't completing the trigger-failover-if-master.sh. We are running it with sentinel, which might add some complexity here. I haven't debugged it yet.
So far we're getting a load of issues with the liveness probe not containing the SENTINELAUTH env from the secret, but it's clearly defined in the spec; and a restart of the pod works. It's happening very frequently though, so I'm wondering if there needs to be a grace period defined on startup and shutdown to prevent it both of these things from happening

I think being able to have separated  Statefulsets for redises and sentinels will make this chart more stable and manageable,
By creating two  Statefulsets and giving sentinel monitor config to monitor an external host

I like the idea of seperate stateful sets, I've been thinking of doing that and making a PR
I suspect this is from preStop hooks not firing and completely successfully. trigger-failover-if-master.sh occasionally doesn't run as expected. When we had the descheduler running it was ~2min between turning on and off each pod, and found that every now and again, that would fail. The rate of failure is low, so it's unlikely occur unless you're hammering it (we haven't had an issue with the ah cluster once we turned off the descheduler.

I wanted to make a PR too. But there are a lot of configs that should propagate this change

I found that there were a couple things wrong with my setup:

because the permissions for the readonly config was set to 420, no preStop hooks were being triggered.

we had a problem with our pvc csi driver that was attaching the same drive to any pvc with the same name in different namespaces.

The permissions were the killer, because nothing was failing over on shutdown.
I'm half way through writing a leader elector in golang for this based on k8s leases. Got it claiming the lease already. I'm not sure it's totally necessary after we've solved these other issues though.

specifically. In the stateful set the volume definitions here:
      volumes:         - configMap:             defaultMode: 420  ####THIS ONE  ensured that the preStop Hooks didn't have the permissions to run. Changed it to 430             name: storebase-redis-session-configmap           name: config         - hostPath:             path: /sys             type: ''           name: host-sys         - configMap:             defaultMode: 493             name: storebase-redis-session-health-configmap           name: health

Also found that that preStopHook:  /readonly-config/..data/trigger-failover-if-master.sh
requires SENTINELAUTH, but it's not defined in the env for the redis container
echo "[K8S PreStop Hook] Start Failover."
get_redis_role() {
  is_master=$(
    redis-cli \
      -a "${AUTH}" --no-auth-warning \
      -h localhost \
      -p 6379 \
      info | grep -c 'role:master' || true
  )
}
get_redis_role

echo "[K8S PreStop Hook] Got redis role."
if [[ "$is_master" -eq 1 ]]; then
  echo "[K8S PreStop Hook] This node is currently master, we trigger a failover."
  response=$(
    redis-cli \
      -a "${SENTINELAUTH}" --no-auth-warning \
      -h <IP_ADDRESS> \
      -p 26379 \
      SENTINEL failover mymaster
  )
  if [[ "$response" != "OK" ]] ; then
    echo "[K8S PreStop Hook] Failover failed"
    echo "$response"
    exit 1
  fi
  timeout=30
  while [[ "$is_master" -eq 1 && $timeout -gt 0 ]]; do
    sleep 1
    get_redis_role
    timeout=$((timeout - 1))
  done
  echo "[K8S PreStop Hook] Failover successful"
else
  echo "[K8S PreStop Hook] This node is currently replica, no failover needed."
fi

^I'd modified the above so I could get some debug data. Along with this in the stateful set:
          lifecycle:
            preStop:
              exec:
                command:
                  - /bin/sh
                  - '-c'
                  - >-
                    echo "running preStop" >> /proc/1/fd/1 &&
                    /readonly-config/trigger-failover-if-master.sh | tee >>
                    /proc/1/fd/1 &&  echo "finished preStop" >> /proc/1/fd/1

Found that running preStops would consistently fail.
running preStop
[K8S PreStop Hook] Start Failover.
[K8S PreStop Hook] Got redis role.
[K8S PreStop Hook] This node is currently master, we trigger a failover.
[K8S PreStop Hook] Failover failed

finished preStop

Found that the Sentinel container had shut down before the command could be executed on the localhost., so it kept getting a failover failed. Changed the sentinel preStop to add in a 10 sec delay to keep it alive while this happened and it seems to work every time now.
          lifecycle:
            preStop:
              exec:
                command:
                  - /bin/sh
                  - '-c'
                  - >-
                    sleep 10

Found that running preStops would consistently fail.
running preStop
[K8S PreStop Hook] Start Failover.
[K8S PreStop Hook] Got redis role.
[K8S PreStop Hook] This node is currently master, we trigger a failover.
[K8S PreStop Hook] Failover failed

finished preStop

Found that the Sentinel container had shut down before the command could be executed on the localhost., so it kept getting a failover failed. Changed the sentinel preStop to add in a 10 sec delay to keep it alive while this happened and it seems to work every time now.
          lifecycle:
            preStop:
              exec:
                command:
                  - /bin/sh
                  - '-c'
                  - >-
                    sleep 10

While this "might" work, it "may not" be consistent, suggest taking a look at my solution instead here: https://github.com/DandyDeveloper/charts/issues/207#issuecomment-1827134022


GP migration PO fixes

This change fixes multiple issues found while attempting to migrate open Purchase Orders during a GP migration.
Fixes

Prevent migrating the PO if the Vendor Id is empty (bad data)
Better line over-receipt support
Prevent migrating line if Quantity < 1
Line count fixes
Create a Purchase Receipt and related posting records to prevent a failure when later manually posting the PO

Resubmitting with different design direction.


[WIP] Introduce binary type

This approaches #420.
The largest problem is that many plugins will be not compatible when this new type is introduced.
I would like to discuss how the binary type should behave, and how to merge this into mainline. Thanks, all.

Really looking forward to this feature being added.

I cannot commit to this feature any more. I really appreciate if someone take over to be merged!

To avoid unexpected LinkageError, we should upgrade Java8 before binary type support.

We're going to introduce Binary type in v0.9, which is expected to be released in a couple of months.
Thanks to this PR, the first implementation would be easy. But I'm afraid we may be going to make another PR to introduce Binary type since:

It's conflicting with the current master at Git's level. (It's because we took much time to do it. Sorry about that.)
Some interfaces have been changed in Embulk. (Ditto. Sorry.)
We'd use another class for Binary (java.nio.ByteBuffer or Embulk's own class), instead of byte[]. (byte[] is not very good to use there -- it's mutable, and too primitive.)

Let me take this opportunity to appreciate @tomykaira 's effort on this. Thanks for your contribution!

New commits to be made in #850.


All routes return 404 when deploying on server

Im trying to upload both Laravel as a backend And Nuxt 2.11 as frontend (universal mode) into a server but after upload every routes returns laravel 404, Im am also using nginx reverse proxy (production mode)
Upload Steps
1- upload backend files and folders ( app,bootstrap,client,config,database,routes,storage,vendor,package.json and lock,composer.json and lock)
2-upload frontend files and folders(.nuxt, and everything insdie my client folder)
3-In my server Ive added a new line at the end of my nginx.conf include /etc/nginx/sites-enabled/*.conf;
4- then in /etc/nginx/sites-enabled I have default.conf with the following content
server {
  # server on port 80 (default http port)
  listen 80;
  server_name rabter.com;

  # proxy for frontend
  location / {
    # nuxt server url
    proxy_pass http://localhost:3000;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_set_header Host $host;
    proxy_cache_bypass $http_upgrade;
  }

  # proxy for api
  location /api/* {
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_set_header X-NginX-Proxy true;
    # laravel server url
    proxy_pass http://localhost:8000;
    proxy_redirect off;
  }
}

5- Ill setup pm2 server so I run pm2 start Laravel-nuxt to start listening to port 3000 ( lunching nuxt)
At this point I access my site which returns 404 on all routes no exception I also came across this https://github.com/iliyaZelenko/laravel-nuxt/issues/1#issuecomment-491484474 which I think it essentially what I want to do even tried the nginx code but didnt work
I have 0 routes in my web.php and all the routes are in api.php.Also this project works fine on localhost both dev and prod mode but when I move to the server its all 404.

Also when Im running nuxt run start thsi message keeps repeating, like its in infinity loop
Request failed with status code 502 12:30:02   at createError (node_modules/axios/lib/core/createError.js:16:15)   at settle (node_modules/axios/lib/core/settle.js:17:12)   at IncomingMessage.handleStreamEnd (node_modules/axios/lib/adapters/http.js:236:11)   at IncomingMessage.emit (events.js:322:22)   at IncomingMessage.EventEmitter.emit (domain.js:482:12)   at endReadableNT (_stream_readable.js:1187:12)   at processTicksAndRejections (internal/process/task_queues.js:84:21) (repeated 0 times)

Hi,
I was facing this issue as well while deploying my website which uses this repo. I wasn't able to configure ngnix or apache in way that /api calls would work. Also laravel forge couldn't set it up.
Solution for me was to setup completely new subdomain for api calls. api.domain.com. Then you setup whole apache/ngnix.conf file on your server. This works for me. Thing is that you have to modify few files in the project. RouteServiceProvider and api.php file if needed. There are many guides on the topic of how to user subdomain for api routing in laravel.


246 user defined thruster groups

User-defined thruster groups (THGROUP_USER+x with x >= 0) should now be fully supported by Vessel class methods and thruster-group related API functions.
Some cleanup of code in Vessel class; converted thruster and thruster-group related lists to std::vector and std::array
GetThrusterGroupLevel called with THGROUP_USER should no longer crash (see #235)

Closes #246

@spacex15 : It would be helpful if you can check if this pull request solves the problem identified in #235 (GetThrusterGroupLevel crashes with THGROUP_USER).

@mschweiger I can confirm that calling GetThrusterGroupLevel(THGROUP_USER) no longer crashes and returns the thruster level for the first user group
but I do not understand how to call GetThrusterGroupLevel for other id groups
calling GetThrusterGroupLevel(THGROUP_USER+ groupindex )? (does not compile)
add another argument for the user group index ?

@spacex15 : you are correct, since the THGROUP_USER member of the THGROUP_TYPE enum was the upper bound of the enum, no valid addition was possible. I have now added an additional member to the enum to provide an extended upper bound. This should now allow constructs like this:
THGROUP_TYPE MyThrusterGroup = (THGROUP_TYPE)(THGROUP_USER+1);

which then allows to use MyThrusterGroup anywhere a THGROUP_TYPE is expected. Does this work for you?

Yes @mschweiger , I tried GetThrusterGroupLevel((THGROUP_TYPE)(THGROUP_USER + indexgroup))
and it works now
thanks


2 cams .... same rtsp port ?

when i use 2 cams with this plug-in, should i use different rtsp ports for the two cams, or does this no matter ?

"plugin" already manages that, you can't choose which port but it uses different ports for different cameras
I currently run multiple cams without problems.


tests: Always check if elements are visible

wait_present should be used just rarely as the meaning of this
function is "Wait until present in dom", while meaning of wait_visible
is "Wait until in dom and visible to user". Tests should really just
check what user sees. Using wait_present incorrectly can lead to
broken code (just now I was fixing accordion which would never open up,
but tests were happy as they were just checking with wait_present)
I haven't tried this, just sed it. Lets see what CI think :)

10 failing tests, seems that mostly on incorrect using of selects (not opening the select, just interacting with it). We have functions to do this properly in test/common/testlib.py so I'll take this as an opportunity to rewrite it into that :)

List of image types is fetched asynchronously. UI does not indicate this in any way, which is not great. But I think we can just simply check for it in tests.


make light/dark mode theme persist on refresh

Description
Fixed the onClick behavior of the Dark/Light Mode toggle so that the theme persists on refresh.
Related Issues
[#1057]
Changes Proposed
Fixed the behavior of  toggleTheme() within Sidebar.jsx to make use of our useTheme hook, and removed redundant lines of code within toggleTheme.
Now that we're properly using useTheme,  we don't need to explicitly write things like:
htmlElement.classList.remove('dark');
setTheme('light');

because these changes happen dynamically for us already in other parts of the code.
Checklist

[x] I have read and followed the Contribution Guidelines.
[x] All new and existing tests passed.
[x] I have updated the documentation to reflect the changes I've made.
[x] My code follows the code style of this project.
[x] The title of my pull request is a short description of the requested changes.

Screenshots
Note to reviewers

There is another PR that may technically fix this issue, but it doesn't make use of the useTheme hook, and instead just replicates it's functionality within toggleTheme.


fix: GoogleService-Info.plist file not found by Firebase SDK unless added to target

Description
I noticed whilst debugging this issue that initialization was occurring from the dart config file, not the GoogleService-Info.plist file. It appears Firebase will not pick up the file unless it is added to the target.
I also think we should stop writing the config to the DefaultFirebaseOptions class for android & iOS. Unnecessary and, I think, confusing to users.
Type of Change

[ ] ✨ feat -- New feature (non-breaking change which adds functionality)
[X] 🛠 fix -- Bug fix (non-breaking change which fixes an issue)
[ ] ❌ ! -- Breaking change (fix or feature that would cause existing functionality to change)
[ ] 🧹 refactor -- Code refactor
[ ] ✅ ci -- Build configuration change
[ ] 📝 docs -- Documentation
[ ] 🗑 chore -- Chore

@Salakar I've updated to abort() ruby script (exit code 1 and writes to stderr), then throw exception in CLI to let user know the issue.
Example:

I tried to fix this here
https://github.com/invertase/flutterfire_cli/pull/88


Install php-bcmath

The php7 distribution on xenial does not include bcmath, which the SDK uses.

https://ocd.appscale.com:8080/job/Daily Build/4656/


Linux port

Make all DirectDraw code optional.
Create a compat GUI for Linux (probably GTK3, or GTK4, via Glade, because that's what I know already)

This has to be the most exciting news in the field of trippy-audio under GNU/linux i read in a long long time!!!
can't wait!


fs: remove macOS-specific copyfile(3)

Using copyfile(3) on macOS apparently results in situations where file
permissions are ignored. Using the same code for other UNIX-like
operating systems seems to fix the issue.
Refs: https://github.com/nodejs/node/issues/26936#issuecomment-476992597
@cjihrig

(Seems like this should get a test, but I don't know enough about libuv's tests to write one quickly so I'm going to sleep now. If someone else writes the test or leaves me hints or links, awesome. I did add a test to Node.js for this. If that's sufficient, great.)

Please add a test case in test/test-fs-copyfile.c with the reproduction from the Node issue. It should fail using the macOS code and pass with your changes.

Do we generally report these sort of bugs to the respective projects?

Test added. PTAL.

(The test fails for me on Mojave with current libuv src but passes with the changes in this PR.)

Not sure if this is the right way to run CI on libuv, but it seems like it probably is?: https://ci.nodejs.org/view/libuv/job/libuv-test-pipeline/49/

Not sure if this is the right way to run CI on libuv

I generally use https://ci.nodejs.org/view/libuv/job/libuv-test-commit/. And, if you want to test in the context of Node's test suite: https://ci.nodejs.org/view/libuv/job/libuv-in-node/

CI: https://ci.nodejs.org/view/libuv/job/libuv-test-commit/1329/
Other CI: https://ci.nodejs.org/view/libuv/job/libuv-in-node/86/
Looks like my test doesn't compile on Windows?

Looks like my test doesn't compile on Windows?

Hi! Welcome to libuv 👋
On a more serious note, I think you can #ifdef the test out on Windows (this is done in other tests like test/test-pipe-set-fchmod.c) because the permissions don't really transfer over there.

OK, used #ifndef to skip the test on Windows. CI runs again:
https://ci.nodejs.org/view/libuv/job/libuv-test-commit/1330/
https://ci.nodejs.org/view/libuv/job/libuv-in-node/87/

The libuv CI is OK. The failures are https://github.com/libuv/libuv/pull/2234. The failures in the Node CI seem unrelated, although it would be preferable if macOS passed there.

The failures in the Node CI seem unrelated, although it would be preferable if macOS passed there.

Resume Build CI: https://ci.nodejs.org/view/libuv/job/libuv-in-node/88/

The failures in the Node CI seem unrelated, although it would be preferable if macOS passed there.

Couldn't figure out how to quickly/easily rebuild just macOS, so here's a full re-run to see what happens:
https://ci.nodejs.org/view/libuv/job/libuv-in-node/89/

(CI re-run is yellow, which I imagine is close-enough to green.)

Now that https://github.com/nodejs/node/pull/26939 landed in Node, I ran the libuv+Node integration CI again. I would expect the new test in that PR, test/known_issues/test-fs-copyfile-respect-permissions.js, to fail on macOS when built with this PR. That's not what I'm seeing though.
Here is the new CI run https://ci.nodejs.org/view/libuv/job/libuv-in-node/90/ where macOS comes back green.

I think https://github.com/nodejs/node/pull/27024 addresses the issue, meaning this is still 👍

Now that nodejs/node#26939 landed in Node, I ran the libuv+Node integration CI again. I would expect the new test in that PR, test/known_issues/test-fs-copyfile-respect-permissions.js, to fail on macOS when built with this PR. That's not what I'm seeing though.
Here is the new CI run https://ci.nodejs.org/view/libuv/job/libuv-in-node/90/ where macOS comes back green.

There's a typo in the test where it is expecting EACCES when it should be expecting EACCESS.   😞 I'll submit a PR to fix.

And there's an additional bug hidden by that one. I knew I should have resisted the nit to add every variant into the test. :-D

I went to submit my PR to fix the issues, but you beat me to it. :-D

Anything more for me to do before this is merge-able?

The integration CI looks good. The only failure was test/known_issues/test-fs-copyfile-respect-permissions.js, which is expected because it's a known issue that gets fixed by this PR. I'll move that test once I update Node with the new libuv release.

Landed in d0fb86cd...c7b87b0d. Thanks, and congrats on your first commit(s).

This seems to remove support for COPYFILE_FICLONE and COPYFILE_FICLONE_FORCE on macOS. I get an ENOSYS error when using the later in node 12. Would be nice to be able to optionally use copy on write for fast copying without shelling out to cp -c.

@devongovett I agree that this change may be regressive. Since we are doing the chmod stuff anyways on the normal path, maybe I can try restoring it. As long as the test passes we are in the clear.