Running docker-mailserver on portainer.io self hosted?

My goal after extensive searching is to deploy the docker-mailserver on a docker compatible OS.
Since I need monitoring and magagement UI I found out Portainer.io is the best option to proceed.
Has anyone setup docker-mailserver on a portainer instance locally?
And is anyone willing to share some details on how to approach this best?
I have currently running portainer and a local docker registry which is setup in portainer.
The docker-mailserver runs as an image inside this.
Any thoughts from the team or anyone with first hand experience in this regard?
Could be an interesting method and im willing to fully document my processes as a a contribution to the wiki entry for further reference if anyone can point me in the right direction!
Kind regards.
The Portainer UI allows to add the docker-compose.yml so that would be fixed.
It allows adding ENV but only 1 by 1 the complete list without comments is:
HOSTNAME=mail
DOMAINNAME=domain.com
OVERRIDE_HOSTNAME=
DMS_DEBUG=0
ONE_DIR=0
POSTMASTER_ADDRESS=
PERMIT_DOCKER=
TLS_LEVEL=
SPOOF_PROTECTION=
ENABLE_SRS=0
ENABLE_POP3=
ENABLE_CLAMAV=0
ENABLE_FAIL2BAN=0
ENABLE_MANAGESIEVE=
POSTSCREEN_ACTION=enforce
SMTP_ONLY=
SSL_TYPE=
VIRUSMAILS_DELETE_DELAY=
ENABLE_POSTFIX_VIRTUAL_TRANSPORT=
POSTFIX_DAGENT=
POSTFIX_MAILBOX_SIZE_LIMIT=
POSTFIX_MESSAGE_SIZE_LIMIT=
REPORT_RECIPIENT=0
REPORT_SENDER=
REPORT_INTERVAL=daily
ENABLE_SPAMASSASSIN=0
SA_TAG=2.0
SA_TAG2=6.31
SA_KILL=6.31
SA_SPAM_SUBJECT=***SPAM*****
ENABLE_FETCHMAIL=0
FETCHMAIL_POLL=300
ENABLE_LDAP=
LDAP_START_TLS=
LDAP_SERVER_HOST=
LDAP_SEARCH_BASE=
LDAP_BIND_DN=
LDAP_BIND_PW=
LDAP_QUERY_FILTER_USER=
LDAP_QUERY_FILTER_GROUP=
LDAP_QUERY_FILTER_ALIAS=
LDAP_QUERY_FILTER_DOMAIN=
DOVECOT_TLS=
DOVECOT_USER_FILTER=
DOVECOT_PASS_FILTER=
ENABLE_POSTGREY=0
POSTGREY_DELAY=300
POSTGREY_MAX_AGE=35
POSTGREY_TEXT=Delayed by postgrey
POSTGREY_AUTO_WHITELIST_CLIENTS=5
ENABLE_SASLAUTHD=0
SASLAUTHD_MECHANISMS=
SASLAUTHD_MECH_OPTIONS=
SASLAUTHD_LDAP_SERVER=
SASLAUTHD_LDAP_SSL=
SASLAUTHD_LDAP_BIND_DN=
SASLAUTHD_LDAP_PASSWORD=
SASLAUTHD_LDAP_SEARCH_BASE=
SASLAUTHD_LDAP_FILTER=
SASLAUTHD_LDAP_START_TLS=
SASLAUTHD_LDAP_TLS_CHECK_PEER=
SASL_PASSWD=
SRS_SECRET=
DEFAULT_RELAY_HOST=
RELAY_HOST=
RELAY_PORT=25
RELAY_USER=
RELAY_PASSWORD=

So this is quite the work and error prone,
Then we have the issue with the setup.sh . @tomav do you think I could setup the docker container according to your docker then lateron hook it to portainer?
As far As I see the portainer is just a wrapper and I could follow the regular documentation steps provided in your documentation. Then only having to 'import' the container in portainer.io would be the most simple option, also providing the setup.sh option to add accounts.
Any thoughts on above?
Im more than willing to document every step to get it working to the WIKI as a thank you for creating the server and the future reference if anyone could point me in the right direction.

So far I found this blogpost:
https://foundries.io/insights/2018/05/08/using-docker-compose.yml-with-portainer/
But I think there will be some work on to make this compatible.
For example the setup.sh script needs to be run from somewhere
Any thoughts @tomav did you ever came across portainer and could point me in right direction on how to approach on this?

Hi,
ultimately Portainer is just a web gui over everything you can do through the commandline already. So if you are struggeling to start this container through the gui you could also start it from the cli and then use the gui to check on its status every once in a while.

But I think there will be some work on to make this compatible.
For example the setup.sh script needs to be run from somewhere

Your linked blog also includes a solution for this. What they recommend is to create a small intermediary container that is then used to execute docker-compose. You could do the same with a checkout of this git repo and then also execute the setup script from inside this container.

Like @fbartels stated except for the setup.sh there are no problems


postgresql11Packages.timescaledb: 2.0.1 -> 2.0.2

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/timescale/timescaledb/releases.
meta.description for postgresql11Packages.timescaledb is: "Scales PostgreSQL for time-series data via automatic partitioning across time and space"
meta.homepage for postgresql11Packages.timescaledb is: "https://www.timescale.com/"
meta.changelog for postgresql11Packages.timescaledb is: ""
Updates performed

Version update

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 2.0.2 with grep in /nix/store/r4bkar9phd4vvcljdjnh4r5dbjjw9dzc-timescaledb-2.0.2

directory tree listing: https://gist.github.com/b11fb926ef73161cf095177a637d4fc3

du listing: https://gist.github.com/064fc3c170b08559bb61fcc122c62388

Rebuild report (if merged into master) (click to expand)

8 total rebuild path(s)

2 package rebuild(s)

2 x86_64-linux rebuild(s)
2 i686-linux rebuild(s)
2 x86_64-darwin rebuild(s)
2 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
postgresql11Packages.timescaledb
postgresqlPackages.timescaledb

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/r4bkar9phd4vvcljdjnh4r5dbjjw9dzc-timescaledb-2.0.2 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A postgresql11Packages.timescaledb https://github.com/r-ryantm/nixpkgs/archive/c83289c7d4c3fdebfa332730877563d6f42dd132.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/r4bkar9phd4vvcljdjnh4r5dbjjw9dzc-timescaledb-2.0.2
ls -la /nix/store/r4bkar9phd4vvcljdjnh4r5dbjjw9dzc-timescaledb-2.0.2/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

timescaledb (postgresql11Packages.timescaledb)

Maintainer pings
cc @volth @marsam for testing.

Result of nixpkgs-review pr 114109 at c83289c7 run on aarch64-linux 1

1 package built:

postgresqlPackages.timescaledb (postgresql11Packages.timescaledb)

1 suggestion:

warning: patch-phase
patchPhase should not be overridden, use postPatch instead.
Near pkgs/servers/sql/postgresql/ext/timescaledb.nix:28:3:
   |
28 |   patchPhase = ''
   |   ^


It should be possible for the user to choose a reportee when login

Description
Go to https://platform.at22.altinn.cloud/accessmanagement/
When the user is not logged in, the user is redirected to login methods. When the user is logged in, user is redirected to https://platform.at22.altinn.cloud/accessmanagement.
We have to force the user to select a reportee?
Additional Information
Tasks

[ ] Redirect to choose reportee page?

Acceptance Criterias

[ ] TBD

#256


Reveal z/x/y for a tile when debugging

There should be some way to reveal the z/x/y for a tile when debugging. This should be in a form that the triplet can be copied and pasted so it can looked at with another program.

This can be done by adding ?debug=true to the viewer's URL or by appending ?debug=true to the tile requests. A few links for reference: Client Side Debugging, Debugging.
I will add some additional documentation about how to do this in the viewer, or maybe we should add a button to the viewer to make it even easier. ;-)

should add a button to the viewer to make it even easier

Yes - although there's probably a better way than editing the vector tiles to do it within the front-end itself.


CVE-2022-24713

SUSE Security has requested an update to address CVE-2022-24713. Upstream commit.
Please take the time to adjust Cargo.lock and upgrade at least regex.
Thank you.

I've merged #617
Thanks for the heads up @jubalh


Something with statistics

Request BBI: Relevant functions from the field of statistics are still missing. Name contact person at BBI.

@cssbue please provide more information

This ticket has no specific content. :joy:


bazel: Add rules_pkg

Signed-off-by: Ryan Northey<EMAIL_ADDRESS>Commit Message: bazel: Add rules_pkg
Additional Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Deprecated:]
[Optional API Considerations:]

/lgtm deps


build(deps): Bump github.com/canonical/edgex-snap-hooks/v2 from 2.2.-beta.5 to 2.2.0 in /snap/local/hooks (Jakarta)

Bumps github.com/canonical/edgex-snap-hooks/v2 from 2.2.0-beta.5 to 2.2.0.
Signed-off-by: Farshid Tavakolizadeh<EMAIL_ADDRESS>

If your build fails due to your commit message not passing the build checks, please review the guidelines here: https://github.com/edgexfoundry/device-gpio/blob/main/.github/Contributing.md
PR Checklist
Please check if your PR fulfills the following requirements:

[x] I am not introducing a breaking change (if you are, flag in conventional commit message with BREAKING CHANGE: describing the break)
[x] I am not introducing a new dependency (add notes below if you are)
[x] I have added unit tests for the new feature or bug fix (if not, why?)
[x] I have fully tested (add details below) this the new feature or bug fix (if not, why?)
[ ] I have opened a PR for the related docs change (if not, why?)

Testing Instructions

New Dependency Instructions (If applicable)

Put on hold. We may want to instead downgrade to module to 2.1.x for Jakarta.


Right-click unselecting structure does not react on sidebar

Selecting a structure changes content of sidebar. Unselecting structure (having no structure selected) should render nothing on sidebar.

after testing, I do like the fact the sidebar stays 'filled'. Having an empty sidebar looks weird. Will not fix this and if it comes up again re-open this issue (or create new one).


Incorrect behavior for pathlib.Path.resolve() when path does not exist

In the following test suite, the test that does not use the fs fixture succeeds while the test that does use the fs fixture fails:
import pathlib

import pytest

def test_pathlib_resolve_without_fs():
    path = pathlib.Path('/var/foo/bar/baz')

    assert path.resolve() == path

@pytest.mark.usefixtures('fs')
def test_pathlib_resolve_with_fs():
    path = pathlib.Path('/var/foo/bar/baz')

    assert path.resolve() == path

Here is a snippet of the results:
===================================================== FAILURES =====================================================
___________________________________________ test_pathlib_resolve_with_fs ___________________________________________

>   ???
E   AssertionError: assert PosixPath('/var') == PosixPath('/var/foo/bar/baz')
E    +  where PosixPath('/var') = <bound method FakePath.resolve of PosixPath('/var/foo/bar/baz')>()
E    +    where <bound method FakePath.resolve of PosixPath('/var/foo/bar/baz')> = PosixPath('/var/foo/bar/baz').resolve

From the documentation (https://docs.python.org/3/library/pathlib.html#pathlib.Path.resolve):
If strict is False, the path is resolved as far as possible and any remainder is appended without checking whether it exists.

A similar bug existed in Python itself, but was fixed in version 3.6.2.
Environment
OS: CentOS Linux release 7.3.1611 (Core)
Python: 3.6.5
pyfakefs: 3.4.1

Interestingly enough, there had been a test in place for this, but it had an error - instead of the intended assertEqual it used assertTrue to check for the resolved name. Urgh... Together with the fact that the implementation of resolve was directly taken from Python (which was buggy for Python at the time) this caused the problem.

Thanks for being so responsive!


Component to support enable field to skip installation

@joaquimrocha proposed this request that currently in Lokomotive when applying cluster with lokoctl , we either execute lokoctl cluster apply by installing the cluster and all the configured components or skip all the components.
It would be nice if all the components supported enabled field, which if set as false skips installing that component.

Maybe we can make --skip-components be allowed to be defined multiple times and specify list of components to skip there?

What would be the point of skipping given component all the time @joaquimrocha?

It'd be for e.g. easily keeping the config for that component but not apply it until needed. Say you want to provide an example configuration but leave the decision of applying the component to the user, then it'd be nice to say "just enable the component".
I did suggest this before knowing that we can have components separated into their own config file, which kind of makes it easier to accomplish the same effect.
It's not blocking anything ATM so there's no urgency from my side, and I am understand if you decline it.

Say you want to provide an example configuration but leave the decision of applying the component to the user, then it'd be nice to say "just enable the component".

The components already have default configuration built-in. And act of "enabling the component is to create a component block, or uncomment it, following the declarative specification of desired state. I'm not sure if adding concept of enabling on top of it make much sense.

I did suggest this before knowing that we can have components separated into their own config file, which kind of makes it easier to accomplish the same effect.

Yes, you can have component configs in separate files as well :+1: But by default we consume all config files in working directory, so I don't think they can be so-easily disabled. But my guts feeling is that using files as an interface here is simple and effective.


Changelog

Would be nice to have a changelog, either in the main ux repository (root dir or in each component) or in Github's releases page.

Should happen with the release process.

Should happen with the release process.


Add files

In this milestone the following were implemented:

Created a to-do-list project

Hey great job so far on your project, however there are still some changes needed.
It would be better to add a working link to your project in the README file. I suggest that you deploy with github pages


Does not take in to account fractions of a percent

As I point out in this post (https://bitcointalk.org/index.php?topic=5443953.msg61879707#msg61879707), fractions of a percent are not accounted for and therefore the calculated percentage risk is often inaccurate.
They are accounted correctly for in the graph produced, however.

Right, this is a result of me taking the form input and running is through parseInt which drops off all decimal precision. I'll change that to parseFloat and it should resolve the discrepancy.


Add delay for specific port

Feature Request
Describe the feature you'd like:
We have different protocol in each pod (http, grpc, db, etc.) and we want to affect each one in different steps to see how system response.

We can implement this through netem filter feature (this can also support delay on specific ipset). However, managing multiple netem rules is quite complicate.
A temporary solution is to split  one pod into multiple pod (with different labels). I think running so many different applications in one pod is not a suggested practice.

It is the same application with different interfaces (http, grpc, jms, jdbc, etc) and we want simulate saturation in one of them.

It is the same application with different interfaces (http, grpc, jms, jdbc, etc) and we want simulate saturation in one of them.

Maybe we can add another type of chaos(such as port chaos).  but it actually make the rule more complex. @cwen0 what do you think

@pcod80 We discuss it internally. Now we are lack resources to do the task recently.  But you can add a new type of chaos by yourself. following the network chaos code and development guide. We very appreciate that if you can do this

duplicated #295


Divider height

Currently, the library has a dividerWidth property, which is similar to the thickness of a divider. However, the term is misleading, as it actually refers to the width of the line, not the separator itself.
Are there plans to introduce a property similar to Divider height, which would allow us to set the actual width of the divider, potentially making it larger than the divider's size?

This has been added in #26 - however, I used the terminology thickness to refer to the thickness of the visible line and size to refer to the total space taken up by the divider. Does that work?
I didn't want to use height or width because the divider can be horizontal or vertical, so the best I could come up with was size.

This was included with release 1.0.0


METRON-1665 Host UIs on Spring Boot and add SSO authentication

This is part of a move towards using KnoxSSO, and LDAP as an authentication method for Metron UIs.
The current UIs are served up by very lightweight nodejs applications, which serve the static bundle files produced by the angular build process, and proxies the rest api. This PR replaces this with Spring Boot applications, using the Spring Security stack.
It also provides a shared project used as the base for the UI host application, which ensures consistent security across all UIs (metron-interface/metron-ui-security) an abstract web hosting application with zuul embedded for proxying rest to avoid cors problems (metron-interface/metron-ui-host) and two further concrete applications to do the actual hosting, the angular front-ends have been subtley changed to package themselves in jars which are included in the hosting applications, aiding the ease and integrity of the front end code on deployment.
Some tests in the rest api have been changed slightly, and a fix has been applied to AlertsUiController to allow users access to delete their own settings. The metron-rest project has also been adapted to work with metron-ui-security, and so inherits the same authentication benefits.
A follow on PR will address changes to the mpack to implement necessary config changes.
Nodejs has been removed from the RPMs, and the deployment process, since it is no longer needed beyond the build stage.
An additional feature is the filter provided to allow JWT cookies to be passed to a backend through zuul as Authorization Bearer headers, allowing simpler stateless interaction with the rest api for example, which opens the possibility of deploying each layer in a load balanced or HA mode using ribbon config.
To test at this stage, you will need to manually configure and launch the metron-alerts and metron-config jars, which behave like init.d scripts. This can be done by launching the jars through the bin/ scripts, but would be a lot easier with the follow on work on mpack elements to configure them. For now, integration tests have been included for key function on the security layer and should be reviewed.
Pull Request Checklist
Thank you for submitting a contribution to Apache Metron.
Please refer to our Development Guidelines for the complete guide to follow for contributions.
Please refer also to our Build Verification Guidelines for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
For all changes:

[x] Is there a JIRA ticket associated with this PR? If not one needs to be created at Metron Jira.
[x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
[x] Has your PR been rebased against the latest commit within the target branch (typically master)?

For code changes:

[x] Have you included steps to reproduce the behavior or problem that is being changed or addressed?

[x] Have you included steps or a guide to how the change may be verified and tested manually?

[x] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
mvn -q clean integration-test install && dev-utilities/build-utils/verify_licenses.sh

[x] Have you written or updated unit tests and or integration tests to verify your changes?

[x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

[x] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?

For documentation related changes:

[ ] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via site-book/target/site/index.html:
cd site-book
mvn site

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
It is also recommended that travis-ci is set up for your personal repository such that your branches are built there before submitting a pull request.

I really like this feature and think it's an important improvement.  This PR is a great start and I don't see any major architectural issues.  I've added some comments inline for the really minor issues.  I noticed that application.yml in metron-config-host and metron-alerts-host is different between the 2 but I can't think of any reason why.  I didn't comment directly but it's probably worth reviewing.
There are some bigger issues I want to call out that are more complex.
I noticed there is a fairly significant gap with respect to groups and roles.  As far as I can tell it's not supported right now which is a regression from the JDBC solution currently in place.  I'm sure this is expected and will be solved in follow on tasks in this feature branch.  I do think we should set ourselves up to be able to do that in future PRs though.
In this PR, metron-ui-host and metron-rest use 2 different authentication mechanisms for testing.  I feel they should use the same one and we should eliminate JDBC or in memory authentication all together.  To do this we need to move this testing infrastructure (the EmbeddedLdap class and supporting files, Spring properties, etc) to metron-security.  This will allow us to switch REST to a common security model during testing, which is essentially what we're doing with this feature.  I've submitted a PR with an example of how we could do that (switching REST to embedded ldap isn't included yet and should be done later to avoid conflicts with other PRs).  This will allow us to eliminate redundant properties across modules but I think the biggest benefit is that our testing will more accurately match a real installation and allow us to catch bugs sooner.
I think it's fair to say we need some documentation.  I would say at a minimum the metron-rest README should be updated to reflect the new authentication approach.  Eventually it would be nice to have architectural level documentation because there are several new components/frameworks being introduced here.  I have no problem with that being done in future PRs.
I also noticed there are several TODOs.  We should at some point document these as follow on tasks so we have a clear understanding of what is expected to work in each PR.

Here is what I think is pending on this PR:

Address the minor comments I've made inline
Update the metron-rest and/or other relevant READMEs with the new authentication approach
Create a follow on Jira to implement a simple LDAP group to role mapping so that we're not introducing a regression
Create a follow on Jira to implement roles properly (would that be outside of this feature branch?)
Add architecture documentation either in this PR or create a follow on Jira for it
Create a follow on Jira to convert metron-rest to use the testing infrastructure in metron-security

Thanks @simonellistonball.  I think this is good enough to start with.  +1

This has been merged into the feature branch.  Can you close @simonellistonball?


With the latest update (1.95.0) there's a copilot icon in the status bar which is essentially an advertisement.
For people who don't use copilot (or generally AI) in their workflow that icon doesn't have any purpose, sits in plain sight all the time and is generally annoying.
Please add a way to remove it.

Ok for anybody else looking for that option:
Turns out the icon can be removed, just not via the regular VSCode Settings:
Right click and then turn off Chat Controls:

This should really be much more obvious.


-webkit- browser issue

Hello,
Here I am facing something different issue in iPhone and iPad in all browsers. Whenever I am open popup and in that popup I have two text boxes, whenever I am typing anything in those text boxes at a time not displaying any character or digit after few seconds displaying that it means blink that whenever typing. I am not getting what's wrong with this.
anyone help me in this please?
Thanks

Can you provide a reduced test case with just Bootstrap code?

Hey there!
We're automatically closing this issue since the original poster (or another commenter) hasn't yet responded to the question or request made to them 14 days ago. We therefore assume that the user has lost interest or resolved the problem on their own. Closed issues that remain inactive for a long period may get automatically locked.
Don't worry though; if this is in error, let us know with a comment and we'll be happy to reopen the issue.
Thanks!
(Please note that this is an automated comment.)


fix(model): InteractionMember add missing props

This PR adds the permissions, pending and avatar properties to the InteractionMember struct.
Reference: https://discord.com/developers/docs/interactions/receiving-and-responding#interaction-object-resolved-data-structure

These will need to be sorted alphabetically where possible.

Ah yea missed those


The content in "Prerequisites" section seems to be incorrect.

Is the method for the site owner role to Grant SharePoint Online site permission to the application correct? The page currently says:
Permission Request XML
For the site owner role, the Permission Request XML is:
 <AppPermissionRequests>
    <AppPermissionRequest Scope="http://sharepoint/content/sitecollection/web" Right="Read"/>
 </AppPermissionRequests>

Work Done:
I tried to grant SharePoint Online site permission to the application by the site owner role and above XML but I have failed test connection to the SharePoint Site URL.
[Error Message]
Failed to get metadata of odata service, please check if service url and credential is correct and your application has permission to the resource.  See server logs for more details.</p>
</div>
</body>
</html>.

When I tried to grant SharePoint Online site permission to the application by the site admin role and following XML,  I have succeeded test connection to the SharePoint Site URL.
[Permission Request XML]
<AppPermissionRequests AllowAppOnlyPolicy="true">
    <AppPermissionRequest Scope="http://sharepoint/content/sitecollection/web" Right="Read"/>
</AppPermissionRequests>

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 8d9fd1db-042c-5f44-ee84-f1a9386fb3e3
Version Independent ID: 81cb2e37-a80c-a48a-68bd-1a03499119a7
Content: Copy data from SharePoint Online List - Azure Data Factory & Azure Synapse
Content Source: articles/data-factory/connector-sharepoint-online-list.md
Service: data-factory
Sub-service: data-movement
GitHub Login: @jianleishen
Microsoft Alias: jianleishen

Thanks for the feedback! We have assigned the issue to the content author to further review this and update the document as appropriate.

@jianleishen , @jonburchel - Could you please review the above feedback from customer and help clarify the document content?
Looks like the pre-requisites section was updated recently which is a bit confusing.
FYI - @Clare-Zheng82
Here is the PR that made changes to earlier content - https://github.com/MicrosoftDocs/azure-docs/commit/708d61e752841ff32412f728ea118a7f450083ad
Thank you

Hi @jianleishen, @jonburchel,
Do you have any update on this?


add sealing/unsealing for trojan messages

Trojan messages should use asymmetric encryption as described in the Book of Swarm:

The recipient only knows that a chunk is a pss message if and when they successfully opened  the Trojan message with the  private key corresponding to the public key that they advertise as their resident key, and do an integrity check/topic matching. Nodes that want to receive such Trojan Messages will keep trying to open all messages that they are closest to. Forwarding nodes (or anyone else apart from sender and recipient) have no way to distinguish between a random encrypted chunk and a trojan message, which means that communication is perfectly obfuscated as generic chunk traffic.
After the recipient has opened the envelope using asymmetric decryption, there is a combined step of integrity check and topic matching. Knowing the length of the payload (from the first 2 bytes of the message), the recipient takes the payload slice and calculates the Keccak256hash of it. Now for each topic the client has a subscription to, it then hashes the payload hash together with the topic. If the resulting segment xor-ed with the topic matches the obfuscated topic id in the message then the message is indeed meant as a message with the said topic and the registered handler is called with the payload as argument.

The specs are as follows:

The message is encoded in a way that allows integrity checking and at the same time obfuscates the topic. The operation to package the payload with a topic is called sealing

// pss

define function seal @payload []byte
    with topic
as
    @seal = hash @payload and @topic  // obfuscate topic
        xor @topic
    return message{ @seal, @payload }

define function unseal message
    with topic
as
    @seal = hash @message payload and @topic
    if @topic == @seal xor @message seal then // check
        return @payload
    return nil

Original issue
(stemming from unresolved thread in #2158)
look into implementing @zelig's suggestion:

an integrity check plus extra obfuscation of the topic can be done if the topic is encrypted with a key that is
key := Hash(topic, payload)

now the integrity check would be:
for topic, handler := range handlers {
    if xor(hash(topic, m.payload),topic) == m.topic {
        go handler(m)
        break
    }
}

see thread here

@zelig , what is the status of this issue?

this is done but a much better way...
we need to document it but i close the issue


No check marks in View menu

1.15.0 x64, Ubuntu 16.04

When I press, say, View->Toggle tree view and tree view appears, there is no check mark against respective item in menu.

Thanks for reaching out!
We require the template to be filled out on all new issues and pull requests. We do this so that we can be certain we have all the information we need to address your submission efficiently. This allows the maintainers to spend more time fixing bugs, implementing enhancements, and reviewing and merging pull requests.
Thanks for understanding and meeting us half way :grinning:


Formatting of Derived Types page

At https://fortran-lang.org/learn/quickstart/derived_types , using Chrome on Windows, if I magnify the page enough with Cntrl+ , and I use the scroll bar on the right, the words at the bottom, Learn, Compilers, Community, Packages, News are sometimes on top of the the text under Quickstart Fortran Tutorial.

This should be fixed with the migration to the new webpage (see #415)


运行报错啊，怎么解决呢？

Compiler message:
../../.pub-cache/hosted/pub.flutter-io.cn/dio-2.2.2/lib/src/dio_http_headers.dart:55:8: Error: The method 'DioHttpHeaders.add' has fewer named arguments than those of overridden method 'HttpHeaders.add'.
void add(String name, value) {
^
org-dartlang-sdk:///third_party/dart/sdk/lib/_http/http.dart:694:8: Context: This is the overridden method ('add').
void add(String name, Object value,
^
../../.pub-cache/hosted/pub.flutter-io.cn/dio-2.2.2/lib/src/dio_http_headers.dart:70:8: Error: The method 'DioHttpHeaders.set' has fewer named arguments than those of overridden method 'HttpHeaders.set'.
void set(String name, Object value) {
^
org-dartlang-sdk:///third_party/dart/sdk/lib/_http/http.dart:703:8: Context: This is the overridden method ('set').
void set(String name, Object value,
^
../../.pub-cache/hosted/pub.flutter-io.cn/cached_network_image-0.8.0/lib/src/cached_network_image_provider.dart:42:24: Error: The method 'CachedNetworkImageProvider.load' has fewer positional arguments than those of overridden method 'ImageProvider.load'.
ImageStreamCompleter load(CachedNetworkImageProvider key) {
^
../../flutter/packages/flutter/lib/src/painting/image_provider.dart:574:24: Context: This is the overridden method ('load').
ImageStreamCompleter load(T key, DecoderCallback decode);
^
Project evaluation failed including an error in afterEvaluate {}. Run with --stacktrace for details of the afterEvaluate {} error.
FAILURE: Build failed with an exception.

What went wrong:
A problem occurred configuring project ':connectivity'.

Could not resolve all artifacts for configuration ':connectivity:classpath'.
Could not download guava.jar (com.google.guava:guava:26.0-jre)
Could not get resource 'https://jcenter.bintray.com/com/google/guava/guava/26.0-jre/guava-26.0-jre.jar'.
Read timed out
Could not download kotlin-reflect.jar (org.jetbrains.kotlin:kotlin-reflect:1.3.11)
Could not get resource 'https://jcenter.bintray.com/org/jetbrains/kotlin/kotlin-reflect/1.3.11/kotlin-reflect-1.3.11.jar'.
Could not GET 'https://jcenter.bintray.com/org/jetbrains/kotlin/kotlin-reflect/1.3.11/kotlin-reflect-1.3.11.jar'.
Received close_notify during handshake
Could not download kotlin-stdlib.jar (org.jetbrains.kotlin:kotlin-stdlib:1.3.11)
Could not get resource 'https://jcenter.bintray.com/org/jetbrains/kotlin/kotlin-stdlib/1.3.11/kotlin-stdlib-1.3.11.jar'.
Could not GET 'https://jcenter.bintray.com/org/jetbrains/kotlin/kotlin-stdlib/1.3.11/kotlin-stdlib-1.3.11.jar'.
Received close_notify during handshake

Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

Get more help at https://help.gradle.org

BUILD FAILED in 1m 26s
Exception: Gradle task assembleDebug failed with exit code 1

一样 ，这个垃圾的东西 就这 都运行不起来

1.升级这两个包：
dio: ^3.0.0
cached_network_image: ^2.2.0
2.把报错的几个类都注释掉
HeaderInterceptors
LogsInterceptors
ResponseInterceptors
3.把HttpManager里引用这3个类的地方注释掉。
就能跑了。

dio: ^3.0.0
cached_network_image: ^2.2.0

2.把报错的几个类都改下代码 return super.onxxx(xxx)
HeaderInterceptors ：
LogsInterceptors：
ResponseInterceptors


ISSUE-298 Pass-thru the default field values for non-union field types so default values are properly maintained and unaffected by the processing of union field type default values.

@enoren Thanks for contributing this PR, I will add a test for this another PR.


YARN-11542. Fix NegativeArraySizeException in IFile's readRawValue me…

…thod
Change-Id: Ie943e06fe287ed08abd51060bacd58b05a465811

Description of PR
We are using bit shifting to double the byte array in IFile's nextRawValue method to store the byte values in it. With large dataset it can easily happen that we shift the leftmost bit when we are calculating the size of the array, which can lead to a negative number as the array size, causing the NegativeArraySizeException.
It would be safer to expand the backing array with a 1.5x factor, and have a check not to extend Integer's max value during that.
How was this patch tested?
For code changes:

[x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 53s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

-1 :x:
mvninstall
4m  1s
/branch-mvninstall-root.txt
root in trunk failed.

+1 :green_heart:
compile
2m 27s

trunk passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
compile
0m 34s

trunk passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09

+1 :green_heart:
checkstyle
0m 36s

trunk passed

+1 :green_heart:
mvnsite
0m 41s

trunk passed

+1 :green_heart:
javadoc
0m 31s

trunk passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javadoc
0m 22s

trunk passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09

+1 :green_heart:
spotbugs
1m 21s

trunk passed

+1 :green_heart:
shadedclient
36m 56s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 34s

the patch passed

+1 :green_heart:
compile
0m 36s

the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javac
0m 36s

the patch passed

+1 :green_heart:
compile
0m 31s

the patch passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09

+1 :green_heart:
javac
0m 31s

the patch passed

-1 :x:
blanks
0m  0s
/blanks-eol.txt
The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply

+1 :green_heart:
checkstyle
0m 32s

the patch passed

+1 :green_heart:
mvnsite
0m 38s

the patch passed

+1 :green_heart:
javadoc
0m 22s

the patch passed with JDK Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1

+1 :green_heart:
javadoc
0m 23s

the patch passed with JDK Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09

+1 :green_heart:
spotbugs
1m 24s

the patch passed

+1 :green_heart:
shadedclient
33m 59s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
7m 47s

hadoop-mapreduce-client-core in the patch passed.

+1 :green_heart:
asflicense
0m 41s

The patch does not generate ASF License warnings.

98m 58s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5894/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/5894

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 82f2548c7328 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 06c375111f46f299e56bec1e33f6da38a2292a44

Default Java
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5894/1/testReport/

Max. process+thread count
1563 (vs. ulimit of 5500)

modules
C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5894/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.


Increase constexpr ops limit

With GCC 10.2.0 I got:

cnl/include/cnl/_impl/../_impl/elastic_integer/../elastic_tag/../../_impl/parse.h:342:28: error: ‘constexpr’ evaluation operation count exceeds limit of 33554432

And even with -fconstexpr-ops-limit=100000000 switch.

@data-man as of #876 the troublesome tests should be disabled by default so you shouldn't need to use the toolchain files. Thanks again for checking out the project.

Thank you!


Use GitHub Actions concurrency to save time in CI

What does this PR do?
Adds concurrency checking to check_code_quality.yml and test-general.yml so that runs from outdated commits will be stopped
Before submitting

[ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
[ ] Did you make sure to update the documentation with your changes?
[ ] Did you write any new necessary tests?

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.


feat: prefix cdn urls

Pull Request Test Coverage Report for Build 77

3 of 3 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-4.8%) to 95.238%

Totals

Change from base Build 74:
-4.8%

Covered Lines:
18

Relevant Lines:
18

💛  - Coveralls


Feature request: 4 gray support

Hi @elad661, thank you for a great library.
Is it possible to add support for 4 gray colors as in official waveshare driver?
It was added there in this commit.

Hi,
Does the 4 gray color mode works with the older 2.7" display, or does it requires newer hardware?

Hi,
Does the 4 gray color mode works with the older 2.7" display, or does it requires newer hardware?

Well, I think it's the same hardware. Mine looks exactly like yours on a gif in README.

Well, I think it's the same hardware. Mine looks exactly like yours on a gif in README.

hmm... looks like (at least in theory) I could add support for a 4 gray mode for partial refresh, but probably not for the fast-updating lookup tables.
However, I don't know when I'll have the time to look into this and I can't promise any results. Waveshare didn't update their documentation explaining how this new functionality works, so implementing it in my library would probably involve copying their new code from here https://github.com/waveshare/e-Paper/blob/master/RaspberryPi_JetsonNano/python/lib/waveshare_epd/epd2in7.py and a lot of trial and error

hmm... looks like (at least in theory) I could add support for a 4 gray mode for partial refresh, but probably not for the fast-updating lookup tables.
However, I don't know when I'll have the time to look into this and I can't promise any results. Waveshare didn't update their documentation explaining how this new functionality works, so implementing it in my library would probably involve copying their new code from here https://github.com/waveshare/e-Paper/blob/master/RaspberryPi_JetsonNano/python/lib/waveshare_epd/epd2in7.py and a lot of trial and error

Okay, thanks. I will try to do some testing by my own. If I find something, I'll let you know.
About documentation, I found that 4 Gray is described in wiki about 3.7inch display.

Okay, thanks. I will try to do some testing by my own. If I find something, I'll let you know.
About documentation, I found that 4 Gray is described in wiki about 3.7inch display.

Hi @elad661, so I started researching official lib and try to understand how it works. I think I will be able to implement 4 colors into your library. Can I make PR after that?
I also managed to get working partial refresh for 4 colors. It seems that its working but there are some pixels which slightly changed color outside the refreshing area. I am attaching video with partial refresh. But you probably cannot see these changed pixels.
https://user-images.githubusercontent.com/17725008/104002826-e1836000-51a1-11eb-9597-bdf65dbb5a3f.mp4

Hi @elad661, so I started researching official lib and try to understand how it works. I think I will be able to implement 4 colors into your library. Can I make PR after that?
I also managed to get working partial refresh for 4 colors. It seems that its working but there are some pixels which slightly changed color outside the refreshing area. I am attaching video with partial refresh. But you probably cannot see these changed pixels.
https://user-images.githubusercontent.com/17725008/104002826-e1836000-51a1-11eb-9597-bdf65dbb5a3f.mp4

Hi,
This looks great! feel free to submit a PR.
I think pixels changing color slightly outside of the refresh area is a problem that also exists without 4-gray support, I don't know if it's a bug in the partial-refresh implementation or a hardware quirk

Hi,
This looks great! feel free to submit a PR.
I think pixels changing color slightly outside of the refresh area is a problem that also exists without 4-gray support, I don't know if it's a bug in the partial-refresh implementation or a hardware quirk

I probably found the reason. In official library there is different LUT than you are using. I tried to use that for BW partial refresh and there is the same problem. I search through multiple libraries for this display and also for Good Display GDEW027W3 which seems to be the same. I found that there are different LUT for partial and full refresh.
So I found LUT for full BW refresh, partial BW refresh and full 4 gray refresh. I haven't found 4 gray partial LUT yet.
I will try to work on PR and also find missing LUT.

I probably found the reason. In official library there is different LUT than you are using. I tried to use that for BW partial refresh and there is the same problem. I search through multiple libraries for this display and also for Good Display GDEW027W3 which seems to be the same. I found that there are different LUT for partial and full refresh.
So I found LUT for full BW refresh, partial BW refresh and full 4 gray refresh. I haven't found 4 gray partial LUT yet.
I will try to work on PR and also find missing LUT.

I'll try to study the code too, I tried it on a waveshare 2.7" 3-colors e-paper. The partial refresh work pretty well, but the screen writing look red/dark red on this one. But overall you did a great job ^^.


Remove can csv upload, can excel upload, can columnar upload in favour of just can upload on Database

After the upload data to database refactor, we are ready to simplify permissions and code.
We currently have 6 REST API endpoints to support this feature and their multiple file types and 3 distinct permissions
(one for each file type). We can easily reduce this to 2 REST API endpoints and 1 permission.
ref: https://github.com/apache/superset/pull/28192

relates to: https://github.com/orgs/apache/projects/345/views/1?filterQuery=can&pane=issue&itemId=68159292


[Bug Report] [el-menu] opened/actived style

Element UI version
2.0.7
OS/Browsers version
win10/Chrome61
Vue version
2.5.9
Reproduction Link
https://jsfiddle.net/xovel/e4yh3yw1/
Steps to reproduce
NaN
What is Expected?
While nav1 is opened or actived, set a special style(eg color and bg-color) to it. NOT the submenu item.
What is actually happening?
Cannot set the opened style. el-menu set an inline css default.

An <el-submenu> cannot be active or highlighted. Only <el-menu-item>s can.


Add wasip1 target

Adds the wasm32-wasip1 target. The rest is formatting

Whatever tool you used for formatting, definitely made it worse

Whatever tool you used for formatting, definitely made it worse

Hahaha I agree. I looked for 20 seconds and couldn't figure out what extension was doing that... do you want me to fix it?


Animation reset fix

Engine fixes

Fix bug with animations not resetting back to original value (fixes #4174). I also reverted the code to allow animations to apply to multiple targets as with blending since it doesn't work in all cases.

@deltakosh Please review the animations change. I have tested blending with https://www.babylonjs-playground.com/#IQN716#9.

I've split the getVerticesData into a separate PR. @sebavan already merged the sandbox fixes.

Something like that:

That won't work to fix the issue I'm trying to fix. The issue I'm trying to fix is to restore the target back to the value before animation is applied.

But is it the responsibility of the animation to do that?


How to save txt vtt and srt outputs? how to set beam_size, initial_prompt, best_of and other parameters

I have checked main page and kaggle and there is no example of these
In reguler I was doing like below
For whisper jax how can I do?
        result = model.transcribe("../input/whisper2/lecture_"+str(lectureId)+".mp3",language="en",beam_size=10,initial_prompt="Welcome to the Software Engineering Courses channel.",best_of=10,verbose=True,temperature=0.0)

        # save SRT

        language = result["language"]
        sub_name = f"/kaggle/working/lecture_"+str(lectureId)+".srt"
        with open(sub_name, "w", encoding="utf-8") as srt:
            write_srt(result["segments"], file=srt)

        # Save output
        writing_lut = {
            '.txt': whisper.utils.write_txt,
            '.vtt': whisper.utils.write_vtt,
            '.srt': whisper.utils.write_txt,
        }

Hi, @sanchit-gandhi. Thanks for the amazing work you've done.
Regarding the decoding options such as beam_size, initial_prompt, etc.  it does seem that it is not possible to change them. Do you confirm? Are there any plans to allow to tune those parameters in the future?

Hey @FurkanGozukara,
To save the transcriptions as a .txt file, you can do the following:
pred_str = pipeline(...)

with open("output.txt", "w") as text_file:
    text_file.write(pred_str)

This will save your predictions to a file called output.txt.
We could definitely extend this to saving .vtt/.srt files. To do so, we'd need to port whisper/utils.py to Whisper JAX.

Please implement srt export, i would be very thankful.

Guys, please provide the code to export in the SRT format. I will be grateful.

is there any update on the SRT export please?

ye i didnt spend any time because it lacks this most important feature

Any updates on enabling temperature setting?

Hey @FurkanGozukara,
To save the transcriptions as a .txt file, you can do the following:
pred_str = pipeline(...)

with open("output.txt", "w") as text_file:
    text_file.write(pred_str)

This will save your predictions to a file called output.txt.
We could definitely extend this to saving .vtt/.srt files. To do so, we'd need to port whisper/utils.py to Whisper JAX.

Hi. When you will port "utils.py" to gift us ability to generate .srt files? I am very need it, but I'm not good at programming.

agree. Keen on  being able to drop back into whisper to write output to SRT, VTT etc.

Hey @FurkanGozukara,
To save the transcriptions as a .txt file, you can do the following:
pred_str = pipeline(...)

with open("output.txt", "w") as text_file:
    text_file.write(pred_str)

This will save your predictions to a file called output.txt.
We could definitely extend this to saving .vtt/.srt files. To do so, we'd need to port whisper/utils.py to Whisper JAX.

Hi!
I just saw this kaggle notebook, maybe you can take a look and take the srt implementation from it?
https://www.kaggle.com/code/idealim/mp3-to-text-and-subtitles-with-whisper-jax/notebook

@sanchit-gandhi


[Untested] Realm screen button not showing properly in iPad

If I use the native client, it's a iphone app which starts with this weird screen that obscures the button, and then it won't connect to anything

Conversation going on here https://chat.zulip.org/#narrow/stream/mobile/subject/iPad/near/280803

Tested on iPad, and the latest release works.


Add chdir functionality.

This fixes a problem finding the ob.cfg file on startup.

Is this ok to merge or are you still working on it?

It failed so let me clean it up.

Well it probably failed because of that problem with travis. Which I can't figure out.

Yeah totally forgot about that. LGTM.


cmd/vet: enable windows/arm before Go 1.14 release

Reminder bug to re-enable windows/arm in cmd/vet/all before releasing Go 1.12.
(windows/arm is #26148)
/cc @jordanrh1

@jordanrh1, what's the status of the windows/arm port and this bug?

This issue was opened as a reminder to remove some temporary code from cmd/vet/all/main.go, but that file no longer exists (vet is not run as part of go test); moreover the windows/arm port is currently inactive, so I think it's safe to close this issue.


[Data Quality][fix] Data Quality Report Display For Admin Role #10919

The permission verification of data quality query is changed from userId to userType
close: #10919

Codecov Report

Merging #10968 (fa0e0bf) into dev (99a134f) will decrease coverage by 0.00%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##                dev   #10968      +/-   ##
============================================
- Coverage     40.31%   40.31%   -0.01%
  Complexity     4858     4858
============================================
  Files           943      943
  Lines         37015    37008       -7
  Branches       4061     4055       -6
============================================
- Hits          14924    14919       -5
+ Misses        20591    20590       -1
+ Partials       1500     1499       -1

Impacted Files
Coverage Δ

...r/api/service/impl/DqExecuteResultServiceImpl.java
70.37% <ø> (-1.06%)
:arrow_down:

...erver/master/processor/queue/TaskEventService.java
75.00% <0.00%> (-5.36%)
:arrow_down:

...rver/master/runner/task/BlockingTaskProcessor.java
75.60% <0.00%> (-0.30%)
:arrow_down:

...eduler/api/service/impl/DataSourceServiceImpl.java
29.49% <0.00%> (ø)

.../dolphinscheduler/plugin/task/datax/DataxTask.java
0.00% <0.00%> (ø)

...cheduler/plugin/task/api/ShellCommandExecutor.java
0.00% <0.00%> (ø)

...er/api/service/impl/TaskDefinitionServiceImpl.java
24.01% <0.00%> (+0.05%)
:arrow_up:

...hinscheduler/plugin/alert/wechat/WeChatSender.java
29.78% <0.00%> (+0.20%)
:arrow_up:

...olphinscheduler/plugin/alert/email/MailSender.java
51.38% <0.00%> (+0.35%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 99a134f...fa0e0bf. Read the comment docs.


Pass original event with custom element

This change passes the original MouseEvent or KeyboardEvent in the CustomEvent that is dispatched on tab-container-changed. It will make it possible for the application code to make decisions based on wether the user clicked a tab or navigated to it with their keyboard.
It might be overkill to send the whole event, we might get away with just sending the typeof of the event but I felt like doing string matching on the application side was a bit yucky. Being able to do a instanceof check seems a bit cleaner. I'd be interested in your thoughts.
Ref: https://github.com/github/github/issues/133906

Hmmm I took at look at the preview code and looks like we are focusing on click events as well as tab-container-change events. Wouldn't simply removing .focus on tab-container-change resolve this issue?

Hmmm I took at look at the preview code and looks like we are focusing on click events as well as tab-container-change events. Wouldn't simply removing .focus on tab-container-change resolve this issue?

I must have missed that, could you direct link me to where the click event handler is?


Add a w64devkit version that also bundled Clang

Inspired by winlibs.com's distribution and w64devkit's -fortran version. This Clang also uses binutils, libstdc++ and other GCC tools so it will not add much space.

While #44 makes some sense as another "flavor" of w64devkit, bundling a
second, superfluous compiler isn't aligned with my goals for the project
even if it shares the rest of the toolchain.


Why not exposing underlying layoutConstraint?

I notice that MASViewConstraint has a private property layoutConstraint which return the underlying NSLayoutConstraint. Why not exposing it directly?
I'm working with another framework (JazzHands) which require to pass it a NSLayoutConstraint to be animated.
I'm using KVC to access the constraint, but it for sure pretty fragile:
[mas_calendarVerticalConstraint valueForKey:@"layoutConstraint"];
So any specific why I should not do this and then for why is not directly exposed?
I see Masonry (and may be wrong) more like a constraint builder not like a constraint "replacement"
Thanks

@huguesbr we don't really want people manipulating these because it more likely than not will cause issues with updating/remaking etc… You can continue doing what you are doing though if need be.
We recommend people fallback to the regular old API if they need to make constraints that play nice with other frameworks.

@robertjpayne got it, thx


Catkin does not respect non-standard link directories for exported dependencies

The new hydro-devel branch of rtt_ros_integration uses
catkin_package(
  DEPENDS rtt ocl
  ...
)

to declare a dependency to Orocos RTT and OCL, which are not catkin packages. Due to a previous call to find_package(OROCOS-RTT...) the variables rtt_FOUND, rtt_INCLUDE_DIRS, rtt_LIBRARY_DIRS, rtt_LIBRARIES, rtt_CFLAGS_OTHER and rtt_LDFLAGS_OTHER are defined at that time.
Unfortunately the export mechanism of catkin currently does not work for this use case unless the rtt_LIBRARIES variable contains absolute paths, as these libraries are not within a CMAKE_PREFIX_PATH/lib folder and catkin does not look at the rtt_LIBRARY_DIRS variable, which would contain the correct paths to the RTT libraries. If rtt_LIBRARIES contains only the names of the libraries, catkin produces linker error messages like this in every package that depends on the exporting package (rtt_roscomm) in this case:
/usr/bin/ld: cannot find -lrtt-geometry_msgs-typekit-gnulinux
/usr/bin/ld: cannot find -lrtt-geometry_msgs-ros-transport-gnulinux
/usr/bin/ld: cannot find -lorocos-rtt-gnulinux
collect2: ld returned 1 exit status

I guess this problem does not only affect this special use case with Orocos RTT, but also all other dependencies that are find_package()-able but have libraries in non-standard link directories. Catkin also does not export the CFLAGS_OTHER and LDFLAGS_OTHER variables, if set.
Is it possible to fix this in catkin itself or is this just a use case that is not supported? One option would probably be to resolve ${depend_name}_LIBRARIES to absolute paths in catkin_pacakge.cmake:159 by calling find_library(... PATHS ${${depend_name}_LIBRARY_DIRS})?

I've also encountered this situation, and the proposed solution works for me just fine. I was wondering if an automated solution to the problem would be possible: Catkin could query the LIBRARY_OUTPUT_DIRECTORY property of a target and take appropriate action if the returned path is different from project_name_DIR. At this moment this is more of an intellectual curiosity, but would there be any obvious gotcha against this approach?.

For the record, see the answer I posted to this question where a solution that is compatible with both multi-CMake project (catkin build,  catkin_make_isolated) and single, nested CMake projects (catkin_make) is proposed.


[BUG] Cannot start CRC on latest ubuntu 20.10

General information

OS: Linux
Hypervisor: KVM
Did you run crc setup before starting it (Yes)?
Running CRC on: Laptop 64Gb RAM

CRC version
CodeReady Containers version: 1.17.0+99f5c87
OpenShift version: 4.5.14 (embedded in binary)

CRC status
CRC VM:          Stopped
OpenShift:       Stopped
Disk Usage:      0B of 0B (Inside the CRC VM)
Cache Usage:     38.56GB
Cache Directory: /home/denis/.crc/cache

CRC config
no output

Host Operating System
NAME="Ubuntu"
VERSION="20.10 (Groovy Gorilla)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.10"
VERSION_ID="20.10"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=groovy
UBUNTU_CODENAME=groovy

Steps to reproduce

src setup
src start

Expected
crc started
Actual
fails with
$./crc start -p ../pull-secret.txt
INFO Checking if oc binary is cached
INFO Checking if podman remote binary is cached
INFO Checking if goodhosts binary is cached
INFO Checking minimum RAM requirements
INFO Checking if running as non-root
INFO Checking if Virtualization is enabled
INFO Checking if KVM is enabled
INFO Checking if libvirt is installed
INFO Checking if user is part of libvirt group
INFO Checking if libvirt daemon is running
INFO Checking if a supported libvirt version is installed
INFO Checking if crc-driver-libvirt is installed
INFO Checking if libvirt 'crc' network is available
INFO Checking if libvirt 'crc' network is active
INFO Checking if NetworkManager is installed
INFO Checking if NetworkManager service is running
INFO Checking if /etc/NetworkManager/conf.d/crc-nm-dnsmasq.conf exists
INFO Checking if /etc/NetworkManager/dnsmasq.d/crc.conf exists
INFO Loading bundle: crc_libvirt_4.5.14.crcbundle ...
INFO Checking size of the disk image /home/denis/.crc/cache/crc_libvirt_4.5.14/crc.qcow2 ...
INFO Creating CodeReady Containers VM for OpenShift 4.5.14...
ERRO Error creating machine: Error creating the VM: Error creating machine: Error in driver during machine creation: virError(Code=1, Domain=10, Message='internal error: process exited while connecting to monitor: 2020-10-29T07:18:12.992977Z qemu-system-x86_64: -blockdev {"driver":"file","filename":"/home/denis/.crc/cache/crc_libvirt_4.5.14/crc.qcow2","aio":"threads","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}: Could not open '/home/denis/.crc/cache/crc_libvirt_4.5.14/crc.qcow2': Permission denied')
Error creating machine: Error creating the VM: Error creating machine: Error in driver during machine creation: virError(Code=1, Domain=10, Message='internal error: process exited while connecting to monitor: 2020-10-29T07:18:12.992977Z qemu-system-x86_64: -blockdev {"driver":"file","filename":"/home/denis/.crc/cache/crc_libvirt_4.5.14/crc.qcow2","aio":"threads","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}: Could not open '/home/denis/.crc/cache/crc_libvirt_4.5.14/crc.qcow2': Permission denied')

$ls -la /home/denis/.crc/cache/crc_libvirt_4.5.14/crc.qcow2
-rwxrwxrwx 1 libvirt-qemu kvm<PHONE_NUMBER>4 Oct 28 23:55 /home/denis/.crc/cache/crc_libvirt_4.5.14/crc.qcow2

Logs
see above

crc 1.16.0 works fine.

This is a duplicate of https://github.com/code-ready/crc/issues/1578 which has a workaround. I don't think it's been fixed yet in git though, so maybe #1578 should be reopened

Workaround gets me to
INFO Checking size of the disk image /home/denis/.crc/cache/crc_libvirt_4.5.14/crc.qcow2 ...
INFO Creating CodeReady Containers VM for OpenShift 4.5.14...
ERRO Error creating machine: Error creating the VM: Error creating machine: Error in driver during machine creation: virError(Code=1, Domain=10, Message='internal error: Process exited prior to exec: libvirt:  error : internal error: cannot apply process capabilities -1')
Error creating machine: Error creating the VM: Error creating machine: Error in driver during machine creation: virError(Code=1, Domain=10, Message='internal error: Process exited prior to exec: libvirt:  error : internal error: cannot apply process capabilities -1')

Any workaround for this one?

@cfergeau NM, I found the problem. I've made a mistake in AppArmor config.
The workaround works for me. Thanks!

crc 1.18.0 still has the same problem.


crash codes

thanks ivan. closing my previous pull request and opening this new request as per the changes suggested by you. please review and let me know if its ok now.

added a global DWORD ret_exception_code.
added new functions ReadDWORDFromPipe in afl-fuzz.c and WriteDWORDCommandToPipe in winafl.c
added switch for exception names in save_if_interesting function.

thanks for the confirmation. i have made the necessary changes and resolved all the conversation.
you can check and merge the pull request.

Thanks once again for working on this!


Error! No file exists at output path '.strings'.

I'm honestly having a hard time understanding why I have this error. I have a Storyboard localized in base + 1 language, Then I have a Localizable.strings file for the NSLocalizedString(s) in code (which I performed using in swift a ".localized" extension to String).
Here's the storyboard with its .strings file localized in italian

here some other xib localized (some using .strings some using localized .xib)

I have this error:
$ nowr-ios (master) $ bartycrouch interfaces -p .
BartyCrouch: Successfully updated strings file(s) of Storyboard or XIB file.
Error! No file exists at output path './nowr-ios/Code/it.lproj/LoginViewController.strings'.

Am I missing something obvious or is there a bug?

Thank you for reporting the issue @superandrew. I can see a few potential problems from your screenshots:

Your nowr-LaunchScreen.storyboard is not localized in Italian (nowr-LaunchScreen.strings is missing in it.lproj
LoginViewController.xib is not String-localized, meaning in it.lproj there should be a LoginViewController.strings file instead of a .xib file
Same is true for SignupViewController.xib

Please make sure you always localize all localizable files in all the languages you support in your project. Otherwise BartyCrouch might not work as expected. If you don't need to localize a Storyboard (like the nowr-LaunchScreen) then you should simply unlocalize the screen entirely so it doesn't appear even in Base.lproj.

I'm closing this due to inactivity.

You should add that to the documentation imo (the fact that if we don't localize we need the storyboard in dev language not base langauge).

"Please make sure you always localize all localizable files in all the languages you support in your project. Otherwise BartyCrouch might not work as expected. If you don't need to localize a Storyboard (like the nowr-LaunchScreen) then you should simply unlocalize the screen entirely so it doesn't appear even in Base.lproj."
This should be in the documentation

Thank you @tirrorex for the text. That's not exactly a PR but I've included it into the README nearly as is (just removed the "nowr-" in front of "LaunchScreen") via 3b5b08490902f0756b3d6b37f951fc98b974f653.

Alright, I know this is closed and old, but a nice addition would be to generate the file that is missing.  I know, PRs are always welcome. :)

Alright, reopening. ;)

Is this still an issue with version 4? I feel like I tackled this in #114. So closing for now until someone reports otherwise.

I'm seeing this currently. If you go to the base project and remove a localization, and bartycrouch runs the build script outlined in the documentation, it still tries to find that localization. Not sure why that is but it would be awesome if someone could fix this!

@thejeff77 Thank you for reporting this. Can you please give me some further information to understand the problem better:

What do you mean by "go to the base project"? Do you mean the Base.lproj directory? Or do you mean a target in Xcode where you remove a language?
How do you know that BartyCrouch is trying to find that localization still? Can you try to run the two commands from the build script manually and give me the full output?

Hey there, thanks for the quick reply!
I'll keep this notification email unread and reply again with error messages next time I enable the bartycrouch update in my build scripts.
For now, to unblock the story, I'll give you what I can right now:
What do you mean by "go to the base project"? -> Your second answer is correct: "... a target in Xcode where you remove a language"
Yeah its the top level target with the xcode icon on it - where you set supported languages for the whole project.
How do you know that BartyCrouch is trying to find that localization still?
After clearing all Derived Data, clearing the Project Data, restarting XCode, rebuilding, deleting & re-installing the app, I still received an error about the removed language not found by bartycrouch from the "bartycrouch update -x" command in the run script phase of the build.
Can you try to run the two commands from the build script manually and give me the full output?
Yes. Stay tuned. For now you should have enough info to see if it is reproducible and unblock the bug. I'm curious if you see the same behavior!
Thanks, and let me know if you have any additional questions.

Turns out removing that localization from the main target doesn't delete the folder.
If the folder isn't deleted, bartycrouch won't ignore it. In my case it was vi.lproj
Might be something to take a look at? I.E. check the target languages instead of the physical file system - as the master of the languages is the target, not the filesystem.


Fix use of VIRTUAL_ENV_PROMPT in setting terminal prompt for venv

Closing in favor of https://github.com/microsoft/vscode-python/issues/22956#issuecomment-1975792411.


Fix stop to read values when current pos reached the end of the page in BinaryPrefixPageDecoder::next_batch

Proposed changes
Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.34 seconds
stream load tsv:          531 seconds loaded<PHONE_NUMBER>9 Bytes, about 134 MB/s
stream load json:         20 seconds loaded<PHONE_NUMBER> Bytes, about 112 MB/s
stream load orc:          64 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          30 seconds loaded 861443392 Bytes, about 27 MB/s
insert into select:          29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size:<PHONE_NUMBER>0 Bytes

run buildall

run buildall

run buildall

run buildall

run buildall

run buildall


fieldpath: add DeleteField function to delete elements from Paved

Description of your changes
Introduces DeleteField to delete a field from the given object.
Fixes https://github.com/crossplane/crossplane-runtime/issues/343
I have:

[x] Read and followed Crossplane's contribution process.
[x] Run make reviewable test to ensure this PR is ready for review.

How has this code been tested
Unit tests with 100% coverage.

I've noticed that two cases weren't handled and added them:

Top-level field could not be removed since iteration assumed there is at least two segments, i.e. accessing i+1. Now we handle this as special case.
In cases where the given path does not exist even before the last node, i.e. metadata.some.another.other where there is no metadata.some, it returned error. Now it's no-op since the target path is already gone.

Added unit tests for both but couldn't reduce cyclomatic complexity without breaking p.delete into pieces that don't make much sense.
@turkenh could you take another look?


Tracks are not shown

Trying the editor in stable.opencast.org you can't use the tracks feature from the editor, here is an example that the tracks are not being loaded to the editor:

Unlike to old editor, the new editor can handle multiple separate tracks. It does however interpret a "composite" track as a single track (because technically, a composite track IS a single track).
Not sure if we need to do anything about this? I guess we could try to implement a legacy option that somehow interprets composite videos as two separate tracks. But that seems like a lot of work, compared to adapting the workflows in such a way that they send single tracks to the editor instead of a composite track.

That makes sence, if that the case,
I agree with you, make that logic is will be a lot of time. Should the new editor load the master tracks individually instead of the composite version?
Making the composite preview takes processing time and is only done because the actual editor can't handle multiple tracks. Right now, if an event does not have a preview, you can't edit it later.

Sorry for the late reply.

Should the new editor load the master tracks individually instead of the composite version?

Yes, the new editor frontend loads all tracks from the defined flavor, not just one. You should provide the tracks you want displayed in that flavor.

Right now, if an event does not have a preview, you can't edit it later.

In the Opencast config file org.opencastproject.editor.EditorServiceImpl.cfg you can specify from which tag or flavor subtype the editor should try to get files from. So if you don't generate preview, you could specify something different in the config? Or is that not the problem?


[Bug] Esc in environment editor with Vim mode not working

Overview

Insomnia Version: v5.0.20
Operating System:  MacOS 10.12.4 (16E195)
Summary:  When editing an environment using vim key map you cannot use esc to exit insert mode. Esc instead causes the settings modal to close.

How To Reproduce

Set Text Editor Key Map to Vim
Go to Manage environments
In the editor, press i to enter insert mode.
Try to exit insert mode by pressing esc

Hmm... this may be a tricky one to fix. The Vim mode is provided directly by Codemirror. All this probably needs is an e.stopPropagation() when the escape command fires but, without forking the Codemirror vim mode, might be hard to do.

Closing as stale.

Please address this. All you need to do is disable ESC closing the environment modal (which already has two buttons for closing).

@kitsu if you want to submit a PR for this I'd be happy to take a look.


Allow different browser/setup in suites

It would be nice if different Browsers / Extensions could be used per suite:
default:
  extensions:
    emuse\BehatHTMLFormatter\BehatHTMLFormatterExtension:
      name: html
      renderer: Twig
      file_name: index
    Behat\MinkExtension:
      sessions:
        default:
          goutte: ~
  autoload:
    '': %paths.base%/tests/features/bootstrap
  suites:
    ie8:
      paths: [ %paths.base%/tests/features ]
      contexts:
        - Behat\MinkExtension\Context\MinkContext
        - SeleniumContext
        - WordPressContext
    iPhone5:
      extensions:
        Behat\MinkExtension:
          sessions:
            default:
              goutte: ~
            iPhone5:
              browser_stack:
                capabilities:
                  browserName: iPhone
                  platform: MAC
                  device: iPhone 5

This wouldn't work as container is not rebuilt between suites in a single profile. Use profiles for extension-bound configs. However, if what you want is custom configuration for the same extension per suite - you already can do it if extension is built that way. Simply put - it is extension responsibility to provide you with a config that allows drastical change of environment between suites.


docs(readme): add javascript starter with webpack

What kind of change does this PR introduce? (docs update)
Adds Javascript starter with Webpack as an alternative solution to the docs

What is the current behavior? (You can also link to an open issue here)

What is the new behavior (if this is a feature change)?

Other information:

@gdi2290 what do you think of this link addition?

sorry, great project but I can't merge it. please merge it here https://github.com/AngularClass/awesome-angular2


disabled download file and override css how ?

How to disabled or remove download file in version 1.12 ?

I have the same question

You can add this to you css stylesheet or global.css if you use tailwind. At the end, the better is to interact directly with the htm; element for more freedom
'''css
#pdf-download {
display: none !important;
}
'''


Update index.html

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: chenfli
If they are not already assigned, you can assign the PR to them by writing /assign @chenfli in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

:star: PR built and available in a preview environment chenfli-nodejs3-pr-8 here

:star: PR built and available in a preview environment chenfli-nodejs3-pr-8 here


New repo/org loading from ansible and automotive

This is done I believe. I loaded these repos in

@cdolfi : Let me know if you agree and close if you do.

@sgoggins once the automotive ones from gh are added this will be good to close


Allow controls for how FileStreamWriter manipulates files

Currently FileStreamWriter implies overwriting an existing file if it exists and assumes file being written is in binary format. Instead, I would like to add a BitField that allows specifying how a file being written to is manipulated.
Maybe:
enum class FileWriteFlag
{
    overwrite = 0b0000'0001, // Allows overwriting existing file
    append = 0b0000'0010, // Allows appending information to existing file
    binary = 0b0000'0100 // Specifies binary file instead of text file
}

Default assumption of FileStreamWriter will be to abort writing if file exists and to write in text format. I think we will mostly be using binary, but the fstream class assumes text as default (I think) so this will keep syntax close to standard library.

Good idea to add such a parameter. Could probably specify a default parameter value. Unless of course you want the programmer to really think about what they want, and force them to be explicit.
Might be worth checking documentation for other systems, such as the Windows API CreateFile, to see what other flags they offer. There might be a more sensible set of flags than whatever the C API decided upon 40+ years ago. In particular, the dwCreationDisposition parameter is worth thinking about.
You make a very good point about being compatible with existing practice concerning text mode. It is best to avoid surprises. Though I guess that's one reason why I've traditionally hated text mode. It makes often surprising conversions to data. I've long felt the text mode default was a mistake of older APIs. Text mode corrupts binary data, but binary mode doesn't corrupt text data. Plus, it feels like the conversion is being done in the wrong place by making it a part of the file API like that, rather than a higher level layer. Interestingly, the Windows API has no concept of a text file.

I was just taking a look back at this. We've had PR #25 open for about 3 months now. Perhaps it's time to revisit. Hopefully we have a better sense of what we need here now.
In terms of support, I'd like to be able to do the following for opening a FileStreamWriter:
Open a file whether it existed before or needs to be created new (default)
Fail if the file already exists
Fail if the file doesn't exist (maybe?)
Truncate existing data to length 0
Do not truncate existing data
Start writing at either the beginning (default) or at the end (append)
I would like to avoid support for any text mode line ending conversions. I think that is better solved at a higher level. In particular, if you use specific methods to read string data (which might convert line endings), you could do so safely in a mixed mode file that contains both binary and text data. That is, line ending conversion is a property of the data itself, not of the file.
In terms of bit flags to support those operations, perhaps:
OpenExisting
OpenNew
Truncate
Append

I would recommend against coupling the default settings of starting to write from the beginning of an existing file with not deleting the existing contents of the file. I think this behavior can cause obscure bugs when the user intends to overwrite the file. It will work if the contents are the same or longer, but if the new content is smaller than the file's current size it will fail when reading back in later in an application. I've done this before, especially before I knew to watch for it.
From my experience, I almost always intend to overwrite an existing file as opposed to writing from the beginning of the file without deleting the existing contents. I think the better default when writing would be to truncate an existing file.
The problem with not truncating by default, means the flag would need to be named differently. I think Truncate is a fairly common word in programming. What about using the word Extend or Continue as the opposite of Truncate? So the default behavior could be Extend = false.

It is worth reviewing the current branch on the subject for how to construct an enum of the flags and the limitations of ifstream's current flags. In particular, I found wrapping a non scoped enum in a namespace the easiest to work with. A scoped enum requires casting when performing bitwise operations. A non-scoped enum will polute the global namespace with all the flags. Wrapping the non-scoped enum in the namespace keeps the enum members out of the global namespace and eases the need for casts when performing the bitwise operations. I may have missed something with the scoped enum that could allow for easier use though as I'm not really experienced with them.
I don't have an opinion on the binary/text flags. I had planned to add them simply to mirror the std library, but I do not have experience using them. If you feel they are detrimental, then we can leave them out for now. They can always be added later if we change our mind.
Thanks,
-Brett

Good point about truncating. This is a stream library after all. For a random access model, the default is likely to open an existing file, such as a database, where it might both read and update, and so you wouldn't want to truncate. For a stream though, you would either perform only read operations, or only write operations. For a write stream, it makes the most sense to truncate and write from the beginning (default), or append to the end of existing data.
We don't need the default flags to be 0. Let's not go too far out of our way to require that. Either way, we end up setting a default value for the open parameter. Now that I think about it, the 0 default is more of an issue for compatibility, if we add new flags in future releases. That would be important for binary compatibility with dynamic linking, and to a lesser extent, source level compatibility with non-default parameter sets.
The details of the enum I'm willing to get a little bit wrong, since we could potentially swap between a namespace enum and an enum class and maintain source level compatibility. I believe an enum class could overload operator |() to implement cast free bit combinations. It may be easier to start with a namespace enum though.

I hadn't thought about operator overloads on a scoped enum. If that is possible, it might make for an interesting solution. I tend to tread carefully when combining creativity and creating operator overloads, although maybe this would be a natural use case.
If other programmers use streams for random write access more often than overwriting a file, maybe the original defaults are better.

For reference, I came across this article on using enum classes as type safe bitmasks.
I don't think random updates are common with streams. That's more for random access read/write files. We should probably design the stream interface for the default case of creating a new initially empty file.

That is a nice article and easy to understand. Thank you for posting.
So, you would be able to conduct bitwise operations between different scoped enums as long as both enum sets were marked to allow bitwise operations. It might be nice to restrict that, but also not that big of a deal.

Err, between different scoped enums? Generally you'd only want it within a single scoped enum, which is what the article describes.
The examples with mixed enums were examples of bad code which the compiler wouldn't fail on for regular enums, but would flag as an error for scoped enums. Hence the case for using scoped enums. By default, or even with the proposed operator overloads, mixing of different scoped enums is not allowed. You'd have to go out of your way with explicit operator overloads taking mixed parameters to allow that. Which of course is a generally bad idea.
Or did I misunderstand what you were saying?

I got the opposite impression. In particular, the implementation given at the end is:
template<typename Enum>
typename std::enable_if<EnableBitMaskOperators<Enum>::enable, Enum>::type
operator |(Enum lhs, Enum rhs)
{
    using underlying = typename std::underlying_type<Enum>::type;
    return static_cast<Enum> (
        static_cast<underlying>(lhs) |
        static_cast<underlying>(rhs)
    );
}

You'll notice the function signature uses operator |(Enum lhs, Enum rhs). In particular, the left hand side, and the right hand side both use the same type Enum (the template parameter). Hence the operator overload only exists when the enum types match.

For reference: ofstream
Available open modes are:

app	seek to the end of stream before each write
binary	open in binary mode
in	open for reading
out	open for writing
trunc	discard the contents of the stream when opening
ate	seek to the end of stream immediately after open

There does not appear to be any flags to test for the presence or absence of a file before opening it.
We can clear the contents of a file by truncating to length 0 with trunc. We can automatically seek to the end of the file with ate.
We probably have no use for app, which ensures every write goes to the end of the file, even if you seek elsewhere first. That would be more useful for random reading + writing, which is not supported by a unidirectional stream.
We have no use for text files, so binary will be permanently set. Similarly, for the output streams, out will be permanently set, while in will be permanently cleared.

I suppose we can pre-check the existence or absence of the file before opening to implement the other semantics. That would be subject to race conditions, so not entirely secure. It would be fine to simply document that though. For most purposes, people simply won't care.
Properly handling pre-checks for file existence or absence is likely to be platform specific. If we decide to someday support that, we could do so with platform specific code, contained in #ifdef blocks.

As the internal file object is contained by FileWriter, we are using a constructor initializer list to initialize it. As such, it will have already initialized and opened a file before the FileWriter constructor runs. Hence trying to check for the existence/absence of a file in the constructor will be too late.
Inside the initializer list, when feeding flags to the internal file object, perhaps we are able to call a static method to translate the flags, which can also pre-check the existence/absence of the file.

In case we revisit this for platform specific solutions to the race condition associated with file existence pre-checks, I thought I'd leave a few links.
CreateFileW - Windows, see dwCreationDisposition values
open - Linux, see O_CREAT and O_EXCL
For converting from a Windows API HANDLE to a FILE*:
Is there a Windows equivalent to fdopen for HANDLEs?


[WIP] Allow SampleOp to take in a dynamic shots value and return a dynamically-shaped result

Context:
Description of the Change:
Change the shots field of SampleOp from an attribute to a proper argument, so that it no longer has to be concrete.
Benefits:
Possible Drawbacks:
Related GitHub Issues:
Based on #1170
[sc-74736]

Regarding testing, there are 3 places that need to be tested throughout the stack:

A valid pennylane frontend qnode gets qjitted into a valid catalyst jaxpr, with shots being a proper argument and shape being either static or dynamic
A valid jaxpr gets lowered into a valid mlir, with sampleop's result shape being either static or dynamic
A mlir with a dynamically shaped sample op can be executed correctly in the backend

Given that qml.device(shots=non_int_literal_expression) is not yet supported at the frontend, step 1 cannot be performed in full. I propose the following:

[lit test] Manually write functions with sample_p.bind(...), i.e. bind primitives directly and do not go through pennylane frontend. This tests that expected uses of the bind() functions produce expected jaxprs.
[lit test] Manually lower these jaxprs to mlir.
[pytest] Manually send a textual IR, with dynamic shaped samples, into the backend through replace_ir.

This leaves the tracing step, i.e. how we actually bind the primitives in jax_tracers.py during qjit, untested; however, until PL device supports dynamic shots, I don't see a workaround for this.

This is still WIP, but since this is a very big change I want CI to tell me how many things are breaking.
I will remove do-not-merge label once ready.

A sidenote on the returned dynamic shape variable: currently we produce
    @catalyst.qjit
    @qml.qnode(
        qml.device("null.qubit", wires=1)
    )  # SampleOp is only legal if there is a device in the same scope
    def f(shots: int):
        obs = compbasis_p.bind()
        x = shots + 1
        # Note that in `primitive.bind(args, kwargs)`, args are treated as jaxpr primitive's
        # proper arguments, and kwargs are treated as primitive's `params`
        # Proper primitive arguments are propagated as jaxpr variables,
        # whereas primitive params are tracers.
        _ = sample_p.bind(obs, x, num_qubits=0)
        #innocent_vector = jax.numpy.zeros((x, 0))
        return _
        #return innocent_vector
        #return jax.numpy.zeros((shots, 37))

    print(f.jaxpr)
    print(f.mlir)

{ lambda ; a:i64[]. let
    transform_named_sequence
    b:i64[] c:f64[b,0] = quantum_kernel[
      call_jaxpr={ lambda ; d:i64[]. let
          e:AbstractObs(num_qubits=0,primitive=compbasis) = compbasis
          f:i64[] = add d 1
          g:f64[f,0] = sample[num_qubits=0] e f
          qdevice[
            rtd_kwargs={}
            rtd_lib=/home/paul.wang/catalyst_new/catalyst/frontend/catalyst/utils/../../../runtime/build/lib/librtd_null_qubit.so
            rtd_name=NullQubit
          ] 0
          h:AbstractQreg() = qalloc 1
          qdealloc h
        in (f, g) }
      qnode=<QNode: device='<null.qubit device (wires=1) at 0x75c3ee8334f0>', interface='auto', diff_method='best'>
    ] a
  in (b, c) }

    func.func public @f(%arg0: tensor<i64>) -> (tensor<i64>, tensor<?x0xf64>) attributes {diff_method = "parameter-shift", llvm.linkage = #llvm.linkage<internal>, qnode} {
      %c0_i64 = arith.constant 0 : i64
      %c = stablehlo.constant dense<1> : tensor<i64>
      %0 = quantum.compbasis  : !quantum.obs
      %1 = stablehlo.add %arg0, %c : tensor<i64>
      %2 = quantum.sample %0 : tensor<?x0xf64>
      quantum.device shots(%c0_i64) ["/home/paul.wang/catalyst_new/catalyst/frontend/catalyst/utils/../../../runtime/build/lib/librtd_null_qubit.so", "NullQubit", "{}"]
      quantum.device_release
      return %1, %2 : tensor<i64>, tensor<?x0xf64>
    }

In other words, to return a dynamically shaped SSA value, the SSA value for the dynamic shape dimension is also returned. Jax calls these returned variables "implicit". See https://github.com/PennyLaneAI/catalyst/blob/main/frontend/catalyst/tracing/jax_dynamically_shaped_arrays.md#explicitimplicit-arguments
We were trying to remove the implicit shape variables, but turns out this was not necessary. They are still present if we compile a dynamically shaped program unrelated to this PR:
@qjit
def workflow(shots: int):
    x = shots + 1
    return jax.numpy.zeros((x, 5))

workflow(3)
'''
[[0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]]
'''
{ lambda ; a:i64[]. let
    transform_named_sequence
    b:i64[] = add a 1
    c:f64[b,5] = broadcast_in_dim[broadcast_dimensions=() shape=(None, 5)] 0.0 b
  in (b, c) }

  func.func public @jit_workflow(%arg0: tensor<i64>) -> (tensor<i64>, tensor<?x5xf64>) attributes {llvm.emit_c_interface} {
    %c = stablehlo.constant dense<5> : tensor<1xi32>
    %cst = stablehlo.constant dense<0.000000e+00> : tensor<f64>
    %c_0 = stablehlo.constant dense<1> : tensor<i64>
    %0 = stablehlo.add %arg0, %c_0 : tensor<i64>
    %1 = stablehlo.convert %0 : (tensor<i64>) -> tensor<i32>
    %2 = stablehlo.reshape %1 : (tensor<i32>) -> tensor<1xi32>
    %3 = stablehlo.concatenate %2, %c, dim = 0 : (tensor<1xi32>, tensor<1xi32>) -> tensor<2xi32>
    %4 = stablehlo.dynamic_broadcast_in_dim %cst, %3, dims = [] : (tensor<f64>, tensor<2xi32>) -> tensor<?x5xf64>
    return %0, %4 : tensor<i64>, tensor<?x5xf64>
  }

We see that, although the implicit shape SSA values are present in the jaxpr and hence in the mlir, they are not actually present in the actual results.
After some investigation, it was discovered that, although the compiled function carries these implicit shape values, when the compiled function is executed, the implicit values are removed. In other words, they are handled downstream when we throw mlir back to python.
Thus, no action needed in this PR, as far as I can tell.
cc @dime10 @erick-xanadu

Moving the David's suggested device shots update cycle into this comment for visibility:
Maybe we can have a "friendly" migration to avoid devices breaking. Would something like this work?

Keep passing down shots in the device kwargs, while we are already setting them via device_init/set_device_shots anyways.
Then, when that change is released, deprecate the functionality and notify devices to remove shots from their device kwargs.
Finally, remove it entirely from our side with the next release.

The only barrier I see is that device constructors would need to be written in such a way that they ignore unknown kwargs 🤔

I looked at the codecov report, they are all false positives. They are, in the following order:

Covered by a pytest marked with xfail
Covered by a lit test (which codecov does not check)
Saying test files are not covered by tests (...)

So all good here.


Incorrect parameters for check_type function from typeguard

Issue description:
As the first parameter passed to the check_type function from typeguard package is unexpected, the execution breaks with the following TypeError:
TypeError: check_type() takes 2 positional arguments but 3 were given
Check the incorrect line of code here: https://github.com/Project-MONAI/monai-deploy-app-sdk/blob/143730939f171f6e562780ea4532d042b6d987f2/monai/deploy/core/io_context.py#L112
More reference:
Typeguard API reference doc: https://typeguard.readthedocs.io/en/latest/api.html#typeguard.check_type
Suggested correction:
Instead, the code must have been:
check_type(value, data_type)

Thank you @KnightCoder for raising this issue. The SDK v0.5 implementation is using the typeguard v2.x API but the v3 brought breaking changes.
We had recommended pinning the version of the typeguard ~=2.12.1 before this PR addresses in the soon to be released patch build v0.5.1

Addressed in release v0.5.1, and pinning typeguard ~=2.12.1 must be removed.


Add webhooks example

Several existing samples use websockets which are great, but it would also be nice to show how to use webhooks for events. Perhaps copying one of the existing examples and re-working it to use a webhook would be a good start.

I'm new with webhooks, and would like to work on this one!


Hotfix: signup verification

This PR will fix the issue of allowing already existing username or email to sign up from the signup page.

LGTM! Thanks!


back to search results button/link on record pages

add a "back to search results" button or link on individual record pages so a user is able to return to their search

moving to lower priority since the browser back buttons work now


Seems package_path is not refreshing

Not sure if it's related to something specific in this repo, but package_path is not refreshing and trying to extract older binary when update is available.
i.e. config is:
"bins": {
        "/home/marko/bin/aws-nuke": {
            "path": "/home/marko/bin/aws-nuke",
            "remote_name": "aws-nuke",
            "version": "v2.18.1",
            "hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
            "url": "https://github.com/rebuy-de/aws-nuke/releases/tag/v2.18.1",
            "provider": "github",
            "package_path": "aws-nuke-v2.18.1-linux-amd64"
        },

update:
   • /home/xxx/bin/aws-nuke v2.18.1 -> v2.19.0 (https://github.com/rebuy-de/aws-nuke/releases/tag/v2.19.0)

Do you want to continue? [Y/n] y
   • Getting v2.19.0 release for rebuy-de/aws-nuke
   • Starting download of https://api.github.com/repos/rebuy-de/aws-nuke/releases/assets/65889581
25.04 MiB / 25.04 MiB [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 7.74 MiB p/s 3s
   ⨯ command failed            error=No files found in tar archive. PackagePath [aws-nuke-v2.18.1-linux-amd64]

you can use bin update -p to fix this.
Bin is failing because the previous package_path was aws-nuke-v2.18.1-linux-amd64 and it can't find it in the new package.
Just released v0.14.1 which improves the error message to clarify this.

Thanks :+1: . Do you know why this is happening in this repo?

Thanks +1 . Do you know why this is happening in this repo?

Yes, because each new binary release has a different name, and bin is "trying" to find the previous binary used and it doesn't find it. Can it be a bit more smart and do a better approximation? Yes, but it's not implemented yet.
Hope that help clarifying :pray:

Gotcha, thanks for explanation!


[Input] Check custom input inputRef implementation

[x] I have followed (at least) the PR section of the contributing guide.

Closes #13894

@henrik1234 It's a great first pull request on Material-UI 👌🏻. Thank you for working on it!