Service key support

This PR add ability for ServiceInstanceAppBindingWorkflow impls to use service keys to expose some backing services credentials (see #285 for more background about related use-case)
The PR is yet lacking component integration and acceptance tests.
The CloudFoundryDeployer class receives PMD warnings that it is getting too large. I failed to move service key responsibility into the existing CloudFoundryOperationsUtils collaborator because this required duplicating too much of the unit testing support code. See https://github.com/orange-cloudfoundry/osb-cmdb-spike/tree/service-key-support-no-pmd-disabling

Started rebasing and fixing new PMD and checkstyle rules from #291 Still some checkstyle fixes to be brought
Original PR before is available at https://github.com/orange-cloudfoundry/osb-cmdb-spike/tree/service-key-support-before-spring-style-rebase

Thanks for the contribution, would you mind rebasing this PR @gberche-orange? Thanks!

@Albertoimpl the PR was rebased against master and adapted to new checkstyles and pmd rules. I'm looking forward to your review.

Thanks a lot for the contribution! I don't see any problem with the implementation nor with the way it is implemented, apart from the lack of ATs and CTs that should be included and the picky notes about the comments I left.
There are a couple of things that do not feel right but I am happy to re-evaluate them in a future release when we start implementing the k8s support.

You can achieve this behaviour by just implementing a CreateServiceInstanceWorkflow.
In a k8s world, there is not a direct mapping to a Service Key, so leaking that name, that is only CF-related, as part of the AppDeployer interface, which is a generic one, doesn't feel right.

As said, if you add the ATs and CTs, I am happy with including it. Thanks a lot for the great work!

Thanks @Albertoimpl for your review. I'll try working on providing the CT and AT now that I see there is interest.

You can achieve this behaviour by just implementing a CreateServiceInstanceWorkflow.

Actually, I'm currently using the service key support in a custom CreateServiceInstanceAppBindingWorkflow, see osbcmdb/AppDeploymentCreateServiceBindingWorkflow.java. I first considered implementing this without SCAB support, but this required to duplicate some of the existing beans such as DefaultBackingServicesProvisionService and CloudFoundryAppDeployer, so I'm happy if this can make it into SCAB and be useful to others.

In a k8s world, there is not a direct mapping to a Service Key, so leaking that name, that is only CF-related, as part of the AppDeployer interface, which is a generic one, doesn't feel right.

Yes. I understand that the service key could map to a K8S service cat ServiceBinding and fetching credentials from the associated Secret, see https://kubernetes.io/docs/concepts/extend-kubernetes/service-catalog/#mapping-the-connection-credentials These pieces of information are stored in secrets that the application in the cluster can access and use to connect directly with the managed service.
So potentially, a K8S-deployer implementation of the service-key methods could be brought in the future. I do agree that then the name "ServiceKey" could be more generic such as "ServiceCredentials".

Great to hear!

Thanks @Albertoimpl for your review. I'll try working on providing the CT and AT now that I see there is interest.

Looking forward to reviewing them. I typically start working always with an AT, then a CT and then I start coding the rest.

DefaultBackingServicesProvisionService and CloudFoundryAppDeployer, so I'm happy if this can make it into SCAB and be useful to others.

Perfect! Thanks for contributing. I am sure this will benefit others.

So potentially, a K8S-deployer implementation of the service-key methods could be brought in the future. I do agree that then the name "ServiceKey" could be more generic such as "ServiceCredentials".

It would be ideal if you made this rename.
Again, thanks a lot for your contributions!

@Albertoimpl Sorry for long silence on this PR. I've rebased the PR and added component test covering service key usage, and addressed suggested changes in the review. Can you please have a new look ?
Note that I'm struggling with checkstyle import rule on https://github.com/orange-cloudfoundry/osb-cmdb-spike/blob/5016f4f5b1bd28d7ba9066bdfe3bc766b8ce397f/spring-cloud-app-broker-integration-tests/src/test/java/org.springframework.cloud.appbroker/integration/CreateBindingWithServiceKeyComponentTest.java#L54

Besides, gradle checkstyle fails on my development box  with symptom
Unable to create Root Module: config {.../src/checkstyle/checkstyle.xml}, which looks similar to https://stackoverflow.com/questions/49645147/checkstyle-unable-to-create-root-module which I need to pursue. Any hint on fixing this would help, as waiting for circleci builds feedback implies a 15 min feedback time.
I tried to follow the CONTRIBUTING.adoc which recommends usage of the eclipse code formatter, and applied the intellij "optimize import" action. Is there other automated import formatting that the app broker team uses within intellij ?

I'll now try to provide an acceptance test. Is there a way I would be able see test results associated with the PR for acceptance tests (such as making the associated concourse pipeline public or adding 3rd party contributors as viewers) ?

@Albertoimpl Re on acceptance test covering service key support. Such test would probably ask for a service key on the brokered service instance (i.e. an OSB service binding call with an empty appId in the bindResource), hence requiring a service binding workflow.
Would that make sense to make this workflow available in core (for integration-tests and acceptance-tests) to use, and leave it up to app broker users to enable it, for instance using a property ?

Identified steps for contributing the AT

[ ] set up the local environment to run the test

The documentation is missing the required authorities for the UAA client

[ ] execute existing test cases
[ ] develop new test

[ ] handle clean up of provisioned element on test failures

[ ] reverse engineer and document element provisionned
[ ] document clean up script manually invoked
[ ] improve the CloudFoundryAcceptanceTest to be resilient to failed/interrupted tests that did not execute cleanup

[ ] execute the new test to get test status

[ ] import concourse ci pipeline

Thanks a lot for the contributions, it is looking great!
We added a different style guide than the documented in CONTRIBUTING, sorry about that, we are going to fix it: https://github.com/spring-cloud/spring-cloud-app-broker/issues/345 and you can find all the check styles over here: https://github.com/spring-cloud/spring-cloud-app-broker/tree/master/src
Regarding the ATs, we have them ready to be public, we need to move the pipeline to a public concourse instance: https://github.com/spring-cloud/spring-cloud-app-broker/issues/346
However, you should be able to run them in your own environments. Let me know if you find any issues with that.
Thanks!

@Albertoimpl I've updated the PR with

service instance deprovision revert (fixes #344)
acceptance test for service key support which passes on my environment.

The circle ci build is still failing on checkstyle/pmd violations that I'm trying to fix.
Can you please resume the review of this PR when possible ?

@Albertoimpl @royclarkson @LittleBaiBai As a less intrusive fix for #344 than
b2324e4 and 068db25, I'm now considering instead to add distinct workflows only triggering for brokered services with only backing services, e.g. ServicesOnlyDeploymentDeleteServiceInstanceWorkflow and ServicesOnlyDeploymentDeleteServiceInstanceWorkflow with the accept() method selecting the right workflow depending on the absence of backing apps.
This should enable both behaviors to co-exist without negatively impacting each other, while sharing with the app broker authors the service key support (which @Albertoimpl found valuable in https://github.com/spring-cloud/spring-cloud-app-broker/pull/292#issuecomment-567013071 )
Please let me know if you have other ideas.
I hope this will help review/merge of this PR, and also convince my team that I'm heading in the right direction when I invest time and effort to contribute to back app broker while developing features for orange-cloudfoundry/osb-cmdb-spike

This osb-cmdb related feature seems to not make much sense into app-broker use-cases.  Closing it.
Thanks however @Albertoimpl for your past review and support on this PR !


How much data is needed to train my own model

Hey, thank you for sharing your wonderful work!
I am trying to train the model with my own data. I wonder how much data I would need to get a decent performance (say, how many pairs of src.npy, ref.npy, and gt.npy)?
Thanks in advance!

It is hard to say. I suppose it depends on how large the rotation is in your scene. 3DMatch has 20k training pairs, while KITTI only has 1400+.

It is hard to say. I suppose it depends on how large the rotation is in your scene. 3DMatch has 20k training pairs, while KITTI only has 1400+.

Thanks for the reply. I wonder how large a scenario should each training pair represents? Say, 3 m * 3 m?

The best way is to use all points. If you use a fixed size, the overlap could be small.

Hello, if I train the model with my own dataset, the dataset needs the  rotation matrix, translation matrix and the overlap rate of two-point cloud. Do I need other conditions besides these?Looking forward to your reply.

Hello, if I train the model with my own dataset, the dataset needs the rotation matrix, translation matrix and the overlap rate of two-point cloud. Do I need other conditions besides these?Looking forward to your reply.

I have a similar problem, I don't know how to calculate the overlap rate of homemade datasets, can you help? Thank you

You can ignore the overlap ratio, but the transformation matrix is needed. You can use this function to compute the overlap.

You can ignore the overlap ratio, but the transformation matrix is needed. You can use this function to compute the overlap.

Are there two options: do not provide overlap.pkl or use the function you provided to calculate the overlap rate to generate overlap.pkl?
thanks

Are there two options: do not provide overlap.pkl or use the function you provided to calculate the overlap rate to generate overlap.pkl? thanks

It is okay to give a random overlap ratio value. It is only used for logging.

Are there two options: do not provide overlap.pkl or use the function you provided to calculate the overlap rate to generate overlap.pkl? thanks

It is okay to give a random overlap ratio value. It is only used for logging.

This means I can randomly generate the overlap.pkl file according to the number of my dataset?thanks！


Bump Dockerfile golang version

Currently building with 1.6-alpine; should use 1.7 or 1.8 if released.

Completed in f09a4558a953e9a27b1cf5bfad1de6aef8bf1c2c


Better describe the 'keep me signed in' option

Overview
We've seen a good amount of user feedback report a lack of trust in using the 'keep me signed in' feature on the login screen. While some users will avoid this option for increased security, it can also be really helpful in streamlining a common action - if you have other controls in place, like a personal device in a secure location like a workplace office.
To support users in making a decision to use this feature it be helpful to add more information about this feature on the login screen
Suggestions:

Include how long a user will be signed in for if the option is selected
Review the wording entirely for usefulness
Include some form of guidance like: link for more information, tooltips, help labels, etc

Acceptance Criteria

[ ] A CMS user understands, to the full extent, what will happen if the 'keep me signed in' option is selected
[ ] The configured time period is included in the label, e.g '... for x days'
[ ] Any wording changes consider how this screen might relate to having the multi-factor authentication module installed, as selecting this option would also skip the MFA login screen on subsequent visits
[ ] Additional information is included where relevant, i.e tooltip
[ ] This change is effected in both the standard login form template and the special 'login-forms' module
[ ] Remove redundant 'title' attribute from the existing checkbox
[ ] User and Developer documentation is updated to reflect the change

PR's

[ ] https://github.com/silverstripe/silverstripe-login-forms/pull/86
[ ] https://github.com/silverstripe/silverstripe-framework/pull/9900

Some broad ideas about what different options we might have visually.

Note that the checkbox currently has a title with some details on it:
 - this should be removed/updated as part of this issue.

whats your thoughts on another survey

I don’t see any harm in sending that survey… Could just send it internally and close it at the end of  today? I’d suggest the following changes though:

Zooming in more on the image (it's hard to read the "for 30 days" part)
Tweak the intro text to something like:

People have the option to skip the following login screen (for their current device) to avoid what could be a repetitive action if you are accessing the CMS often. While using this option streamlines the process, it is reducing the level of security so people need to be aware of the impacts.
People should only use this feature if they trust the device they are using. They will remain authenticated for 30 days and the functionality applies to the login screen and MFA (if they use it).

@bergice said there could also be changes to the Wātea, Starter, and Simple themes.
I'd say those themes should just replicate the changes made for login-forms, what do you think @clarkepaul ?
If so, we can either amend the following AC to reflect updating these themes as well, or create a follow up issue to address those changes

" - [ ] This change is effected in both the standard login form template and the special 'login-forms' module"

@bergice said there could also be changes to the Wātea, Starter, and Simple themes.
I'd say those themes should just replicate the changes made for login-forms, what do you think @clarkepaul ?
If so, we can either amend the following AC to reflect updating these themes as well, or create a follow up issue to address those changes

" - [ ] This change is effected in both the standard login form template and the special 'login-forms' module"

I agree they should look the same, but there will be small changes as they have different layouts/styling and icons as well.
The checkbox is part of the form, but the help icon/link is rendered as part of the template next to the header.

So, clarifying what it sounds like still needs to happen:

Get consensus on checkbox label wording - pending survey results.
@bergice to outline whether the hover text in @clarkepaul favoured designs is possible, or how much effort would be involved.

As an aside, I'm updating the ACs with reference to the Wātea, Starter, and Simple themes. Aside from confirming they reference the new checkbox label, they don't need further work (like adding help labels). With the login-forms module installed, it will override these theme login templates, and login-forms was made the default for new sites from CMS 4.6+.

So, clarifying what it sounds like still needs to happen:

Get consensus on checkbox label wording - pending survey results.
@bergice to outline whether the hover text in @clarkepaul favoured designs is possible, or how much effort would be involved.

As an aside, I'm updating the ACs with reference to the Wātea, Starter, and Simple themes. Aside from confirming they reference the new checkbox label, they don't need further work (like adding help labels). With the login-forms module installed, it will override these theme login templates, and login-forms was made the default for new sites from CMS 4.6+.

The bootstrap tooltips could potentially work if we override the templates.
I was gonna add the help link / button to the other themes, but if you don't want me to that's fine as well.

To further clarify a point made in standup this morning, the following ACs need only to apply to the login-forms template:

[ ]  Additional information is included where relevant, i.e tooltip

I've updated that AC to the following to be more clear:

[ ]  ~Additional information is included where relevant, i.e tooltip~

[ ] Additional guidance is added to the login template to support the user in making a safe decision (i.e. tooltip and 'help' link)

@clarkepaul regarding the new designs at https://projects.invisionapp.com/share/BN10MV40YMZT#/screens/448746237_Login_Forms_-_Keep_Me_Signed_In_V2 :

The bootstrap tooltips only work in admin, and I'm not too keen on adding this to the front-end.
The new question mark tooltip icon is essentially a new component. Is this something we are going to be using elsewhere as well? I'd probably prefer having it as a React component instead and adding it to the pattern library, but it seems out of scope of this issue. This also makes it tricky as the whole form is rendered using form fields, and it would have to look different on different themes as well, plus the font icons may be different too.

My suggestions:

Maybe it is easier to just fix the normal tooltip that was there to begin with?
I've already added the Need Help? link, so it should be fine to keep it. But keep in mind that we're only looking at adding it to the login-forms module at this stage.

The question mark is just an icon in the pattern library currently but you probably can't access that in login forms (correct me if my assumption is wrong) so would need that icon in the login forms module? The popover is already a component in the React library but you might not be able to access that so I thought you'd be replicating it as we do replicating the look of form fields.
The old tooltip is not really a tooltip (was a title), its not intuitive to have a question mark icon to trigger it as the delay is too long. Keen to chat in person if possible.

Had a chat to @clarkepaul, we'll mock up a new tooltip quickly with css/js then I'll probably just override the checkbox template on login-forms to render the question-mark icon and tooltip.

Mockup https://codepen.io/clarkepaul/pen/OJWjYRL I expect you'd write the JS as you see fit.

For User and Developer documentation is updated to reflect the change - we'll handle that via https://github.com/silverstripe/silverstripe-session-manager/issues/15

PRs have been merged


Fix/19987: Prioritize using name of policy from policy

Details
Workspace name does not change when the room is set to "Public" because we use report.policyName first if it exists and when policy name is updated the BE doesn't update report.policyName. So we fix it by using name of policy from policy first
Fixed Issues

$ https://github.com/Expensify/App/issues/19987
PROPOSAL: https://github.com/Expensify/App/issues/19987#issuecomment-1575397392
Tests

Navigate to the "New room" option
Enter a room name and select a workspace
Set the visibility to "Public" and click on "Create room"
Proceed to the settings > Workspace section
Attempt to edit the workspace name
Verify that the workspace name is updated

[x] Verify that no errors appear in the JS console

Offline tests

same above
QA Steps

Navigate to the "New room" option
Enter a room name and select a workspace
Set the visibility to "Public" and click on "Create room"
Proceed to the settings > Workspace section
Attempt to edit the workspace name
Verify that the workspace name is updated

[ ] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[x] I have checked off every checkbox in the PR author checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

https://github.com/Expensify/App/assets/129500732/842ff645-0ce8-4036-870a-b07a02481ab2

Mobile Web - Chrome
https://github.com/Expensify/App/assets/129500732/dc3a9b58-c601-4b67-a28a-0fd3ab7103bc

Mobile Web - Safari

https://github.com/Expensify/App/assets/129500732/5a511d69-e738-40b9-9e46-3a80ea2d2e08

Desktop
https://github.com/Expensify/App/assets/129500732/801cd496-dba8-4425-ad5f-e9214542a8b5

iOS

https://github.com/Expensify/App/assets/129500732/ecefbf5d-d80e-46a1-b97e-c3bb99f4d30f

Android

https://github.com/Expensify/App/assets/129500732/3ac074e9-4380-44b3-bf84-bed61ca91dcd

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/puneetlath in version: 1.3.30-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/luacmartins in version: 1.3.30-5 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


chore: release next version

Automated changes by create-pull-request GitHub action

:tada: This PR is included in version 0.12.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


[launcher-package] Overwrites ReservedCodeCache settings specified on the command line

steps
Version: Sbt + launcher version 1.3.13
Repro steps:

Make an empty directory anywhere and cd into it
Run sbt -v
Observe output:

[process_args] java_version = '8'
[sbt_options] declare -a sbt_options=([0]="-Xss8m")
# Executing command line:
java
-Dfile.encoding=UTF-8
-Xms1024m
-Xmx1024m
-Xss4M
-XX:ReservedCodeCacheSize=128m
-Xss8m
-jar
/usr/share/sbt/bin/sbt-launch.jar

Try to override -XX:ReservedCodeCacheSize with sbt -v -J-XX:ReservedCodeCacheSize=256M
Observe output:

sbt -v -J-XX:ReservedCodeCacheSize=256M
[process_args] java_version = '8'
[sbt_options] declare -a sbt_options=([0]="-Xss8m")
# Executing command line:
java
-Dfile.encoding=UTF-8
-Xms1024m
-Xmx1024m
-Xss4M
-XX:ReservedCodeCacheSize=128m
-Xss8m
-jar
/usr/share/sbt/bin/sbt-launch.jar

Actual: ReservedCodeCacheSize parameter is ignored
Expected: ReservedCodeCacheSize parameter is passed to JVM
problem
It looks like the packager intentionally ignores and overwrites this argument: https://github.com/sbt/sbt-launcher-package/blob/master/src/universal/bin/sbt#L177
expectation
Actual: ReservedCodeCacheSize parameter is ignored
Expected: ReservedCodeCacheSize parameter is passed to JVM
workaround
If you add -Xmx1G -XX:ReservedCodeCacheSize=256M to .jvmopts in this directory the issue is fixed. It seems that you must specify -Xmx (any value) for this to work.
To test:
echo "-Xmx1G -XX:ReservedCodeCacheSize=256M" > .jvmopts && sbt -v -J-XX:+PrintCodeCache (and ctrl+c out of the shell to get the code cache print)
Looks like:

notes
It's also worth noting that this "just set Xmx workaround" doesn't appear to work on the command line - only via the .jvmopts route.

The simplest quick-fix solution to me seems to be: "Don't set -XX:ReservedCodeCacheSize if it's already set" - ignoring any other weirdness about why things only happen if -Xmx is set in a jvmopts.

@henricook Thanks for the details bug report!
I'm a bit torn on this one. The relevant logic I think is the function below - https://github.com/sbt/sbt-launcher-package/blob/7d75936b0248446297a0559e21e31fcefece7c25/src/universal/bin/sbt#L181-L211
addDefaultMemory() {
  # if we detect any of these settings in ${JAVA_OPTS} or ${JAVA_TOOL_OPTIONS} we need to NOT output our settings.
  # The reason is the Xms/Xmx, if they don't line up, cause errors.
  if [[ "${java_args[@]}" == *-Xmx* ]] || \
     [[ "${java_args[@]}" == *-Xms* ]] || \
     [[ "${java_args[@]}" == *-XX:+UseCGroupMemoryLimitForHeap* ]] || \
     [[ "${java_args[@]}" == *-XX:MaxRAM* ]] || \
     [[ "${java_args[@]}" == *-XX:InitialRAMPercentage* ]] || \
     [[ "${java_args[@]}" == *-XX:MaxRAMPercentage* ]] || \
     [[ "${java_args[@]}" == *-XX:MinRAMPercentage* ]]; then
    :
  elif [[ "${JAVA_TOOL_OPTIONS}" == *-Xmx* ]] || \
       [[ "${JAVA_TOOL_OPTIONS}" == *-Xms* ]] || \
       [[ "${JAVA_TOOL_OPTIONS}" == *-XX:+UseCGroupMemoryLimitForHeap* ]] || \
       [[ "${JAVA_TOOL_OPTIONS}" == *-XX:MaxRAM* ]] || \
       [[ "${JAVA_TOOL_OPTIONS}" == *-XX:InitialRAMPercentage* ]] || \
       [[ "${JAVA_TOOL_OPTIONS}" == *-XX:MaxRAMPercentage* ]] || \
       [[ "${JAVA_TOOL_OPTIONS}" == *-XX:MinRAMPercentage* ]] ; then
    :
  elif [[ "${sbt_options[@]}" == *-Xmx* ]] || \
       [[ "${sbt_options[@]}" == *-Xms* ]] || \
       [[ "${sbt_options[@]}" == *-XX:+UseCGroupMemoryLimitForHeap* ]] || \
       [[ "${sbt_options[@]}" == *-XX:MaxRAM* ]] || \
       [[ "${sbt_options[@]}" == *-XX:InitialRAMPercentage* ]] || \
       [[ "${sbt_options[@]}" == *-XX:MaxRAMPercentage* ]] || \
       [[ "${sbt_options[@]}" == *-XX:MinRAMPercentage* ]] ; then
    :
  else
    addMemory $sbt_default_mem
  fi
}

it says if you have memory-related setting in arguments or SBT_OPTS, it would not create the default memory setting. Currently, -XX:ReservedCodeCacheSize is not recognized as one of them. So what would happen if we add -XX:ReservedCodeCacheSize into this logic is that no -Xmx setting would be set but -XX:ReservedCodeCacheSize would be set. Maybe that's what you want, but it feels like an easy pitfall someone could fall into. The workaround would be to specify both -Xmx and -XX:ReservedCodeCacheSize.
$ SBT_OPTS= sbt -v -J-Xmx1G -J-XX:ReservedCodeCacheSize=256M
[process_args] java_version = '8'
[sbt_options] declare -a sbt_options='()'
# Executing command line:
java
-Dfile.encoding=UTF-8
-Xmx1G
-XX:ReservedCodeCacheSize=256M
-jar
/Users/eed3si9n/.sdkman/candidates/sbt/1.3.13/bin/sbt-launch.jar

Thanks for your response Eugene, I was feeling the same thing. I realise that setting Xmx often (always?) goes hand-in-hand with setting ReservedCodeCacheSize so I see your point.
I'd kept this open because I thought command line override wasn't working, but I see in my 'notes' section screenshot above I've missed of a - before the J for Xmx :man_facepalming:
So I think we can close this, it's probably enough that this issue will be Googleable with this one piece of easy to follow advice
If you want to set -XX:ReservedCodeCacheSize via sbt you must also set -Xmx


(chore) modify upgradable contracts to use uups pattern using DAOUpgradableContract

@aliberkhsl i've commited to master
please review

@aliberkhsl


Documentation for turbo info.

Description
Looks like we were missing docs for turbo info!

I've learned we're not ready for turbo info to be a publicly known command yet. Stay tuned!


Remove IDispose pattern in favor of RemoveFromWorld()

I used IDispose it so that a user can call myBody.Dispose() and it will remove the body from the world. There are some issues with this:

The Disposable pattern (IDisposable) is meant for releasing unmanaged resources. Using it for anything else can have unintended consequences.
When an object falls out of scope, not mean the object will be garbage collected right away. Dispose() will therefore not be called until the GC comes to collect.
Since the World object already have a reference to the body, even when the body falls out of the current scope, it does not get garbage collected. There are tricks like WeakReference, but it is currently not implemented and might introduce bugs into user code, since they will need to explicitly state they need a strong reference in the World list.
IDispose gave the ability to use the 'using' pattern like seen below, but it is an esoteric feature since most bodies seem to be long lived.

using (Body b = new Body(_world))
{
   //Do work with body
}

//Body will be removed from world here.

Instead, I propose a RemoveFromWorld() method that needs to be explicitly called whenever a body should leave the world. This is exactly the same as calling World.RemoveBody(body), but sometimes users don't have access to the World object, but do have the Body object.
Any comments, improvements or votes (thumb up/down) are appreciated.

Sounds good! Especially with regard to pooling (#5), this change might also help understand users the role of the physics body as something they have to manage lifetimes of.


Publish all MAS data to GBIF

We do not only want to publisch the MBAG - MAS data to GBIF, but all MAS data from sovon. This former data is collected

at certain sampling units part of the MBAG - MAS sampling frame
starting from 2022
within specific counting periods during the year
within a specific radius (300 m)

Counts using the MAS protocol have been happening since 2018 and sometimes fall outside the above time periods, radius ...
With this PR, we also add the latter data to the GBIF publication. Here are the most important changes:

column sample_mas

indicates whether the observations is part of the  MBAG - MAS data (TRUE) or not (FALSE)
is this a good column name? ask data publishing colleagues for DarwinCore?

coordinateUncertaintyInMeters

sample_mas TRUE: 10 m if distance to observer is < 100 m and 0.1*distance if distance to observer is > 100 m
sample_mas FALSE: 30 m

Merci @hansvancalster. is_mas_sample klinkt goed!


Fix and improve .assertValidHTML() test method

The BaseTestCase.assertValidHTML() helper method was not parsing the passed in content argument, but instead self.panel.content.  Fix this issue, and also remove the msg argument which was never supplied by any callers.  Finally, fix a test case failure exposed by changing the method to parse the supplied content.

Thank you!


only publish the dist folder on npm

Why do you think this is better than publishing source code and readme? What's the issue now?

It is publishing the readme anyway. But why do you want to make everybody download the source, the travis file, etc.?

Also I'm having a weird bug because the .babelrc is included

I often look into source code of other libraries, esp when working offline.
Not sure what issues you're having but not having .babelrc in just this one package won't solve the issue for you I guess.

it actually does

this is the only library which is publishing the .babelrc (over 30ish libraries i'm using)

That is just a coincidence in your case and it won't solve your issue.
That's all .babelrc files inside this package's node_modules:
$ find node_modules -name '.babelrc'
node_modules/attr-accept/.babelrc
node_modules/babylon/.babelrc
node_modules/depcheck/.babelrc
node_modules/escope/.babelrc
node_modules/eslint-plugin-jsx-a11y/.babelrc
node_modules/estraverse/.babelrc
node_modules/jsx-ast-utils/.babelrc

It does solve my issue:
Unknown plugin "add-module-exports" specified in ".../node_modules/react-dropzone/.babelrc" at 0, attempted to resolve relative to ".../node_modules/react-dropzone" while parsing file: .../node_modules/react-dropzone/dist/index.js
. Because it's the only one requiring the add-module-exports plugin.
Anyway, I'll just continue to rm the .babelrc file on post install and download a twice as big module...

Why are you transpiling installed packaged in the first place?

I'm not transpiling, that's the weird thing. But I guess it's more a browserify/babelify issue than react-dropzone.

Regarding the package size: I don't think that transferring another 20KB is an issue, is it? By doing so I allow developers to look up the source code and tests for the package, which is often acts better as any documentation.

That's one way to look at it. The other way is that we end up with gigantic node_modules directories which could be a lot smaller.
But it's not a big deal for me, I have a good connection.

I'm not familiar with browserify so I can't help but it is definitively an issue with it. In webpack, you just exclude node_modules from babel loader.

If you wanted to let developers see the source files as well, you could easily just push src and dist then ;)

And also manually add README and license and changelog. And babelrc and other required stuff in case someone wants to build from source.
On a serious note: I don't get the issue with publishing the source code of the package on npm and won't perform any actions until I'm convinced this is bad.


[BUG] Not working in IntelliJ IDEA

Describe the bug
I'm writing origins files alongside mod code, so I wanted to load the JSON schemas into IntelliJ IDEA. I tried adding the schemas as described in these jetbrains docs, but most things don't work properly. I don't know if this is an issue with the schemas, if VSCode just extends the schema definition, or if IntelliJ doesn't have full support for this, but could you check it out?
To Reproduce

Create a datapack (or mod project)
Open in IntelliJ IDEA
Go to Settings > Languages & Frameworks > Schemas and DTDs > JSON Schema Mappings
Press the + button
Put in https://raw.githubusercontent.com/SnaveSutit/origins-mod-json-schemas/main/schemas/apoli/power.json for "Schema file or URL"
Press the lower +
Select "Add file path pattern
Input **/data/*/powers/**.json

It should look like this:

Click "OK"
Open an origin power file

Expected behavior
All descriptions and autocomplete should appear like in VSCode.
Screenshots
Descriptions not appearing

No label appearing on string values

Autocomplete/descriptions does work for universal properties at the top level (except type) and in condition

Keybindings seem to cause issues

Markdown does not apply - this is definitely not your fault though

I think I can fix the markdown descriptions not being parsed issue though: #20

@JosiahFu FYI the schemas use JSON Schema version 7, so make sure you configure that as well.
However, in my testing that doesn't seem to have fixed anything. But it for sure won't work on version 4.

Yep, neither version worked completely

I'm also unable to find any information online where others have encountered such an issue, so until I can find someone who knows JSON schemas in Intellij I'm stuck.

Alright, well thanks anyway for looking into it!


How to pass extra options from UI into a language service?

I'm building a web page using React. The web page (component Playground) contains a Monaco Editor, behind which there is a language server conforming to language server protocol. At the moment the editor provides especially the feature of formatting.
Here is our-language-server.js, it calls the formatCode function developed in OCaml and transformed to JavaScript.
    import {TextEdit, Position} from 'vscode-languageserver-protocol';

    export class OurLanguageService {
        format (document, range, option) {
            let edits = [];
            edits.push(TextEdit.del(range));
            //@ts-ignore
            let t = formatCode(document.getText(), <parameters>);
            edits.push(TextEdit.insert(Position.create(0, 0), t));
            return edits;
        }
    }

In the react component ResizableMonacoEditor, we have for example:
    monaco.languages.register({
      id: LANGUAGE_ID,
      extensions: [],
      aliases: [],
      mimetypes: [],
    });

    const m2p = new MonacoToProtocolConverter();
    const p2m = new ProtocolToMonacoConverter();
    let ourLanguageService = new OurLanguageService();

    function createDocument (model) {
      return TextDocument.create(MODEL_URI, model.getModeId(), model.getVersionId(), model.getValue());
    }

    monaco.languages.registerDocumentRangeFormattingEditProvider(LANGUAGE_ID, {
      provideDocumentRangeFormattingEdits (model, range, options, token) {
        const document = createDocument(model);
        const edits = ourLanguageService.format(document, m2p.asRange(range), m2p.asFormattingOptions(options));
        return p2m.asTextEdits(edits)
      }
    });

    const monacoOurModel = monaco.editor.createModel(value, LANGUAGE_ID, MONACO_URI);

    class ResizableMonacoEditor extends React.Component {

      _handleEditorDidMount (editor, monaco) {
        this.editor = editor;
        this.editor.setModel(monacoOurModel);
        this.forceUpdate();
        this.handleResize();

        if (this.props.editorDidMount) {
          this.props.editorDidMount(editor, monaco);
        }
      }

      render () {
        let _handleEditorDidMount = this._handleEditorDidMount.bind(this)
        return <Suspense fallback={
Loading
}>
          <MonacoEditor {...this.props} editorDidMount={_handleEditorDidMount} />
        </Suspense>
      }
      ... ...

In React component Playground, we have
    _handleEditorDidMount (editor, monaco) {
      this.editor = editor;
      this.formatAction = this.editor.getAction('editor.action.formatDocument');
    }

    runFormat () {
      clearTimeout(this.timeout);

      this.timeout = setTimeout(() => {
        this.editor.getModel().updateOptions({ ... ... })
        this.formatAction.run();
      }, 300);
    }

    inputChange (event) {
      let value = event.target.value;
      this.setState((s) => ({ code: value }));
      this.runFormat();
    }

     return (
       ... ...
       <ResizableMonacoEditor value={this.state.code} editorDidMount={_handleEditorDidMount} height="100%" options={monacoOptions} />

In the web page, users will be able to tune several parameters about the formatting of the code, for example ribbonWidth, which indicates the maximum number of non-indentation characters that should appear on a line; and limitWidth, which indicates the maximum number of characters that should appear on a line. I want to pass all these parameters via components and finally to formatCode(document.getText(), <parameters>, but I don't know how.
It seems that we could use these options to pass values, but none of them is equivalent to my ribbonWidth or limitWidth.
Does anyone have any idea?

In the web page, users will be able to tune several parameters about the formatting of the code, for example ribbonWidth, which indicates the maximum number of non-indentation characters that should appear on a line; and limitWidth, which indicates the maximum number of characters that should appear on a line.

@chengtie ribbonWidth and limitWidth are not defined in the protocol for FormattingOptions. You should not expect these options to be respected by a language server (I guess there are some out there that expects these options?). I presume you are okay with this and that the OCaml language server supports these options...?

I want to pass all these parameters via components and finally to formatCode(document.getText(), <parameters>, but I don't know how.

This sounds more like a generic question related to React and/or the Monaco Editor than the LSP.

@rcjsuen is correct. The LSP only defines formatting options that are language independent. Language specific formatting options can be passed but are subject to the client and server.

Objections to close the issue?

If you know, please give me some hints about how "language specific formatting options can be passed"...

@chengtie Not 100% sure what you need/want, but I think probably you should use the 'configuration' pieces of the protocol.
Either this: (older and somewhat deprecated)
https://microsoft.github.io/language-server-protocol/specifications/specification-current/#workspace_didChangeConfiguration
Or this: (intended to replace the older part of the protocol)
https://microsoft.github.io/language-server-protocol/specifications/specification-current/#workspace_configuration

@chengtie I answered on the issue you created in the vscoe-languageserver-node repository

@kdvolder thanks for the pointers !
Another one is: [key: string]: boolean | number | string; here: https://microsoft.github.io/language-server-protocol/specifications/specification-current/#textDocument_formatting


Switch ZTF_get_lightcurve to get data from parquet files

Closes #107

Get data from ZTF parquet files (dr17) instead of light curve service
Use TAP to lookup the ZTF field, CCD, and quadrant info needed to find the object in the parquet files
The structure of the loaded data is significantly different than what was previously being loaded from the light curve service, so the rest of the code has been updated to accommodate it.
This no longer makes plots or displays a tqdm progress bar. This is because it no longer does a single for-loop over the objects. I think those were mostly for debugging, but if they are actually still wanted I can put them back in.

With these changes, one run of the Yang sample (30 objects) took about 2 minutes which is on-par (but at the high end) with the other functions grabbing light curves.
To do before merging

[x] handle "repeated lightcurves"
[ ] convert hmjd -> mjd
[ ] check that this returns the same results as the original code

Details
Data is fetched in two steps:

For each object:  Call TAP service to lookup the ZTF field, etc. needed to find this object in the parquet files.

Takes ~3 seconds per object. estimating 17 days for 500,000 objects (without parallelization)
TAP service limits to one object per call

For each parquet file:  Load data using a filter for objectid (returned by Step 1)

Takes ~30-45 seconds for all files (Yang sample, 30 objects)
This probably scales better than 1-to-1 with the number of objects because multiple objects will be pulled from each file.

Thoughts on parallelization (future work)
Right now, Step 1 takes 3x longer than Step 2.
One option would be to run Step 1 continuously as a background job, periodically returning batches of results. The main process could then run each batch through Step 2 and be done before the next batch is returned.
If that is too slow, Step 1 could be parallelized by making several simultaneous TAP calls. If these are returned faster than Step 2 can handle serially, run batches in parallel by launching more background workers.

@xoubish a comment in the original code says:

## Neeeded to remove repeated lightcurves in each band. Will keep only the longest following (Sanchez-Saez et al., 2021)

Would this "repeated lightcurves" problem be solved by using just the single object that is closest to the sample coordinates? Or do we really want the object with the longest light curve, even if it's not the closest spatially?

That is a valid point. Please check and if by changing the coordinate match strategy only one light curve per waveband is left you are right. I thought that there might have been multiple light curves saved per band for the same object.

Afaik there may be duplicates for objects that fall on multiple chips?

You are right. Just checked the mentioned paper: "When doing this for a particular object, the ZTF API can return
more than one light curve in a given filter. This happens because ZTF treats independently the light curves observed in a particular field, filter, and CCD-quadrant, and thus, if a source appears in more than one combination of these three, ZTF will provide more than one light curve. "

The original code was returning MJD, but these parquet files have HMJD ("Heliocentric-based Modified Julian Date"). Does anyone know how to convert HMJD -> MJD? I don't see this method in astropy or anywhere else.

@xoubish The Sánchez-Sáez paper says they keep the light curve with longest timespan:

to avoid having multiple light curves for any particular source, we only used the longest light curve (in terms of its timespan) of each filter of a particular object.

I interpreted this as meaning the largest difference between max and min MJD. But if I'm reading your original code right, I think you were keeping the one with the largest number of data points. Can you confirm which one you actually want?

The original code was returning MJD, but these parquet files have HMJD ("Heliocentric-based Modified Julian Date"). Does anyone know how to convert HMJD -> MJD? I don't see this method in astropy or anywhere else.

would this help? https://docs.astropy.org/en/stable/time/index.html#barycentric-and-heliocentric-light-travel-time-corrections

(I somewhat fail to see why they used heliocentric, rather than barycentric time, but I suppose the difference small enough to treat it as barycentric. Then there are already a couple of scales for it in astropy: https://docs.astropy.org/en/stable/time/index.html#time-scale (note that mjd is a format, while here we talk about the scale, nevertheless I would suggest to check out both sections in the docs as it clears up the concepts nicely). )

@bsipocz thank you.
Can I ask a scientist (maybe @xoubish ?) to take on the task of converting the HMJD into whatever format you want in the MultiIndexDFObject? I can leave it as HMJD for now and open an issue to remind us.

I suppose if we properly parse stuff into an astropy Time instance, then it should work out of the box. But then of course conversions need to be done when moving to pandas, etc as those know nothing about astronomical times.

I confirmed with Shooby that she wants the light curve with the most datapoints, not (necessarily) the one with the largest delta-MJD. I'm making that change now.
Unless there are objections, I'll merge this by the end of the day so it can be included in tomorrow's HQ demo dry run (@xoubish @bsipocz @jkrick?). I'll open Issues for anything that still needs to be done, but I think nothing that remains should prevent this from being included in the HQ demo.

No objections, but please confirm that this doesn't cause any problems downstream, since I might not have the chance to check it.

please confirm that this doesn't cause any problems downstream

Meaning, 'make sure the whole notebook runs and the light curve plots look reasonable'?

Spoke with Jessica and decided to leave HMJD as-is and add a note that if the science depends on precise times this will need to be corrected.

Reading the parquet files over HTTP turned out to be much slower from the SMCE than reported in the PR's main comment (which was based on tests from the IRSA science platform), and is also not the recommended way to access those files (recommended to download). We've decided to upload the files to S3, though the specific bucket is still being determined. In the meantime, I've uploaded them to a testing bucket, and updated the code, so that this PR can move forward.

@jkrick @bsipocz this is ready, if you want to review. The files are still in-transit to the S3 bucket. Hopefully that will be complete by tomorrow morning. The Yang sample can be used for testing -- all of those files are in the bucket already.
I think this is still going to be slow relative to the other archive calls. I need to do more time testing to see how it scales. I have experimented with using a "helper" dataset to replace the TAP query, which I think I've mentioned offline. It looks like it could make that step 10 times faster.
This PR makes a necessary change in order to run at scale: using the parquet files instead of the light curve service. I think this PR should be merged and future improvements can go in other PRs.

I believe all the parquet files are in the S3 bucket now.

working on the IRSA SP:
in requirements.txt I added hpgeom and s3fs, but the second cell crashes with a few errors, it seems quite snarled.  last error is a PermissionError: Forbidden
I don't know how to solve this, so switching to SMCE...
working on the SMCE:
in requirements.txt I needed to add hpgeom to make the second cell in the notebook pull the ZTF functions
Then I can run the light curve call for a sample of 30 and it takes ~3min. This is almost exactly what the ZTF API query took on a fast ZTF API day from the IRSA SP.
I also ran it on some of the other literature coords to make sure it worked outside of our nominal sample and it did fine.
Other than the requirements update, looks good and can be merged when you are ready.
out of curiosity, you made 'service' a variable in ZTF_get_lightcurve . Does that mean there are other services?

Thanks @jkrick. This is in a temporary S3 bucket that is available automatically from the SMCE but requires user credentials for access from IRSA SP or elsewhere. I added an explanatory comment in the notebook.
Re: the service -- I'm not aware of another service. I made this a variable in case the call to pyvo.dal.TAPService is making an external call to the archive to create a client or something. No need to re-do such a call for every coord; that could negatively impact performance so better to do it once and send it in as a variable. I didn't track down whether it actually does make an external call, was just easier to be safe.


Photo is published without a signature (comment)

🤷♂ we can't do much on the web version, sorry.


Delete sites that are set to false

@TylerSheffels @millerdev i refactored the nginx conf a bit to delete any sites that are set to false. wondering if you think i should just delete everything in the sites-enabled dir even if it's not explicitly set to false in the conf (that would be slightly more difficult given the setup). this at least will provide us with a little less of a headache. still need to test on staging/prod. have just been testing this locally
cc: @gcapalbo

I don't feel like I have a lot of context here, so maybe not the best judge.


image-builder: FIXES repository RHSM that has to be supplied.

Image-builder use a general structure for a general repository where the rhsm has to be present.
Fixes THEEDGE-2127
Type of change
What is it?

[X] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Documentation update
[X] Tests update
[X] Refactor

Checklist:

[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[X] New and existing unit tests pass locally with my changes
[X] I run go fmt ./... to check that my code is properly formatted
[X] I run go vet ./... to check that my code is free of common Go style mistakes

Codecov Report

Merging #1039 (d821780) into main (d2c978f) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1039   +/-   ##
=======================================
  Coverage   30.33%   30.33%
=======================================
  Files          46       46
  Lines        5598     5598
=======================================
  Hits         1698     1698
  Misses       3664     3664
  Partials      236      236

Impacted Files
Coverage Δ

pkg/clients/imagebuilder/client.go
31.96% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d2c978f...d821780. Read the comment docs.


How to pass property id from Navigator.NavigationBar and how to access in RightButton of NavigationBarRouteMapper

How to pass property id from Navigator.NavigationBar and how to access in RightButton of NavigationBarRouteMapper
<Navigator.NavigationBar id={name} style={{backgroundColor: '#246dd5'}}
routeMapper={NavigationBarRouteMapper} />
RightButton(route, navigator, index, nextState) {
return (
<TouchableOpacity style={{flex: 1, justifyContent: 'center'}}
onPress={() => navigator.parentNavigator.push({id: route.id})}>
<Text style={{color: 'white', margin: 10,}}>
{route.title}

);
},

I'm not familiar with the Navigator myself but I'd recommend asking a question on Stack Overflow and tag it with react-native: http://stackoverflow.com/questions/tagged/react-native
Many people from the community hang out on Stack Overflow and will be able to see your question. Using Stack Overflow for questions also helps us use Github issues to keep track of bugs that need to be fixed.


Could you add LICENSE file?

Thanks for your excellent work!
We are using the repository for our project but having difficulty due to the missing license information.
It would be very helpful if you could add any license information. MIT or BSD v3 may be convenient for us (and other users) since there might be no dependency with GPL-licensed code. Then, we will refer to this repository from our work.
P.S. We found some errors in the part to organize the rotation poset but maybe we could fix them. We will send a pull request soon.

Thanks for bringing this to my attention. I've added a BSD 2-clause license (which is actually slightly more permissive than the the BSD 3-clause license that you mentioned).
I'd love it if you could identify or fix any bugs in the algorithms!

@maxkapur Thank you very much for your prompt action! An intern student of my team will send the pull request in a few days!


flash time

Hi
On a custom imx7 board we try to flash with uuu an 8GB sdimg. command:
FB: flash -raw2sparse all IMAGE.sdimg
This takes about 10 minutes.
Is there a way to speed up flashing? What is the recommended way of doing this? How to flash thousands of devices in a manufacturing process?
We want to keep manufacturing process as simple as possible and therefore want to avoid starting linux from ramdisk with partitioning, flashing, ...
Thanks in advance
Ruben

Try build one small sdimg.
I am not sure if there are good tools can convert to android sparse file format, which can discard some data, which file system don't care.
you connect many devices into difference port, so many devices can be programmed parallel.
Remove message from uart port.  Debug message takes some time to transfer to uart port even no connected.

Thanks @nxpfrankli for your response and sorry for late response.
regarding point 3 difference ports: I have to burn per-device data like mac-address and serial number. The values are provided by a script which is called from uuu with stdout (see link.
If I have multiple information (e.g. mac address needs 3 fuses to burn), how can I handle concurrency here? Is there a identifier from uuu which I can use as parameter for my script which provides mac addresses?

So far, no. maybe you need a global counter file by shared file.  I leave flexibility to user.

Do you have any idea how to share information between 2 calls from uuu script?
Otherwise i have to split into two steps: one long-running parallel flash job and a second, fast configure job which runs sequential

counter(){
    a=$1
    while true;do
        if [ ! -e /tmp/counter.lock ];then
            touch /tmp/counter.lock
            curr_count=`cat /tmp/count.txt`
            n_count=`echo "${curr_count}  + $a" | bc`
            echo ${n_count} > /tmp/count.txt
            rm -fv /tmp/counter.lock
            break
        fi
        sleep 1
    done
}


ci: Update workflows based on http-proxy's

This is technically a breaking change because we used to push images to Docker Hub (undocumented). This PR replaces it with the GitHub Container Registry. It's quite a hassle to push to both registries and Docker Hub's recent changes to announce removing free organizations and paddling backwards a few days later makes it the option with a less certain future. I want to rehaul the README entirely in a different PR.

Publishing to ghcr instead of dockerhub is fine as the old version is still available but perhaps we should mention this in an annoucement in the Discord server for all to see? Perhaps also in this repo's README?

I updated the README


found multiple with same username

OS: Docker Linux
Provider: LDAP
Settings file:

docker run
-e WebSettings__EnableHttpsRedirect='false'
-e AppSettings__UseAutomaticContext='false'
-e AppSettings__LdapSearchBase='DC=xxx,DC=xxx'
-e AppSettings__LdapHostnames__0='mypdc.xxx.xxx'
-e AppSettings__DefaultDomain='xxx.xxx'
-e AppSettings__IdTypeForUser='SAM'
-e AppSettings__LdapPort='389'
-e AppSettings__LdapUsername='DN of the ldap admin user'
-e AppSettings__LdapPassword='password of the ldap user'
-e ClientSettings__UseEmail='false'
-it
-p 80:80 passcore:latest
Describe the bug
the user is never found. I use the samAccountName in the form.
in log I find:
found multiple with same username: [vittoriam]
but the user with samAccountName vittoriam is clearly unique
the log is the following:
Hosting environment: Production ,Content root path: /app ,Now listening on: http://[::]:80 ,Application started. Press Ctrl+C to shut down. ,info: Microsoft.AspNetCore.Hosting.Diagnostics[1] ,      Request starting HTTP/1.1 GET http://dockerdmz-test.sit.local/api/password   ,info: Microsoft.AspNetCore.Routing.EndpointMiddleware[0] ,      Executing endpoint 'Unosquare.PassCore.Web.Controllers.PasswordController.Get (Unosquare.PassCore.Web)' ,info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[3] ,      Route matched with {action = "Get", controller = "Password"}. Executing controller action with signature Microsoft.AspNetCore.Mvc.IActionResult Get() on controller Unosquare.PassCore.Web.Controllers.PasswordController (Unosquare.PassCore.Web). ,warn: Zyborg.PassCore.PasswordProvider.LDAP.LdapPasswordChangeProvider[0] ,      option [LdapIgnoreTlsErrors] is ENABLED; invalid certificates will be allowed ,warn: Zyborg.PassCore.PasswordProvider.LDAP.LdapPasswordChangeProvider[0] ,      option [LdapStartTls] is DISABLED in combination with non-standard TLS port [389] ,info: Microsoft.AspNetCore.Mvc.Infrastructure.SystemTextJsonResultExecutor[1] ,      Executing JsonResult, writing value of type 'Unosquare.PassCore.Web.Models.ClientSettings'. ,info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[2] ,      Executed action Unosquare.PassCore.Web.Controllers.PasswordController.Get (Unosquare.PassCore.Web) in 70.8579ms ,info: Microsoft.AspNetCore.Routing.EndpointMiddleware[1] ,      Executed endpoint 'Unosquare.PassCore.Web.Controllers.PasswordController.Get (Unosquare.PassCore.Web)' ,info: Microsoft.AspNetCore.Hosting.Diagnostics[2] ,      Request finished in<PHONE_NUMBER>0000002ms 200 application/json; charset=utf-8 ,info: Microsoft.AspNetCore.Hosting.Diagnostics[1] ,      Request starting HTTP/1.1 POST http://dockerdmz-test.sit.local/api/password application/json 127 ,info: Microsoft.AspNetCore.Routing.EndpointMiddleware[0] ,      Executing endpoint 'Unosquare.PassCore.Web.Controllers.PasswordController.Post (Unosquare.PassCore.Web)' ,info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[3] ,      Route matched with {action = "Post", controller = "Password"}. Executing controller action with signature System.Threading.Tasks.Task1[Microsoft.AspNetCore.Mvc.IActionResult] Post(Unosquare.PassCore.Web.Models.ChangePasswordModel) on controller Unosquare.PassCore.Web.Controllers.PasswordController (Unosquare.PassCore.Web).
,warn: Zyborg.PassCore.PasswordProvider.LDAP.LdapPasswordChangeProvider[0]
,      found multiple with same username: [vittoriam]
,info: Microsoft.AspNetCore.Mvc.Infrastructure.ObjectResultExecutor[1]
,      Executing ObjectResult, writing value of type 'Unosquare.PassCore.Web.Models.ApiResult'.
,info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[2]
,      Executed action Unosquare.PassCore.Web.Controllers.PasswordController.Post (Unosquare.PassCore.Web) in 251.6901ms
,info: Microsoft.AspNetCore.Routing.EndpointMiddleware[1]
,      Executed endpoint 'Unosquare.PassCore.Web.Controllers.PasswordController.Post (Unosquare.PassCore.Web)'
,info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
,      Request finished in 286.1524ms 400 application/json; charset=utf-8
,`

Can you retest against the master branch?

I updated to the master branch
Fast-forward
.github/ISSUE_TEMPLATE/Bug_report.md               |   1 +
.../Unosquare.PassCore.Common.csproj               |   2 +-
.../Unosquare.PassCore.PasswordProvider.csproj     |   2 +-
.../ClientApp/Components/ChangePasswordForm.tsx    |   2 +-
.../ClientApp/package-lock.json                    | 104 ++++++++++----------
src/Unosquare.PassCore.Web/ClientApp/package.json  |   8 +-
.../Unosquare.PassCore.Web.csproj                  |   2 +-
.../LdapPasswordChangeProvider.cs                  | 106 ++++++++++-----------
.../Zyborg.PassCore.PasswordProvider.LDAP.csproj   |   2 +-
9 files changed, 110 insertions(+), 119 deletions(-)
But the problem remains.

Can you post the log?
On Sun, Nov 3, 2019, 4:54 AM nikinname<EMAIL_ADDRESS>wrote:

I updated to the master branch
Fast-forward
.github/ISSUE_TEMPLATE/Bug_report.md | 1 +
.../Unosquare.PassCore.Common.csproj | 2 +-
.../Unosquare.PassCore.PasswordProvider.csproj | 2 +-
.../ClientApp/Components/ChangePasswordForm.tsx | 2 +-
.../ClientApp/package-lock.json | 104 ++++++++++----------
src/Unosquare.PassCore.Web/ClientApp/package.json | 8 +-
.../Unosquare.PassCore.Web.csproj | 2 +-
.../LdapPasswordChangeProvider.cs | 106 ++++++++++-----------
.../Zyborg.PassCore.PasswordProvider.LDAP.csproj | 2 +-
9 files changed, 110 insertions(+), 119 deletions(-)
But the problem remains.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/unosquare/passcore/issues/397?email_source=notifications&email_token=AANRRMGTAA7VJXV6CEKEMNDQR2UXFA5CNFSM4JGPTPN2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEC5PWNY#issuecomment-549124919,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AANRRMBKEKS2XG5544RMQXTQR2UXFANCNFSM4JGPTPNQ
.

Yes,
I repeat the command I used to run docker:
docker run  -e WebSettings__EnableHttpsRedirect='false'  -e AppSettings__UseAutomaticContext='false' -e AppSettings__LdapSearchBase='DC=sit,DC=local' -e AppSettings__LdapHostnames__0='mypdcfqdn' -e AppSettings__DefaultDomain='MYDOMAIN' -e AppSettings__IdTypeForUser='SAM' -e AppSettings__LdapPort='389' -e AppSettings__LdapUsername='DN of an admin user' -e AppSettings__LdapPassword='somepassword' -e ClientSettings__ChangePasswordForm__UsernameLabel='Inserire il proprio nome utente' -e ClientSettings__UseEmail='false' -it -p 80:80 passcore:latest
Of corse I changed sensible info
This it the log trying wit an user that has samAccountName = vittoriam
Application started. Press Ctrl+C to shut down.
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://dockerdmz-test.sit.local/
info: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[6]
The file /index.html was not modified
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 30.558ms 304 text/html
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://dockerdmz-test.sit.local/App.54a8830a.js
info: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[6]
The file /App.54a8830a.js was not modified
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 1.0391000000000001ms 304 application/javascript
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://dockerdmz-test.sit.local/react.fc4dbac6.js
info: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[2]
Sending file. Request path: '/react.fc4dbac6.js'. Physical path: '/app/wwwroot/react.fc4dbac6.js'
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 20.9018ms 200 application/javascript
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://dockerdmz-test.sit.local/react-dom.66c8cbbe.js
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://dockerdmz-test.sit.local/main.21408868.js
info: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[6]
The file /main.21408868.js was not modified
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 0.6124ms 304 application/javascript
info: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[2]
Sending file. Request path: '/react-dom.66c8cbbe.js'. Physical path: '/app/wwwroot/react-dom.66c8cbbe.js'
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 95.8014ms 200 application/javascript
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://dockerdmz-test.sit.local/favicon.ico
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 41.6544ms 0
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://dockerdmz-test.sit.local/api/password
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[0]
Executing endpoint 'Unosquare.PassCore.Web.Controllers.PasswordController.Get (Unosquare.PassCore.Web)'
info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[3]
Route matched with {action = "Get", controller = "Password"}. Executing controller action with signature Microsoft.AspNetCore.Mvc.IActionResult Get() on controller Unosquare.PassCore.Web.Controllers.PasswordController (Unosquare.PassCore.Web).
warn: Zyborg.PassCore.PasswordProvider.LDAP.LdapPasswordChangeProvider[0]
option [LdapIgnoreTlsErrors] is ENABLED; invalid certificates will be allowed
warn: Zyborg.PassCore.PasswordProvider.LDAP.LdapPasswordChangeProvider[0]
option [LdapStartTls] is DISABLED in combination with non-standard TLS port [389]
info: Microsoft.AspNetCore.Mvc.Infrastructure.SystemTextJsonResultExecutor[1]
Executing JsonResult, writing value of type 'Unosquare.PassCore.Web.Models.ClientSettings'.
info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[2]
Executed action Unosquare.PassCore.Web.Controllers.PasswordController.Get (Unosquare.PassCore.Web) in 65.2153ms
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[1]
Executed endpoint 'Unosquare.PassCore.Web.Controllers.PasswordController.Get (Unosquare.PassCore.Web)'
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 134.592ms 200 application/json; charset=utf-8
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://dockerdmz-test.sit.local/passcore-logo.265a699e.png
info: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[2]
Sending file. Request path: '/passcore-logo.265a699e.png'. Physical path: '/app/wwwroot/passcore-logo.265a699e.png'
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 7.0502ms 200 image/png
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://dockerdmz-test.sit.local/osi.38a8cb73.png
info: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[2]
Sending file. Request path: '/osi.38a8cb73.png'. Physical path: '/app/wwwroot/osi.38a8cb73.png'
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 3.5129ms 200 image/png
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://dockerdmz-test.sit.local/License_icon-mit.svg.fd4b276c.png
info: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[2]
Sending file. Request path: '/License_icon-mit.svg.fd4b276c.png'. Physical path: '/app/wwwroot/License_icon-mit.svg.fd4b276c.png'
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://dockerdmz-test.sit.local/logo.c481326b.png
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 4.2596ms 200 image/png
info: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[2]
Sending file. Request path: '/logo.c481326b.png'. Physical path: '/app/wwwroot/logo.c481326b.png'
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 2.3969ms 200 image/png
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 POST http://dockerdmz-test.sit.local/api/password application/json 127
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[0]
Executing endpoint 'Unosquare.PassCore.Web.Controllers.PasswordController.Post (Unosquare.PassCore.Web)'
info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[3]
Route matched with {action = "Post", controller = "Password"}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.IActionResult] Post(Unosquare.PassCore.Web.Models.ChangePasswordModel) on controller Unosquare.PassCore.Web.Controllers.PasswordController (Unosquare.PassCore.Web).
warn: Zyborg.PassCore.PasswordProvider.LDAP.LdapPasswordChangeProvider[0]
found multiple with same username: [vittoriam]
info: Microsoft.AspNetCore.Mvc.Infrastructure.ObjectResultExecutor[1]
Executing ObjectResult, writing value of type 'Unosquare.PassCore.Web.Models.ApiResult'.
info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[2]
Executed action Unosquare.PassCore.Web.Controllers.PasswordController.Post (Unosquare.PassCore.Web) in 215.834ms
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[1]
Executed endpoint 'Unosquare.PassCore.Web.Controllers.PasswordController.Post (Unosquare.PassCore.Web)'
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in<PHONE_NUMBER>0000002ms 400 application/json; charset=utf-8

could anyone help?

found multiple with same username: [vittoriam]```

This means that you have multiple usernames in your active directory that contain the same SamAccountName. Either limit your `LdapSearchBase` or Fix the duplicate user.

Of course there is only one! I have checked
PS C:\WINDOWS\system32> Get-ADUser -Filter 'samaccountname -eq "vittoriam"' | select samaccountname

samaccountname
--------------
vittoriam

The error is somewere else

@nikinname Can you run the same command with the current master branch and provide logs.
Thanks

Application started. Press Ctrl+C to shut down.
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://dockerdmz-test.sit.local/
info: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[6]
The file /index.html was not modified
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 27.201500000000003ms 304 text/html
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://dockerdmz-test.sit.local/App.54a8830a                                                                                .js
info: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[6]
The file /App.54a8830a.js was not modified
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 1.5186000000000002ms 304 application/javascript
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://dockerdmz-test.sit.local/react.fc4dba                                                                                c6.js
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://dockerdmz-test.sit.local/react-dom.66                                                                                c8cbbe.js
info: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[2]
Sending file. Request path: '/react.fc4dbac6.js'. Physical path: '/app/www                                                                                root/react.fc4dbac6.js'
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 13.916400000000001ms 200 application/javascript
info: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[2]
Sending file. Request path: '/react-dom.66c8cbbe.js'. Physical path: '/app                                                                                /wwwroot/react-dom.66c8cbbe.js'
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 14.638300000000001ms 200 application/javascript
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://dockerdmz-test.sit.local/main.2140886                                                                                8.js
info: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[6]
The file /main.21408868.js was not modified
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 0.3779ms 304 application/javascript
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://dockerdmz-test.sit.local/favicon.ico
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 42.042ms 0
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://dockerdmz-test.sit.local/api/password
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[0]
Executing endpoint 'Unosquare.PassCore.Web.Controllers.PasswordController.                                                                                Get (Unosquare.PassCore.Web)'
info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[3]
Route matched with {action = "Get", controller = "Password"}. Executing co                                                                                ntroller action with signature Microsoft.AspNetCore.Mvc.IActionResult Get() on c                                                                                ontroller Unosquare.PassCore.Web.Controllers.PasswordController (Unosquare.PassC                                                                                ore.Web).
warn: Zyborg.PassCore.PasswordProvider.LDAP.LdapPasswordChangeProvider[0]
option [LdapIgnoreTlsErrors] is ENABLED; invalid certificates will be allo                                                                                wed
warn: Zyborg.PassCore.PasswordProvider.LDAP.LdapPasswordChangeProvider[0]
option [LdapStartTls] is DISABLED in combination with non-standard TLS por                                                                                t [389]
info: Microsoft.AspNetCore.Mvc.Infrastructure.SystemTextJsonResultExecutor[1]
Executing JsonResult, writing value of type 'Unosquare.PassCore.Web.Models                                                                                .ClientSettings'.
info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[2]
Executed action Unosquare.PassCore.Web.Controllers.PasswordController.Get                                                                                (Unosquare.PassCore.Web) in 74.06230000000001ms
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[1]
Executed endpoint 'Unosquare.PassCore.Web.Controllers.PasswordController.G                                                                                et (Unosquare.PassCore.Web)'
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 146.5329ms 200 application/json; charset=utf-8
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://dockerdmz-test.sit.local/passcore-log                                                                                o.265a699e.png
info: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[2]
Sending file. Request path: '/passcore-logo.265a699e.png'. Physical path:                                                                                '/app/wwwroot/passcore-logo.265a699e.png'
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 1.3931ms 200 image/png
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://dockerdmz-test.sit.local/osi.38a8cb73                                                                                .png
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://dockerdmz-test.sit.local/License_icon                                                                                -mit.svg.fd4b276c.png
info: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[2]
Sending file. Request path: '/osi.38a8cb73.png'. Physical path: '/app/wwwr                                                                                oot/osi.38a8cb73.png'
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 0.8613000000000001ms 200 image/png
info: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[2]
Sending file. Request path: '/License_icon-mit.svg.fd4b276c.png'. Physical                                                                                path: '/app/wwwroot/License_icon-mit.svg.fd4b276c.png'
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 1.0443ms 200 image/png
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://dockerdmz-test.sit.local/logo.c481326                                                                                b.png
info: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[2]
Sending file. Request path: '/logo.c481326b.png'. Physical path: '/app/www                                                                                root/logo.c481326b.png'
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 0.43510000000000004ms 200 image/png
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 POST http://dockerdmz-test.sit.local/api/passwor                                                                                d application/json 127
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[0]
Executing endpoint 'Unosquare.PassCore.Web.Controllers.PasswordController.                                                                                Post (Unosquare.PassCore.Web)'
info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[3]
Route matched with {action = "Post", controller = "Password"}. Executing c                                                                                ontroller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCo                                                                                re.Mvc.IActionResult] Post(Unosquare.PassCore.Web.Models.ChangePasswordModel) on                                                                                controller Unosquare.PassCore.Web.Controllers.PasswordController (Unosquare.Pas                                                                                sCore.Web).
warn: Zyborg.PassCore.PasswordProvider.LDAP.LdapPasswordChangeProvider[0]
found multiple with same username: [vittoriam]
info: Microsoft.AspNetCore.Mvc.Infrastructure.ObjectResultExecutor[1]
Executing ObjectResult, writing value of type 'Unosquare.PassCore.Web.Mode                                                                                ls.ApiResult'.
info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[2]
Executed action Unosquare.PassCore.Web.Controllers.PasswordController.Post                                                                                (Unosquare.PassCore.Web) in<PHONE_NUMBER>0000002ms
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[1]
Executed endpoint 'Unosquare.PassCore.Web.Controllers.PasswordController.P                                                                                ost (Unosquare.PassCore.Web)'
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 260.6336ms 400 application/json; charset=utf-8
^CApplication is shutting down...

I am still bangin my head please let me know if there something wrong or missing in my config
in the ezample below I obviously changed domain name and passwords the assumption is:
domain = my.domain
I use samAccountName to login
docker run
-e WebSettings__EnableHttpsRedirect='false'
-e AppSettings__LdapSearchBase='DC=my,DC=domain'
-e AppSettings__DefaultDomain='my.domain'
-e AppSettings__LdapSecureSocketLayer='false'
-e AppSettings__LdapStartTls='false'
-e AppSettings__LdapSearchFilter='"(sAMAccountName={Username})"'
-e AppSettings__LdapHostnames__0='mypdc.my.domain'
-e AppSettings__LdapPort='389'
-e AppSettings__LdapUsername='CN=some user,OU=someOU,DC=my,DC=domain'
-e AppSettings__LdapPassword='someverysecretpassword'
-e AppSettings__LdapSearchFilter='(sAMAccountName={Username})'
-e ClientSettings__UseEmail='false'
-it -p 80:80 passcore:latest


api-dev is returning bad user information

This is causing failures for several proposals. Tested only against 2024-1 cycle so far. Some examples of errors are:
None, lsdc-dmx, n2sn-dataadmin-sst1 (should be sst), I think that's most/all I've seen, but I may have missed others as I didn't run against all proposals..
Error while processing proposal 313306.
Traceback (most recent call last):
  File "/tmp/test-proposals/create_proposal_directories.py", line 835, in write_posix_acls
    acl = posix1e.ACL(text=",".join(acl_entries))
OSError: [Errno 22] Invalid argument

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/tmp/test-proposals/create_proposal_directories.py", line 455, in main
    process_proposal(
  File "/tmp/test-proposals/create_proposal_directories.py", line 203, in process_proposal
    update_acls(
  File "/tmp/test-proposals/create_proposal_directories.py", line 889, in update_acls
    write_posix_acls(
  File "/tmp/test-proposals/create_proposal_directories.py", line 838, in write_posix_acls
    raise RuntimeError(
RuntimeError:
ACL object creation failed (POSIX). Details:

u::rwx,u:nsls2data:rwx,u:None:rwx,u:None:rwx,g::rwx,g:n2sn-right-dataadmin:rwx,g:n2sn-right-dataadmin-tes:rwx,o::---

Error while processing proposal 313683.
Traceback (most recent call last):
  File "/tmp/test-proposals/create_proposal_directories.py", line 835, in write_posix_acls
    acl = posix1e.ACL(text=",".join(acl_entries))
OSError: [Errno 22] Invalid argument

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/tmp/test-proposals/create_proposal_directories.py", line 455, in main
    process_proposal(
  File "/tmp/test-proposals/create_proposal_directories.py", line 256, in process_proposal
    update_acls(
  File "/tmp/test-proposals/create_proposal_directories.py", line 889, in update_acls
    write_posix_acls(
  File "/tmp/test-proposals/create_proposal_directories.py", line 838, in write_posix_acls
    raise RuntimeError(
RuntimeError:
ACL object creation failed (POSIX). Details:

u:softioc-sst1:rwx,u:softioc:rwx,u:bluesky-sst1:rwx,u:workflow-sst1:r-x,u::r-x,u:nsls2data:r-x,g:n2sn-dataadmin-sst1:r-x,g:n2sn-dataadmin:r-x,g::r-x,o::---

Error while processing proposal 314563.
Traceback (most recent call last):
  File "/tmp/test-proposals/create_proposal_directories.py", line 835, in write_posix_acls
    acl = posix1e.ACL(text=",".join(acl_entries))
OSError: [Errno 22] Invalid argument

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/tmp/test-proposals/create_proposal_directories.py", line 455, in main
    process_proposal(
  File "/tmp/test-proposals/create_proposal_directories.py", line 203, in process_proposal
    update_acls(
  File "/tmp/test-proposals/create_proposal_directories.py", line 889, in update_acls
    write_posix_acls(
  File "/tmp/test-proposals/create_proposal_directories.py", line 838, in write_posix_acls
    raise RuntimeError(
RuntimeError:
ACL object creation failed (POSIX). Details:

u::rwx,u:nsls2data:rwx,u:None:rwx,u:None:rwx,g::rwx,g:n2sn-right-dataadmin:rwx,g:n2sn-right-dataadmin-fis:rwx,o::---

Error while processing proposal 311875.
Traceback (most recent call last):
  File "/tmp/test-proposals/create_proposal_directories.py", line 835, in write_posix_acls
    acl = posix1e.ACL(text=",".join(acl_entries))
OSError: [Errno 22] Invalid argument

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/tmp/test-proposals/create_proposal_directories.py", line 455, in main
    process_proposal(
  File "/tmp/test-proposals/create_proposal_directories.py", line 203, in process_proposal
    update_acls(
  File "/tmp/test-proposals/create_proposal_directories.py", line 889, in update_acls
    write_posix_acls(
  File "/tmp/test-proposals/create_proposal_directories.py", line 838, in write_posix_acls
    raise RuntimeError(
RuntimeError:
ACL object creation failed (POSIX). Details:

u::rwx,u:nsls2data:rwx,u:workflow-fmx:rwx,u:softioc-fmx:rwx,u:lsdc-dmx:rwx,g::rwx,g:n2sn-right-dataadmin:rwx,g:n2sn-right-dataadmin-fmx:rwx,o::---

Failed proposals:
313306
313683
314563
311875
Proposals successfully handled: 6

@stuartcampbell for visibility

lsdc-dmx was just a typo in the database


zilencer: Disambiguate no MX records from the domain not existing.

This switches to dnspython, since it offers the higher-level
resolve_name method to look up both A and AAAA records, and set
timeouts.

Self-review checklist

[ ] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[ ] Explains differences from previous plans (e.g., issue description).
[ ] Highlights technical choices and bugs encountered.
[ ] Calls out remaining decisions and concerns.
[ ] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[ ] Each commit is a coherent idea.
[ ] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[ ] Visual appearance of the changes.
[ ] Responsiveness and internationalization.
[ ] Strings and tooltips.
[ ] End-to-end functionality of buttons, interactions and flows.
[ ] Corner cases, error conditions, and easily imagined bugs.

Looks good to me, marked to merge once CI passes. Thanks @alexmv!


Remove the references completed boolean

Context
We know longer uses the references_completed boolean to indicate that the references section is complete.
It needs to be removed as part of pulling out the decoupled references flag
Changes proposed in this pull request

Migration to drop the refererences_completed column
Pull out all the associated code

Link to Trello card
https://trello.com/c/pizcHLeY/2307-🚢-💔-epic-remaining-dev-work-for-decoupled-references
Things to check

[ ] This code does not rely on migrations in the same Pull Request
[x] If this code includes a migration adding or changing columns, it also backfills existing records for consistency
[x] API release notes have been updated if necessary
[x] New environment variables have been added to the Azure config

LGTM. I think there's a chance of errors in DuplicateApplication unless you deploy the code removal first and then merge the column drop after. @tijmenb - can you confirm?

What @malcolmbaig brings up makes sense; @davidgisbey could we split out the two steps into two PRs? :+1:

Yea 100% I'll do it now.


Support multiple reverse foreign key relationships for a single table

This will work by seeking out tables that have a foreign key reference back to the current table.
What about cases where a table has multiple references back? For example:
articles
--------
id
headline
body
created_by - fk to user
updated_by - fk to user

If there was only one fk to user, the users table could have a articles_list field pointing back to it.
In this case, I will instead create these two fields:
articles_created_by_list
articles_updated_by_list

So a GraphQL query could look like this:
{
  users(first: 5) {
    nodes {
      name
      articles_created_by_list(first: 5) {
        totalCount
        nodes {
          id
          title
        }
      }
      articles_updated_by_list(first: 5) {
        totalCount
        nodes {
          id
          title
        }
      }
    }
  }
}

Originally posted by @simonw in https://github.com/simonw/datasette-graphql/issues/27#issuecomment-669975859

Demo: https://datasette-graphql-demo.datasette.io/graphql/fixtures?query=
{
  users {
    nodes {
      id
      name
      issues_by_user_list {
        nodes {
          id
          title
          updated_by {
            name
          }
          user {
            name
          }
        }
      }
      issues_by_updated_by_list {
        nodes {
          id
          title
          updated_by {
            name
          }
          user {
            name
          }
        }
      }
    }
  }
}

Docs: https://github.com/simonw/datasette-graphql/blob/main/examples/related_multiple.md


Chess: Move F7 to H6

Please do not change the title. Just click "Submit new issue".

@tmchuynh done! Successfully played move f7h6 for current game.
Thanks for playing!


.gitignore: init and ignore bin folder

Signed-off-by: Wei Fu<EMAIL_ADDRESS>

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Don't force semver tag to start with a 'v'

Is your feature request related to a problem? Please describe.
In the majority of the project I can see the semver tag doesn't start with a v, and I see no good reason why they must.
When I work on an open project if I want to use your auto-versioning tool I must ask to start the semver tag with a v, only for the use of your tool.
Describe the solution you'd like
Just check if the tag matches the regex ^((\d+)\.(\d+)\.(\d+)), and don't force the use of 'v'.

@cbcrouse because this is an opinionated choice you impose. The best would be to let the user choose it's own format.
For instance, take the ASPNetCore repo, the tags are in the format of .NET x.x.x, in OData it's ASP.NET Core OData x.x.x
I'd suggest to either add a argument like --format-tag 'My Tag v##Version##' (or any kind of templating synthax) or add to arguments like --tag-prefix and --tag-suffix e.g.: --tag-prefix 'v' --tag-suffix '-alpha'

Thanks for the additional clarification @JBBianchi. The ASPNetCore link you mentioned, the tag is not actually .NET x.x.x, but rather vX.X.X-suffix, e.g., v7.0.0-rc.1. The same is similar for the OData link you shared. The values you mentioned are simply the release title they are using which can be typed in manually.

However, I do see the opportunity here for increased flexibility with the tool and will spend some time considering the best approach for this. I would like to keep this tag as close to semantic versioning standards as possible.

Awsome, thanks!