Proposal: Extend Auth model to branches

There are use-cases where assigning access rights on a per-branch basis would be useful. Given that inputs are defined per-branch (and we've moved to a more branch-centric model with 1.7), this seems to make sense. I imagine this complementing repo access controls:

read access to a branch would allow you to read any commits on that branch
write access to a branch would allow you to add commits on that branch (but only repoint the branch at a new commit)
write access to a repo would be required to re-point a branch at any arbitrary commit (though pointing at earlier commits on that branch could possibly be allowed with only branch access)
read access to a repo would allow you read access to all branches on that repo

Use case 1: Giving each user a branch on an ingress repo, so users both don't step on each others' toes, and so we know which data came from where (this second use-cases is, admittedly, basically a workaround for not having a notion of user-based provenance/auditing, ie tying changes in the system to a specific user, but seems like a decent one if it's specifically data auditing that is of concern)
The synchronization use-case, however, would also be relevant for machine users
Use case 2: storing per-user data for the dash. As discussed with @msteffen , there is some config data for the dash that it would be useful to store on a per-user basis (whether they've already gone through a tour, display preferences, if they've dismissed notifications, etc). These don't really have anything to do with pach core, so probably don't belong in pach APIs/in etcd. I could write a separate web service to store these config values, but it would be more convenient (and -- since they're infrequently written and latency-insensitive -- equally effective) to just store them in pach. I could create a repo per-user, but having a single __dashboard__ repo or something similar with a branch per user seems like it would be better than cluttering up the repo namespace with a bunch of __dashboard_username__ repos
Use case 3: This also solves the problem of requiring a copy pipeline after an anonymization pipeline with stats enabled: simply allow read access to the output master branch, but not to the stats branch
Open question: should there be a way to create new branches on a repo, without giving access to the existing data? This would be needed for use-case 2 (dash needs to create a per-user branch on their behalf when they first log in, but we don't want to give them access to other users' data). This makes more sense as a capability within the context of permissions/scopes in the new auth model proposed in #2291

This idea seems correct to me, @msteffen you know more about auth and have more of the design in your head so you'll know for sure if this is a good idea.
It seems we're gradually migrating all of our repo specific things into branch specific things, this has happened already with inputs and provenance. Maybe we should start building things to be per branch whenever we're planning to make them per repo, we seem to always change our mind later.


add pivithuru sandeepanie to contributors list

@deafheaven, would you like to reopen this pull request so that I can merge it to this project?


mrsk deploy gives me ERROR (NoMethodError): undefined method `remove' for "Mrsk::Commands::Builder::Multiarch":String

% mrsk deploy
Acquiring the deploy lock...
Log into image registry...
  INFO [91e5d11c] Running docker login -u [REDACTED] -p [REDACTED] as jason@localhost
  INFO [91e5d11c] Finished in 0.720 seconds with exit status 0 (successful).
  INFO [036062d9] Running docker login -u [REDACTED] -p [REDACTED] on <IP_ADDRESS>
  INFO [036062d9] Finished in 0.154 seconds with exit status 0 (successful).
Build and push app image...
Releasing the deploy lock...
  Finished all in 1.4 seconds
  ERROR (NoMethodError): undefined method `remove' for "Mrsk::Commands::Builder::Multiarch":String

    target.class.to_s.remove("Mrsk::Commands::Builder::").underscore.inquiry
                     ^^^^^^^

Rails 7.1.0.alpha
MRSK 0.15.1
for reference, I am writing this (WIP) setup guide for MRSK on EC2
https://jasonfleetwoodboldt.com/courses/rails-7-crash-course/mrsk-on-amazon-ec2

my config/deploy.yml file is
# Name of your application. Used to uniquely configure containers.
service: helios-panoptes

# Name of the container image.
image: jasonheliosdevshop/helios-panoptes

# Deploy to these servers.
servers:
  - <IP_ADDRESS>

# Credentials for your image host.
registry:
  username:
    - DOCKER_USERNAME
  password:
    - DOCKER_PASSWORD

env:
  secret:
    - RAILS_MASTER_KEY
ssh:
  user: ubuntu

builder:
  args:
    RUBY_VERSION: "<%= Pathname.pwd.join('.ruby-version').read.strip.delete_prefix('ruby-') %>"
    BUNDLER_VERSION: "<%= Bundler::LockfileParser.new(Pathname.pwd.join('Gemfile.lock').read).bundler_version %>"
    RAILS_MASTER_KEY: "<%= ENV['RAILS_MASTER_KEY'] %>"

String#remove is proved by active_support/core_ext/string/filters but it doesn't look this module is required anywhere.

Having a similar issue:

It looks like this recent change to Rails may be the cause of the issue
https://github.com/rails/rails/pull/48959

if rails is alpha not sure if they want Issue reports via Github. either way, are you sure it is in Rails? if so, I'll close here.

Yeah, but it seems to be an intended change so it's still on MRSK to fix the usage of it.

Kamal was relying on Rails to load the required module. This is not a great idea. As we have seen, our dependencies may choose to drop features we are using. Much better to be explicit. It's more wordy, but does not impact performance. Ruby will only load a file once. If it is already loaded, subsequent require calls will simply return nil.
This has been a good learning opportunity.


Create mailmask.me.md

I'm getting a pretty consistent 229.7KB --> https://tools.pingdom.com/#5e5069e61dc00000

@bradleytaunt You are correct. I could have sworn that it was less this morning.


read() on Windows might incorrectly block forever

First of all - good job on getting a lot of the Linux / OSX quirks out of the way! This one feels a lot better.
To me, especially the ability to cleanly exit a blocked reader when the port is closed was a pain and it seems to work flawlessly now. Previously to work around that issue, I would set a timeout on the reading and wrap the InputStream with a retry loop that would give up when closed. I was now going to get rid of this hack, but then ran into another problem: sometimes my read()s would block forever despite having pending data.
I investigated a little and found a problematic race condition in JTermiosImpl (Windows). Apparently, you are aware of it, since the code and the comments indicated that you tried to work around the fact that Windows would only give you "new data" notifications, as opposed to "have data" (a common problem with edge-triggered vs. level-triggered events). If data arrives between the time you poll and the time you WaitForMultipleObjects, you'll miss the event. However, the way you've worked around it relied on the user-set timeout - you'll block the client way more than necessary, or even worse (such as in my case) - forever! I'm not a Windows API expert, so I cannot advise you on a cleaner solution as far as this goes. However, at least on the Java front you can do better:
deadline = now + user timeout
do {
if (poll for events) break
timeout = min(MAX_LAG, deadline - now)
WaitForMultipleObjects(..., timeout)
} while (now < deadline)
// We will get here at most MAX_LAG time after an event occurred or deadline expired.
This is not very elegant, but at least obviously correct.
Keep up the good work!!!

Closing this for now as I've not heard back from anyone for a while -- feel free to re-open


Ip2366 on this board

This is a IP2366 based power/charger module
Any possibilities for this codes to work on that board?

@pls share BOM file. It is difficult to see the chip label in the photo you provided

@sheedyaja, okay, based on the components, the schematic seems similar to the IP2366. You can desolder these resistors (see the attached image) and use the points where they were connected as I2C_SCL, I2C_SDA, and I2C_INT

However, be cautious—this might be a revision of the IP2366 that does not support I2C (previously discussed in issue #3).
That said, if you're lucky, you can use this library with your board.

@D-314 will try tomorrow, thanks

@D-314 update, it doesn't work

@sheedyaja, could you please share a photo with your connections?


Bad cert on https://docs.hoverfly.io/

I get a There are issues with the site's certificate chain (net::ERR_CERT_COMMON_NAME_INVALID). when visiting https://docs.hoverfly.io/.

Fixed


Prefer init_reg over reg2var for empty classical registers

reg2var is too verbose for empty registers. I would like to propose init_reg instead which  is not only concise but preserves valuable register size information.
I also added a Makefile and config files for pre-commit and ruff. While doing the latter, I ended up fixing some typos and improving some code.
I am happy to hear other suggestions such as perhaps using init_reg to completely replace reg2var by passing an initial value in the form of a decimal i64.

Have you discussed the changes with @peter-campora and if this will be supported by the backend in the same way?

Yes, once this new init_reg API is stabilized and accepted here, I will ensure the backend supports it as well.

I am not really sure if this is the right thing to do, I would assume that it might be better to keep the register size outside of qir. It is not really clear to me what the expected returns type are if we execute operations on registers with different sizes. Can you maybe give some more details on that?

The return types continue to be i64. Since the backend only consumes QIR, I believe that's the only place to store register size information, though I am open to alternatives if they exist. Think of the first argument to init_var as an additional metadata for the backend to distinguish it from real integers from other frontends such as Q# -> QIR. This extra information may also be used to decide do other optimizations at the backend (if needed).

We usually try to separate clean up / ci / infrastructure changes from new features. Do you think it would be possible to split this up into two PRs?

I can do that. But my logical changes are on top of the linting changes such as black pass, which seemed like was not run in quite some time. So if the linting changes look okay, I can create a PR for those first.

We usually try to separate clean up / ci / infrastructure changes from new features. Do you think it would be possible to split this up into two PRs?

I can do that. But my logical changes are on top of the linting changes such as black pass, which seemed like was not run in quite some time. So if the linting changes look okay, I can create a PR for those first.

The current used pylint and black is run on every PR, for me it looks like that ruff is just following a different convention.
I think it would be good to move that to a different PR. If there are benefits for changing the package we are using for that, we could think about changing that for our other pytket extensions as well.
This should be updated in the readme as well that we are now using ruff, see https://github.com/CQCL/pytket-qir#formatting
I think it would be good to have an additional PR for changing the ci to use ruff and run ruff once on the code to have a clean commit on develop which only contains the changes related to that. (And doing the precommit, makefile, ... changes in an additional separate PR after that.)
I am happy to help you with that, splitting this up, might be a little bit more work.

The current used pylint and black is run on every PR, for me it looks like that ruff is just following a different convention.
I think it would be good to move that to a different PR. If there are benefits for changing the package we are using for that, we could think about changing that for our other pytket extensions as well.
This should be updated in the readme as well that we are now using ruff, see https://github.com/CQCL/pytket-qir#formatting

It's possible that since I used the most recent version of black, it did things differently. But it seems compatible with what was on the CI.
BTW, ruff doesn't format things. Most of the formatting was performed by black. But I will double check that when I create a separate PR.

If the changes from you are done by just updating the black version we could do that as well and just add ruff in addition to that?

Created a separate PR for linting/automation improvements: https://github.com/CQCL/pytket-qir/pull/58


apple add ctl_info.

closes #3045.

Duplicate of https://github.com/rust-lang/libc/pull/3046?

true :)

Maybe this one is better?

@siegfried @JohnTitor whatever you prefer ;-)

Oh, I didn't check the diff carefully! Anyway, I don't mind either. For now, I'm going to merge #3046 but feel free to follow-up if you need these consts :)


--no-quarantine flag not working anymore

When brew cu is called with the --no-quarantine argument, app installations now fail with:
Usage: brew reinstall [options] formula

…

Error: invalid option: --no-quarantine

I suppose this is because of the change to brew reinstall instead of brew cask reinstall.

Hey @F30 , I did double check this and it seems it's working again. Therefore I'm closing this issue, feel free to create a new one if the issue occurs again.


[$250] [Held requests] Hold badge remains on paid expense until refresh the page

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 9.0.3-1
Reproducible in staging?: Y
Reproducible in production?: Y
If this was caught during regression testing, add the test name, ID and link from TestRail: https://expensify.testrail.io/index.php?/tests/view/4678849&group_by=cases:section_id&group_order=asc&group_id=309128
Email or phone of affected tester (no customers<EMAIL_ADDRESS>Issue reported by: Applause - Internal Team
Action Performed:

User A creates expense with User B
User B navigates to transaction thread and select Hold from 3-dot menu
User B pays the expense elsewhere
Take a note of Hold badge on the chat header on both User A and User B

Expected Result:
Since the expense is payed hold status should not be displayed anymore
Actual Result:
Hold status remains displayed even expense is payed
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos
Add any screenshot/video evidence

https://github.com/Expensify/App/assets/78819774/44a7d821-2080-46d1-a7f7-078f784eadbf
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~0197333211c4d3993e
Upwork Job ID:<PHONE_NUMBER>232770128
Last Price Increase: 2024-08-08

Issue OwnerCurrent Issue Owner: @getusha

@trjExpensify  FYI I haven't added the External label as I wasn't 100% sure about this issue. Please take a look and add the label if you agree it's a bug and can be handled by external contributors

Proposal
Please re-state the problem that we are trying to solve in this issue.
The hold banner doesn't disappear until we refresh the page for a paid expense.
What is the root cause of that problem?
When we pay the expense, we don't optimistically clear the hold status of the report. The pusher or API response doesn't clear the hold status either until we reload the page (from OpenReport).
What changes do you think we should make in order to solve the problem?

BE: clears the hold status from the API response and/or pusher.
FE: Optimistically clear the hold status
In IOU.getPayMoneyRequestParams, add below optimistic data:
...(full ? TransactionUtils.getAllReportTransactions(iouReport.reportID).map((transaction) => ({
    onyxMethod: Onyx.METHOD.MERGE,
    key: `${ONYXKEYS.COLLECTION.TRANSACTION}${transaction.transactionID}`,
    value: {
        comment: {
            hold: null,
        },
    }
})) : []),

(and revert it back in failureData)
What the code above does is, get all the expense report transactions and clear the hold status. We only do this if the full is true which means we pay all hold and non-hold requests.

I have a feeling @robertjchen is working on fixing this already. Maybe he can confirm.
Bit of a sidebar, but for the whack copy and stuff, you're talking care of that here: https://github.com/Expensify/App/pull/42736

Yes! This should be addressed with the optimistic changes in: https://github.com/Expensify/App/pull/42573
We can put this on hold until those changes are out and then re-test after to make sure this is fixed 👍

Cool, done!

Still on hold for https://github.com/Expensify/App/pull/42573

Same, melvin. Keep holdin', holdin', holdin'... yeah.

Alright, that PR has been deployed to prod: https://github.com/Expensify/App/pull/42573#issuecomment-2256646597

Taking this off hold. @lanitochka17 can we get a re-test please? Thanks!

Issue is still reproducible
https://github.com/user-attachments/assets/344382af-d423-4f2a-8c10-906f43b477ca

Cool, shipping on.

@getusha proposal from @bernhardoj here to review: https://github.com/Expensify/App/issues/44616#issuecomment-2197167370

@bernhardoj why not just check if the expense is settled to display the component?
index 812baee2d10..2441eb8e9db 100644
--- a/src/components/MoneyReportHeader.tsx
+++ b/src/components/MoneyReportHeader.tsx
@@ -196,7 +196,7 @@ function MoneyReportHeader({policy, report: moneyRequestReport, transactionThrea
     );

     const getStatusBarProps: () => MoneyRequestHeaderStatusBarProps | undefined = () => {
-        if (hasOnlyHeldExpenses) {
+        if (hasOnlyHeldExpenses && !ReportUtils.isSettled(transaction?.reportID)) {
             return {title: translate('violations.hold'), description: translate('iou.expensesOnHold'), danger: true};
         }
         if (allHavePendingRTERViolation) {

That works but that doesn't solve the issue that the transaction data still has the hold status which means if we call TransactionUtils.isOnHold, it will return true.

What do you think, @getusha?

@bernhardoj's proposal looks good to me
We could also apply a quick fix since it is a pretty straight-forward issue that exists until we refresh the page.
🎀 👀 🎀 C+ Reviewed!

Yep, agreed @bernhardoj's proposal looks good 👍

PR is ready
cc: @getusha

Looks like the backend is not sending Onyx updates when something is being unheld as part of a payment/approval operation. Testing out some solutions locally to see if it'll help here.

Got a potential BE fix in review

Fix was merged and should go out on Monday, we can re-test then.

Auth deploy was blocked, will re-test after the deploy.

Backend changes were deployed!

Tested in prod and confirmed fixed!
https://github.com/user-attachments/assets/11ecaf96-be90-4a32-ab63-7505822ff621

@robertjchen can we reopen this issue? We still have a pending PR that clears the hold optimistically.

Okay, so @getusha can proceed to review that PR now? https://github.com/Expensify/App/pull/47192

PR in final review

FE PR merged 👍

Payments are due here today. Payment summary as follows:

$250 to @getusha for the C+ review
$250 to @bernhardoj for the fix

Both of you can go ahead and request on NewDot, closing!

Requested in ND.

$250 approved for @bernhardoj


Patch

Seems like a cut-and-paste from the Create processing.  Info shouldn't be set on cleanup or close.

@microsoft-github-policy-service agree

Not sure who to report this to, but the automated CodeQL process that kicked off when I submitted the PR is hitting out of disk space errors...
https://github.com/jeffbromberger/Windows-driver-samples/actions/runs/4104889708

PRs should - barring the exceptional circumstance - go to 'develop' branch please.

Not sure who to report this to, but the automated CodeQL process that kicked off when I submitted the PR is hitting out of disk space errors...
https://github.com/jeffbromberger/Windows-driver-samples/actions/runs/4104889708

Thank you for mentioning. Known issue.  Working on fix.

PRs should - barring the exceptional circumstance - go to 'develop' branch please.

Sorry, first time contributing to this :)  Would you like me to close this one and create a new one to develop?

I'll go ahead and re-submit to develop

Jeff. Thank you!


Better user experience when signing up with an existing email

Right now, if you attempt to sign up with an email already associated with a user, you get a generic "something went wrong" message.
We should send the user an email informing them that they already have a user, and that they can log in with that.

Reviewed options and concluded with this:
We want to set up a flow that gives the same result when you register with an unused email and with a previously registered one. The user gets a message about needing to verify their account, and we send out an email to the user in both scenarios. If the user already has an account we ask them to log in or reset their password, and if it is a new user we ask them to verify the account.
We do not want to inform the user of the email already being in use in the registration form as this could pose a small security risk.


Odd behaviour of NC mesh with internal boundary faces

Hi all,
I am working on a model (related to https://github.com/mfem/mfem/issues/4545) for which I need to add internal boundary facets to perform some integration. The mesh is a non-conforming mixed mesh (involving HEX, TET, PRISMA, and PYRAMID elements).
I have been having some trouble with maintaining previously added boundary attributes or face orientations after GeneralRefinement of some domains.
The order of operations I am using is something like this:
# Create empty mesh
mesh = mfem.Mesh( 3, 0, 0 )

# Add vertices, elements, etc.
...

# Build faces and boundary
mesh.FinalizeTopology()
mesh.Finalize()

# Changing element and boundary attributes
...

# Add internal boundary facets used for integrators
...

mesh.SetAttributes()

# Finalize connectivity and topology (is this even needed)
mesh.FinalizeTopology()
mesh.Finalize(True)

# Make sure mesh is non-conforming
mesh.EnsureNCMesh( simplices_nonconforming = True )

# Make parallel mesh
pmesh = mfem.ParMesh(MPI.COMM_WORLD, mesh)

# Refinement
pmesh.GeneralRefinement(refinement_list)
pmesh.SetAttributes()

I realize the workflow seems a bit convoluted, but it's (for the most part) dictated by the workflow we're using. This also works well for the most part however the refinement step seems to mess around with the boundary attributes or the boundary face orientation.

If I add boundary faces before calling EnsureNCMesh it seems like these get reoriented which messes up the internal boundary integrators.
If I add boundary faces after calling EnsureNCMesh the orientation seems fine but then GeneralRefinement() + SetAttributes() resets the boundary attributes.

I also realize that there have been some discussions on correct handling of attributes in NCmeshes (and some PRs that address some ambiguity of stored attributes) but I wasn't quite sure if this applies here.
I guess the question is: what is the correct order of "mesh generation" + "EnsureNCMesh" + "ParMesh" + "Adding boundary faces" that preserves all the information correctly?
As always, thanks a lot! (and happy Thanksgiving!)
Mathias

ping: @rw-anderson


Check live datasets checksums too

Raise ValueError also for live datasets when hashes of downloaded files do not match

Addresses #494


Support TLS for spark dependencies

Spark dependencies job uses Java keystore for certificates. The docker image allows to configure java opts with ssl configuration - JAVA_OPTS=-Djavax.net.ssl..
The certs can be mounted via volume and volume mounts which are part of JaegerCommonSpec.  The issue that these certs have to be imported into java keystore/thruststore.
https://developers.redhat.com/blog/2017/11/22/dynamically-creating-java-keystores-openshift/ suggests using init container for the job.
Todos:

[ ]  allow configuring JAVA_OPTS to point to keystore/thruststore
[ ]  import certs into keystore/thruststore

Dependent issue in spark-dependencies https://github.com/jaegertracing/jaeger-operator/issues/294

@jpkrohling this is not ES specific. The dependency job supports multiple backends.

Hi @pavolloffay are there any updates on this?

There aren't ane news. If anybody has free cycles feel free to take it.

Spent a bit doing some digging through things and came across this page https://hub.helm.sh/charts/jaegertracing/jaeger
The page mentions that in order to have a version of the spark job working one would need to get the certificates from the elasticsearch cluster and do
$ keytool -import -trustcacerts -keystore trust.store -storepass <some pass> -alias es-root -file <es.pem>

$ kubectl create configmap jaeger-tls --from-file=trust.store --from-file=<es.pem>

And then use the jaeger-tls configmap within the spark job definition as such:
spec:
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: jaeger-spark
            args:
              - --java.opts=-Djavax.net.ssl.trustStore=/tls/trust.store -Djavax.net.ssl.trustStorePassword=<some pass>
            volumeMounts:
              - name: jaeger-tls
                mountPath: /tls
                subPath:
                readOnly: true
          volumes:
            - name: jaeger-tls
              configMap:
                name: jaeger-tls
          ...

Which all sounds like it is in accordance with the issue description.
Is all that right?

Correct the certs have to be imported into java keystore in the spark image.

My idea was to implement a script in the spark image that would do that if the certs are specified as env vars.

Hey guys, is there any work around for this atm? I'm trying to follow the thread of TLS related changes and options. Is there any skip-tls JAVA_OPT or something that can be used temporarily?

Hey guys, is there any work around for this atm? I'm trying to follow the thread of TLS related changes and options. Is there any skip-tls JAVA_OPT or something that can be used temporarily?

This would probably a question to ask in the repository that holds the code for the spark dependencies processor. https://github.com/jaegertracing/spark-dependencies

This would probably a question to ask in the repository that holds the code for the spark dependencies processor. https://github.com/jaegertracing/spark-dependencies

Hi, maybe this is stupid question, but i couldn't run a spark job with following configuration:
spark:
  enabled: true
  cmdlineParams:
    java.opts: "-Djavax.net.ssl.trustStore=/tls/trust.store -Djavax.net.ssl.trustStorePassword=changeit"
  extraConfigmapMounts:
    - name: jaeger-tls
      mountPath: /tls
      subPath: ""
      configMap: jaeger-tls
      readOnly: true

Here is error:
/entrypoint.sh: 37: exec: --java.opts=-Djavax.net.ssl.trustStore=/tls/trust.store -Djavax.net.ssl.trustStorePassword=changeit: not found

Can someone show me, what i'm doing wrong? or maybe there are any workaround, how to get spark job works.
Thnx

@aleksandrovpa I don't have much time to look at this at the moment, but you may find what you need in these two items:
https://github.com/jaegertracing/jaeger-operator/issues/1332
https://github.com/jaegertracing/jaeger-operator/pull/1359

@aleksandrovpa we're just composing the cron job manually in order to bootstrap the trust store through an init container. You can check this out https://github.com/MS3Inc/tavros/blob/main/tests/integration/targets/playbooks/provision_playbook/example.com/platform/jaeger/default/cronjob-spark-dependencies.yaml#L39

is it currently the only solution to write our own job ? i see that the dependencies definition now allow to mount volumes but i am not sure how to create the java trustStore without an init container like the one in @jam01 link

Hi, maybe this is stupid question, but i couldn't run a spark job with following configuration:
spark:
  enabled: true
  cmdlineParams:
    java.opts: "-Djavax.net.ssl.trustStore=/tls/trust.store -Djavax.net.ssl.trustStorePassword=changeit"
  extraConfigmapMounts:
    - name: jaeger-tls
      mountPath: /tls
      subPath: ""
      configMap: jaeger-tls
      readOnly: true

Here is error:
/entrypoint.sh: 37: exec: --java.opts=-Djavax.net.ssl.trustStore=/tls/trust.store -Djavax.net.ssl.trustStorePassword=changeit: not found

Can someone show me, what i'm doing wrong? or maybe there are any workaround, how to get spark job works. Thnx

It is much simple: entrypoint.sh from docker image is a shell script. So it used JAVA_OPTS env to pass options to java process. You need to pass env variable JAVA_OPTS by extraEnv param from values.yaml like this:
   spark:   extraEnv:     - name: "JAVA_OPTS"        value: "-Djavax.net.ssl.trustStore=/tls/trust.store -Djavax.net.ssl.trustStorePassword=changeit"

The spark-dependencies container has a __cacert_entrypoint.sh script in the / root path which does the required work of taking any certificates found in the /certificates path merges them with the system ca chain and then creates a JKS compatible one in the JRE cacerts location.
The entrypoint script just needs wrapping or embedding that script in it and the problem will be solved moving forwards.
The image tag I'm referring to is:
$ docker images --digests ghcr.io/jaegertracing/spark-dependencies/spark-dependencies
REPOSITORY                                                    TAG       DIGEST                                                                    IMAGE ID       CREATED      SIZE
ghcr.io/jaegertracing/spark-dependencies/spark-dependencies   latest    sha256:a307e1baf5815682f7581f2d0f8734510e511c0ae4c326c8501bee2647f6c3dd   4a8f3ad09d74   5 days ago   442MB


Missing comparisons, branches and wrong tree when repeating analyses!

When repeating analyses of files in the same directory, a branch goes missing and the tree is incorrectly drawn. One row in the distance matrix is also missing, to the issue seems to be with the files being compared rather than the drawing of the tree.

This is another example of what it could look like - here with all 4 branches present, but the tree all wrong:

I cloned the latest version from GitHub today, and still the get the same behavior.

As it turned out, this was an issue with reading scan number information from certain dialects of MGF. This has been addressed now, with a default that compares all scans if no scan numbers can be read from the MGF files.


Update Hugo Links to Editors Syntaxes

Add more links to editors syntaxes for Hugo, as found at:

https://github.com/tajmone/hugo/wiki#editor-syntaxes

These link were already updated at some point, I just forgot to reference the Issue in the commit message.


Trouble with Material Design Icons integration

There's no way I'm doing something wrong here, right? Like, it's just as easy as typing in the name of the icon and that's it, right?
Here are some of the icons I tried that just gave me the "circle with a line through it" 'no' icon instead:

account-clock
align-horizontal-left
alpha-w-circle
arrow-collapse-down
cloud-check-outline
email-multiple

I know the icon integration itself works - it suggests book-open-outline and that's fine. Airplane works. I'm not sure if there's a pattern to what works and doesn't work- or why some of them don't work in the first place.
I'm running this through Container Station on QNAP's QTS if that means anything

Hi. I just checked and all these icons work for me. Are you using the latest app version? And are you sure that icon name doesn't contain any spaces? So instead account-clock it's  account-clock or account-clock .

Hi. I just checked and all these icons work for me. Are you using the latest app version? And are you sure that icon name doesn't contain any spaces? So instead account-clock it's  account-clock or account-clock .


Port tokenHandler mechanism from go-nats (option processInbound())

Fixes: #265
Description
I wasn't sure of the best place to plug this in. This PR introduces the code directly in processInbound(), which is very close to how it's done in go-nats.

Coverage increased (+0.2%) to 89.405% when pulling 3f1452dabb58f95bff886d8b837da2d209bce979 on andriy-bulynko:token-handler-processInbound into e5f4558904f0a88183d2c542f33351b546d1d3b6 on nats-io:master.

@andriy-bulynko I have the same comments as the other PR. We don't use Promises on node-nats.

@aricart: Ok, thanks. Just to be clear, is a callback approach like tokenHandler(function(token, error){}) also off limits?

@andriy-bulynko tokenHandler(function(error, token) would be fine. Just no Promise related code.

@aricart: Promises are out. Asynchronous callbacks are in.

Is this a problem with code or with build? Is there a way to kick the build to try again?

That's so weird.

There might be CI issues going on now...

replaced with the sync version


Characters disappear if url/label contains valid attributes.

[text]({a=b})

yields
doc
  para
    link destination="{ab}"
      str text="text"

instead of
doc
  para
    link destination="{a=b}"
      str text="text"

Can see same thing with
[hi]({#ab})

[hi]({.ab})

When we match a destination, we (inlines.ts l. 440)
        // convert all matches inside destination to str

The problem is that the attribute parser gives us events for key and value, but not for the =; similarly for the identifier or class, but not the markers # or ..  Interior whitespace is also not given an event, so it too will be lost:
% ./djot
[hi]({a=b c=d})

hi

Probably the best fix is to use a different strategy for matching destinations. You might ask, why not just use the position after the ( as the start of the destination and the position before the ) as the end?  Because the destination might be broken up by newlines and thus might not occupy a contiguous segment of the input text.

One idea for fixing this would be to have the events parser create multiple +destination/-destination pairs when the URL is split over lines.
parse.ts could then look for multiple destinations and concatenate them. If we were sure that each +destination/-destination pair was on a single line, we could just take all the text between the start and the end.

Reopening; I had to revert the fix because it was buggy.
See #64.

I think the problem is with strMatches in inlines.ts.

Similar issue: [link]({.class})

The function converts all matches to str, but some of the characters, like ., don't generate match events, so they are lost.


Remove hard-coded VSPK API version

METROAE-494

why is the nuage_vspk file deleted

the nuage_vspk file hard-coded the api version, it was easier to use the nuage_vspk Ansible module instead of making changes to our file. The Ansible module provides the same functionality - the only difference is that ir requires the api version as part of the authentication, which I made sure to add in all cases that we use it


Chapter 13 Odometer Request Dangerous Permissions for Marshmallow Causes Crash

When requesting permission for ACCESS_FINE_LOCATION, Android has declared this as "Protection Level: Dangerous", which means that starting with Android Marshmallow, the app must explicitly ask the user for permission before it can be used.
Because there wasn't a request for permission, this app crashes immediately on devices running Android Marshmallow. I have a solution that I believe is working correctly here: https://github.com/TBVanderley4545/Odometer.git. I added checkSelfPermissions in onCreate and onStart in MainActivity.java and implemented onRequestPermissionResult. I also added checkSelfPermission in OdometerService.java in onCreate() and onDestroy().
Thank you for writing your book. It's been a great help.

Sorry, the link didn't work the first time. https://github.com/TBVanderley4545/Odometer.git

Hi! Good catch :-)
I believe this should only be a problem if the targetSdkVersion in the app/build.gradle is set to 23. Prior to that level, I believe there won’t be a problem.
The code in the book currently sets the target SDK back a little, so the issue should not arise with the code from there. For that reason, I won’t update the code on this repo.
However... Android Studio may generate projects with a targetSdkVersion set to 23, so we will add a couple of additional pages to the book to explain the consequences of moving to API 23, with the new permissions model.
Thanks so much for reporting this :-)
DG


factory openSUSE Urllib3Using.py on Python 2

When using branch factory Nuitka-<IP_ADDRESS>6202417, on openSUSE Python 2.7, .spec
[  774s] Consider output of recursively compiled program: Urllib3Using.py
[  774s] Comparing output of 'Urllib3Using.py' using '/usr/bin/python2' with flags silent, expect_success, standalone, remove_output ...
[  839s] --- /usr/bin/python2 (stderr)
[  839s] +++ nuitka (stderr)
[  839s]
[  839s] @@ -1,5 +1,4 @@
[  839s]
[  839s]  Traceback (most recent call last):
[  839s] -  File "/home/abuild/rpmbuild/BUILD/Nuitka-<IP_ADDRESS>6202417.e23ccdbb/tests/standalone/Urllib3Using.py", line 29, in <module>
[  839s] -    from http.server import BaseHTTPRequestHandler, HTTPServer
[  839s] +  File "/home/abuild/rpmbuild/BUILD/Nuitka-<IP_ADDRESS>6202417.e23ccdbb/tests/standalone/Urllib3Using.dist/Urllib3Using.py", line 29, in <module>
[  839s]  ImportError: No module named http.server
[  839s]
[  839s] Error, outputs differed.
[  839s] Error exit! 1
[  839s] error: Bad exit status from /var/tmp/rpm-tmp.ysUnJG (%check)

This issue comes from Urllib3Using.py trying to use the http.server import, which seems to be only in the standard library of python 3 and not python 2.

@kayhayen  would you like me to work on this issue?

Confirming this only occurs on Python 2.  It passes on Python 3

@tommyli3318 Yes, please do so. I think the issue there is that http.server is a Python3 rename, and the Python2 module should be used as a fallback. I have on factory or develop, I forget, added a requirement on http.server as a stop gap measure the other day. But of course we would love to see this run on Python2 too.

@jayvdb What which version of Python2.7 are you using? I just ran this using Windows with python2.7.16 and I didn't get this error. Importing http.server seems to raise no issue.

@tommyli3318 you probably have a backport installed? This is clearly one of the one Python3 namespace cleanups

All issues with openSUSE in the title are using latest openSUSE rpms.  In this case, for Python 2, the python and python-base rpms build according to recipes at https://build.opensuse.org/package/show/devel:languages:python:Factory/python .  On the RHS are logs, which show the steps of the build.
As @kayhayen mentioned, you must have a backport, as http isnt valid in vanilla Python 2.
https://docs.python.org/3/library/http.html
vs
https://docs.python.org/2.7/search.html?q=http&check_keywords=yes&area=default
p.s. I'd love to know of a good backport of http for Python 2, as I could add that backport to my collection at https://build.opensuse.org/project/show/home:jayvdb:py-backports , which usually make it into the openSUSE main collection.  At the end of the year, no more Python 2 stuff will be allowed into the main collection, and they will probably evict them then, but they should still be available in the released version of the OS, giving Python 2 a bit more life for those who need it.

I see, I wasn't even aware that I have backports installed. It might've just came with my Python3 installation. Thanks for clarifications

@jayvdb
I think https://docs.python.org/2.7/glossary.html#term-2to3 was why my Python2 code was working

2to3 is an installation tool.  It doesnt help unless you install something which provides http and explicitly asked 2to3 to help (usually getting python2 stuff working under python3, not the other way around).

I think https://python-future.org/standard_library_imports.html may be where your http.server is coming from.

Yup.  I installed python2-future, and the urllib3 test is now passing on Python 2.
It might be useful to move future to near the top of your list of PyPI packages to test, and then add it as a runtime dependency of Nuitka and derivatives on Python 2.  It may solve a lot of heartache, as your tests will often have to copy the same voodoo that future has already built.
Soon this will become a loosing battle, as the moment any of the libraries you are testing need future, to properly test the others you need to isolate the test rig for the others, otherwise futures magic (and voodoo by similar libraries) will polute the testing environment.  And libraries are also having the same problem -- increasingly future is (indirectly) being used by some part of their test environment, so they dont notice they even depend on its magic.

When you say

add it [future] as a runtime dependency of Nuitka
do you mean make it so that Nuitka doesn't run unless future is present?
If I'm wrong please elaborate. Sorry I'm new to big open-source projects like Nuitka so I'm not familiar with technical terms.

do you mean make it so that Nuitka doesn't run unless future is present?

Yes, for Python 2, and only for Python 2.  For the things it silently does to make Python 2 much more similar to Python 3.  But that is a big decision.  Beyond the scope of this problem.

Nah, "Urllib3Using.py" is first and foremost a program tasked to test that "urllib3" is working correctly, with all the Pythons. Therefore It is supposed to be written portable, and not have too unusual dependencies.
All we there need is a small HTTP server, and I believe that https://docs.python.org/2/library/simplehttpserver.html fits the bill here simple enough, so that is what we are going to use.
Then, what the test shows, is that the working program continue to work when compiled with Nuitka in standalone mode. No more.
For nice and more complete test coverage, we are going to run the test suite of urllib3 against the compiled package, but that is not what this test is supposed to do. There we are going to have their test dependencies installed. For now, standalone tests are not really allowed to require lots of things that are not normally there.

This is fixed in the current factory and soon 0.6.4, I am closing it therefore.


[Bugfix][TIR] Put ThreadSync Pass after MergeSharedMemory Pass

As discussed in #17439, The phase of ThreadSync injection should be applied when the memory allocations are all deterministic.

CC @Lunderberg @vinx13


[QUESTION] How to generate a native debug build?

I can't find any information on how to generate a debug build?
Since Skia doesn't have a debug layer like DirectX, Vulkan or OpenGL, sometimes it is insanely hard to track down why a function returns false or null.  So having a debug build can be very handy.
Is it just a matter of passing -c Debug to ./bootstrapper.ps1? That didn't seem to work for me, but maybe something else went wrong.

You need to set is_official_build=false https://github.com/mono/SkiaSharp/blob/master/native/windows/build.cake#L26

Thanks for that.
Would it make sense to have these set to false automatically when passing Debug to the -c|--configuration arg? The latter seems to specify the MSBuild configuration, somehow it feels logical that the .NET debug build would use native debug bits too?

Just in case other people need this, I also had to add the clang cflag -Wno-everything in externals\skia\gn\BUILD.gn, otherwise the compiler choked on various warnings that were promoted to errors.
And instead of wasting hours to get this done, good advise might also be to read the Skia comments, in my case the reason for ReadPixels returning false was trying to convert from kPremul to kOpaque  , something that no-one asked for it seems, see below. Oh boy, now I realize how spoiled I was with the DirectX debug layer 😉
/**
 *  Returns true if Skia has defined a pixel conversion from the |src| to the |dst|.
 *  Returns false otherwise.  Some discussion of false cases:
 *      We will not convert to kIndex8 unless it exactly matches the src, since color tables
 *      are immutable.
 *      We do not convert to kGray8 when the |src| is not kGray8 in the same color space.
 *      We may add this feature - it just requires some work to convert to luminance while
 *      handling color spaces correctly.  Currently no one is asking for this.
 *      We will not convert from kAlpha8 when the |dst| is not kAlpha8.  This would require
 *      inventing color information.
 *      We will not convert to kOpaque when the |src| is not kOpaque.  This could be
 *      implemented to set all the alpha values to 1, but there is still some ambiguity -
 *      should we use kPremul or kUnpremul color values with the opaque alphas?  Or should
 *      we just use whatever the |src| alpha is?  In the future, we could choose to clearly
 *      define this, but currently no one is asking for this feature.
 *      We will not convert to a particular color space if |src| is nullptr.  The color space
 *      conversion is not well-defined.
 */
static inline bool SkImageInfoValidConversion(const SkImageInfo& dst, const SkImageInfo& src) {

@Ziriax

Would it make sense to have these set to false automatically when passing Debug to the -c|--configuration arg? The latter seems to specify the MSBuild configuration, somehow it feels logical that the .NET debug build would use native debug bits too?

Thanks for the feedback, I'll update the cake tasks.

@mattleibow  Super! Please note that I also had to tweak the clang compiler settings, see my other comment (this might be because I'm using a newer version of LLVM than the M68 branch on which master is based)

Closing as this was added in #1418


CASMTRIAGE-7061: Add new known issue for keycloak

Description

Adds in new known error that happens during a upgrade from csm 1.4 to csm 1.5.
Checklist

[x] If I added any command snippets, the steps they belong to follow the prompt conventions (see example).
[x] If I added a new directory, I also updated .github/CODEOWNERS with the corresponding team in Cray-HPE.
[x] My commits or Pull-Request Title contain my JIRA information, or I do not have a JIRA.

/backport release/1.5


chore: Bump all packages to newest versions

Checklist:

[x] issue number linked above after pound (#)

replace "Closes " with "Contributes to" or other if this PR does not close the issue

[x] manually checked my feature / checking not applicable
[x] wrote tests / testing not applicable
[x] attached screenshots / screenshot not applicable

bump nuxt-auth version

Tests are failing


openssl-1.1.0 on windows - SSL_get_error() internally reset the value being returned by WSAGetLastError()

openssl-1.1.0 on windows - SSL_get_error() internally reset the value being returned by WSAGetLastError()
I revealed this fact during my experiment when I built Python2.7 with OpenSSL-1.1.0 and tested.
Top problem is failing test - test_ftplib.py
After debug I found out that ssl related logic looks like:

call SSL_read
call SSL_get_error
when result of SSL_get_error is SSL_ERROR_SYSCALL ask for WSAGetLastError
WSAGetLastError returns 0

Debugger shows that WSAGetLastError() is reset exactly by the winapi call of TlsGetValue()
SSL_get_error()
  -> ERR_peek_error()
    -> get_error_values()
      -> CRYPTO_THREAD_get_local()
        -> TlsGetValue()  /* WSAGetLastError is reset here */

As result after call of SSL_get_error() it is impossible to check low level socket error.
With openssl-1.0.0 the situation was different.
Please confirm whether this is a bug in OpenSSL or not ?

It's an OpenSSL bug. As you note, libssl's calling convention requires that nothing in SSL_get_error modify errno or GetLastError. The problem is, unlike most functions, TlsGetValue clears the last error on success, so that you can distinguish successful or failing NULL returns. This interferes with SSL_get_error.
https://msdn.microsoft.com/en-us/library/windows/desktop/ms686812(v=vs.85).aspx
The fix is to sample GetLastError before TlsGetValue and then restore it afterwards. (TlsGetValue promises never to fail if the index was allocated via TlsAlloc, so there's no particular need to route error-handling in here.)
https://github.com/google/boringssl/blob/master/crypto/thread_win.c#L193-L209
https://github.com/google/boringssl/blob/master/crypto/err/err_test.cc#L224-L237

Same problem with 1.1.1g, SSL_get_error reset error for WSAGetLastError
When read socket have timeout.


DEVEX-739 iOS 12 Preperation

Use Cocoapods for OCMock and OHHTTP mocks: Frameworks aren't ABI compatible.
Updated some unit tests for iOS 12.

👍


Assets break after being stopped

This is a great template for learning how Phoenix and Docker plays together but I have trouble getting a consistent build running.

If I completely prune everything in Docker docker system prune -a --volumes, do a fresh git clone and build with docker-compose up --build then everything looks fine.
I then stop it with Ctrl-C.
I run docker-compose up and when I visit the page, I see the assets are not being generated, even though I can see Webpack runs in the logs. Same problem with running docker-compose up --build.
I have tried docker compose up (no -) too but same problem.
Same problem running docker-compose down and then docker-compose up.

What I see is [debug] ** (Phoenix.Router.NoRouteError) no route found for GET /css/app.css (HelloWeb.Router) and for the other resources too.
I am on macOS 11.2.3 and Docker 3.3.3 (64133). Any idea what is causing this?

❯ docker-compose up
Docker Compose is now in the Docker CLI, try `docker compose up`

Starting hellophoenix_webpack_1  ... done
Starting hellophoenix_postgres_1 ... done
Starting hellophoenix_web_1      ... done
Attaching to hellophoenix_webpack_1, hellophoenix_postgres_1, hellophoenix_web_1
postgres_1  |
postgres_1  | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres_1  |
yarn run v1.22.5
warning package.json: No license field
$ /node_modules/webpack/bin/webpack.js --mode=development --progress --watch
37% building 1/2 entries 9/9 dependencies 5/6 modules
webpack_1   | warn - You have enabled the JIT engine which is currently in preview.
webpack_1   | warn - Preview features are not covered by semver, may introduce breaking changes, and can change at any time.
postgres_1  | 2021-06-07 10:20:15.730 UTC [1] LOG:  starting PostgreSQL 13.3 (Debian 13.3-1.pgdg100+1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
postgres_1  | 2021-06-07 10:20:15.730 UTC [1] LOG:  listening on IPv4 address "<IP_ADDRESS>", port 5432
postgres_1  | 2021-06-07 10:20:15.730 UTC [1] LOG:  listening on IPv6 address "::", port 5432
postgres_1  | 2021-06-07 10:20:15.732 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1  | 2021-06-07 10:20:15.736 UTC [28] LOG:  database system was shut down at 2021-06-07 10:17:14 UTC
postgres_1  | 2021-06-07 10:20:15.739 UTC [1] LOG:  database system is ready to accept connections
assets by status 49.2 KiB [compared for emit]
webpack_1   |   assets by path *.png 29.8 KiB 4 assets
webpack_1   |   assets by path *.html 1.87 KiB
webpack_1   |     asset 502.html 1020 bytes [compared for emit] [from: static/502.html] [copied]
webpack_1   |     asset maintenance.html 892 bytes [compared for emit] [from: static/maintenance.html] [copied]
webpack_1   |   asset favicon.ico 14.7 KiB [compared for emit] [from: static/favicon.ico] [copied]
webpack_1   |   asset safari-pinned-tab.svg 1.9 KiB [compared for emit] [from: static/safari-pinned-tab.svg] [copied]
webpack_1   |   asset manifest.json 489 bytes [compared for emit] [from: static/manifest.json] [copied]
webpack_1   |   asset browserconfig.xml 246 bytes [compared for emit] [from: static/browserconfig.xml] [copied]
webpack_1   |   asset robots.txt 185 bytes [compared for emit] [from: static/robots.txt] [copied]
webpack_1   | assets by status 224 KiB [emitted]
webpack_1   |   asset js/app.js 173 KiB [emitted] (name: main)
webpack_1   |   asset images/phoenix.png 37.8 KiB [emitted] [from: static/images/phoenix.png] [copied]
webpack_1   |   asset css/app.css 13.4 KiB [emitted] (name: main)
webpack_1   | Entrypoint main 186 KiB = css/app.css 13.4 KiB js/app.js 173 KiB
webpack_1   | runtime modules 937 bytes 4 modules
webpack_1   | cacheable modules 159 KiB
webpack_1   |   modules by path ../../node_modules/ 158 KiB
webpack_1   |     ../../node_modules/phoenix_html/priv/static/phoenix_html.js 2.21 KiB [built] [code generated]
webpack_1   |     ../../node_modules/phoenix/priv/static/phoenix.js 38.9 KiB [built] [code generated]
webpack_1   |     ../../node_modules/topbar/topbar.min.js 4 KiB [built] [code generated]
webpack_1   |     ../../node_modules/phoenix_live_view/priv/static/phoenix_live_view.js 113 KiB [built] [code generated]
webpack_1   |   modules by path ./app/ 1020 bytes
webpack_1   |     ./app/app.css 50 bytes [built] [code generated]
webpack_1   |     ./app/app.js 966 bytes [built] [code generated]
webpack_1   | css ../../node_modules/css-loader/dist/cjs.js!../../node_modules/postcss-loader/dist/cjs.js!./app/app.css 13.4 KiB [code generated]
webpack_1   | webpack 5.36.1 compiled successfully in 1613 ms
assets by status 273 KiB [cached] 14 assets
webpack_1   | Entrypoint main 186 KiB = css/app.css 13.4 KiB js/app.js 173 KiB
webpack_1   | cached modules 159 KiB (javascript) 13.4 KiB (css/mini-extract) 937 bytes (runtime) [cached] 10 modules
webpack_1   | ./app/app.css 50 bytes [built]
webpack_1   | webpack 5.36.1 compiled successfully in 137 ms
web_1       | Erlang/OTP 24 [erts-12.0.2] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1]
web_1       |
web_1       | [info] Running HelloWeb.Endpoint with cowboy 2.9.0 at :::8000 (http)
web_1       | [info] Access HelloWeb.Endpoint at http://localhost:8000
web_1       | Interactive Elixir (1.12.1) - press Ctrl+C to exit (type h() ENTER for help)
web_1       | iex(1)> [info] GET /
web_1       | [debug] Processing with HelloWeb.PageController.home/2
web_1       |   Parameters: %{}
web_1       |   Pipelines: [:browser]
web_1       | [info] Sent 200 in 19ms
web_1       | [info] GET /css/app.css
web_1       | [info] GET /js/app.js
web_1       | [debug] ** (Phoenix.Router.NoRouteError) no route found for GET /css/app.css (HelloWeb.Router)
web_1       |     (hello 0.1.0) lib/phoenix/router.ex:402: HelloWeb.Router.call/2
web_1       |     (hello 0.1.0) lib/hello_web/endpoint.ex:1: HelloWeb.Endpoint.plug_builder_call/2
web_1       |     (hello 0.1.0) lib/plug/debugger.ex:136: HelloWeb.Endpoint."call (overridable 3)"/2
web_1       |     (hello 0.1.0) lib/hello_web/endpoint.ex:1: HelloWeb.Endpoint.call/2
web_1       |     (phoenix 1.5.9) lib/phoenix/endpoint/cowboy2_handler.ex:65: Phoenix.Endpoint.Cowboy2Handler.init/4
web_1       |     (cowboy 2.9.0) /mix/deps/cowboy/src/cowboy_handler.erl:37: :cowboy_handler.execute/2
web_1       |     (cowboy 2.9.0) /mix/deps/cowboy/src/cowboy_stream_h.erl:306: :cowboy_stream_h.execute/3
web_1       |     (cowboy 2.9.0) /mix/deps/cowboy/src/cowboy_stream_h.erl:295: :cowboy_stream_h.request_process/3
web_1       |     (stdlib 3.15.1) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
web_1       |
web_1       | [debug] ** (Phoenix.Router.NoRouteError) no route found for GET /js/app.js (HelloWeb.Router)
web_1       |     (hello 0.1.0) lib/phoenix/router.ex:402: HelloWeb.Router.call/2
web_1       |     (hello 0.1.0) lib/hello_web/endpoint.ex:1: HelloWeb.Endpoint.plug_builder_call/2
web_1       |     (hello 0.1.0) lib/plug/debugger.ex:136: HelloWeb.Endpoint."call (overridable 3)"/2
web_1       |     (hello 0.1.0) lib/hello_web/endpoint.ex:1: HelloWeb.Endpoint.call/2
web_1       |     (phoenix 1.5.9) lib/phoenix/endpoint/cowboy2_handler.ex:65: Phoenix.Endpoint.Cowboy2Handler.init/4
web_1       |     (cowboy 2.9.0) /mix/deps/cowboy/src/cowboy_handler.erl:37: :cowboy_handler.execute/2
web_1       |     (cowboy 2.9.0) /mix/deps/cowboy/src/cowboy_stream_h.erl:306: :cowboy_stream_h.execute/3
web_1       |     (cowboy 2.9.0) /mix/deps/cowboy/src/cowboy_stream_h.erl:295: :cowboy_stream_h.request_process/3
web_1       |     (stdlib 3.15.1) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
web_1       |
web_1       | [info] GET /images/phoenix.png
web_1       | [debug] ** (Phoenix.Router.NoRouteError) no route found for GET /images/phoenix.png (HelloWeb.Router)
web_1       |     (hello 0.1.0) lib/phoenix/router.ex:402: HelloWeb.Router.call/2
web_1       |     (hello 0.1.0) lib/hello_web/endpoint.ex:1: HelloWeb.Endpoint.plug_builder_call/2
web_1       |     (hello 0.1.0) lib/plug/debugger.ex:136: HelloWeb.Endpoint."call (overridable 3)"/2
web_1       |     (hello 0.1.0) lib/hello_web/endpoint.ex:1: HelloWeb.Endpoint.call/2
web_1       |     (phoenix 1.5.9) lib/phoenix/endpoint/cowboy2_handler.ex:65: Phoenix.Endpoint.Cowboy2Handler.init/4
web_1       |     (cowboy 2.9.0) /mix/deps/cowboy/src/cowboy_handler.erl:37: :cowboy_handler.execute/2
web_1       |     (cowboy 2.9.0) /mix/deps/cowboy/src/cowboy_stream_h.erl:306: :cowboy_stream_h.execute/3
web_1       |     (cowboy 2.9.0) /mix/deps/cowboy/src/cowboy_stream_h.erl:295: :cowboy_stream_h.request_process/3
web_1       |     (stdlib 3.15.1) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
web_1       |
^CGracefully stopping... (press Ctrl+C again to force)
Stopping hellophoenix_web_1      ... done
Stopping hellophoenix_webpack_1  ... done
Stopping hellophoenix_postgres_1 ... done

Hi,
Webpack will take a few seconds to run btw, so it's normal not to see your assets load immediately but it should self resolve when Webpack finishes building them.
After you up the project and Webpack runs do you have static files that exist in priv/static/?

Tried waiting ten minutes but nothing happens still. First time it's pretty quick to get up and running.
There are files in priv/static but no css, images or js directories.

Did you modify anything in the code base, even something as minor as the .env file?
Your log output shows that Webpack is running successfully and generated everything within about 1.6 seconds.
webpack_1   | webpack 5.36.1 compiled successfully in 1613 ms
webpack_1   | Entrypoint main 186 KiB = css/app.css 13.4 KiB js/app.js 173 KiB
webpack_1   | cached modules 159 KiB (javascript) 13.4 KiB (css/mini-extract) 937 bytes (runtime) [cached] 10 modules
webpack_1   | ./app/app.css 50 bytes [built]
webpack_1   | webpack 5.36.1 compiled successfully in 137 ms

Nothing has been modified, I only copy the two files mentioned in the README. I am also perplexed seeing the output.
Here is how I reproduce it:

docker system prune -a --volumes
git clone git@github.com:nickjj/docker-phoenix-example.git
cd docker-phoenix-example
cp docker-compose.override.yml.example docker-compose.override.yml
cp .env.example .env
docker-compose up --build (takes some time now)
At this point it works fine, I see the Phoenix bird
I stop it with Ctrl-C
docker-compose up
I wait until I see Webpack output
Now the page is broken, the Phoenix bird is gone

It is very weird, might be a bug with Docker itself, I don't have much experience debugging this.

After you up the first time and everything works, do you see the static files in priv/static/? If so what are the file permissions? Which user:group owns them?

 I run docker-compose up and now I get
❯ tree priv/static
priv/static
├── 502.html
├── android-chrome-192x192.png
├── android-chrome-512x512.png
├── apple-touch-icon.png
├── browserconfig.xml
├── favicon.ico
├── maintenance.html
├── manifest.json
├── mstile-150x150.png
├── robots.txt
└── safari-pinned-tab.svg

0 directories, 11 files

I also see them disappear in VS Code.

Can you show me up the full terminal output after running docker-compose up a second time when it doesn't produce the files?

Very weird, now I see the following output (directories are now there, wasn't before, but they are still empty)
❯ tree priv/static
priv/static
├── 502.html
├── android-chrome-192x192.png
├── android-chrome-512x512.png
├── apple-touch-icon.png
├── browserconfig.xml
├── css
├── favicon.ico
├── images
├── js
├── maintenance.html
├── manifest.json
├── mstile-150x150.png
├── robots.txt
└── safari-pinned-tab.svg

3 directories, 11 files

The full output is
❯ docker-compose up
Docker Compose is now in the Docker CLI, try `docker compose up`

Starting hellophoenix_postgres_1 ... done
Starting hellophoenix_webpack_1  ... done
Starting hellophoenix_web_1      ... done
Attaching to hellophoenix_postgres_1, hellophoenix_webpack_1, hellophoenix_web_1
postgres_1  |
postgres_1  | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres_1  |
postgres_1  | 2021-06-07 11:23:34.780 UTC [1] LOG:  starting PostgreSQL 13.3 (Debian 13.3-1.pgdg100+1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
yarn run v1.22.5
warning package.json: No license field
$ /node_modules/webpack/bin/webpack.js --mode=development --progress --watch
37% building 1/2 entries 9/9 dependencies 5/6 modules
webpack_1   | warn - You have enabled the JIT engine which is currently in preview.
webpack_1   | warn - Preview features are not covered by semver, may introduce breaking changes, and can change at any time.
postgres_1  | 2021-06-07 11:23:34.782 UTC [1] LOG:  listening on IPv4 address "<IP_ADDRESS>", port 5432
postgres_1  | 2021-06-07 11:23:34.782 UTC [1] LOG:  listening on IPv6 address "::", port 5432
postgres_1  | 2021-06-07 11:23:34.785 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
assets by status 49.2 KiB [compared for emit]
webpack_1   |   assets by path *.png 29.8 KiB 4 assets
webpack_1   |   assets by path *.html 1.87 KiB
webpack_1   |     asset 502.html 1020 bytes [compared for emit] [from: static/502.html] [copied]
webpack_1   |     asset maintenance.html 892 bytes [compared for emit] [from: static/maintenance.html] [copied]
webpack_1   |   asset favicon.ico 14.7 KiB [compared for emit] [from: static/favicon.ico] [copied]
webpack_1   |   asset safari-pinned-tab.svg 1.9 KiB [compared for emit] [from: static/safari-pinned-tab.svg] [copied]
webpack_1   |   asset manifest.json 489 bytes [compared for emit] [from: static/manifest.json] [copied]
postgres_1  | 2021-06-07 11:23:34.788 UTC [31] LOG:  database system was shut down at 2021-06-07 11:23:30 UTC
postgres_1  | 2021-06-07 11:23:34.791 UTC [1] LOG:  database system is ready to accept connections
webpack_1   |   asset browserconfig.xml 246 bytes [compared for emit] [from: static/browserconfig.xml] [copied]
webpack_1   |   asset robots.txt 185 bytes [compared for emit] [from: static/robots.txt] [copied]
webpack_1   | assets by status 224 KiB [emitted]
webpack_1   |   asset js/app.js 173 KiB [emitted] (name: main)
webpack_1   |   asset images/phoenix.png 37.8 KiB [emitted] [from: static/images/phoenix.png] [copied]
webpack_1   |   asset css/app.css 13.4 KiB [emitted] (name: main)
webpack_1   | Entrypoint main 186 KiB = css/app.css 13.4 KiB js/app.js 173 KiB
webpack_1   | runtime modules 937 bytes 4 modules
webpack_1   | cacheable modules 159 KiB
webpack_1   |   modules by path ../../node_modules/ 158 KiB
webpack_1   |     ../../node_modules/phoenix_html/priv/static/phoenix_html.js 2.21 KiB [built] [code generated]
webpack_1   |     ../../node_modules/phoenix/priv/static/phoenix.js 38.9 KiB [built] [code generated]
webpack_1   |     ../../node_modules/topbar/topbar.min.js 4 KiB [built] [code generated]
webpack_1   |     ../../node_modules/phoenix_live_view/priv/static/phoenix_live_view.js 113 KiB [built] [code generated]
webpack_1   |   modules by path ./app/ 1020 bytes
webpack_1   |     ./app/app.css 50 bytes [built] [code generated]
webpack_1   |     ./app/app.js 966 bytes [built] [code generated]
webpack_1   | css ../../node_modules/css-loader/dist/cjs.js!../../node_modules/postcss-loader/dist/cjs.js!./app/app.css 13.4 KiB [code generated]
webpack_1   | webpack 5.36.1 compiled successfully in 1639 ms
assets by status 273 KiB [cached] 14 assets
webpack_1   | Entrypoint main 186 KiB = css/app.css 13.4 KiB js/app.js 173 KiB
webpack_1   | cached modules 159 KiB (javascript) 13.4 KiB (css/mini-extract) 937 bytes (runtime) [cached] 10 modules
webpack_1   | ./app/app.css 50 bytes [built]
webpack_1   | webpack 5.36.1 compiled successfully in 140 ms
web_1       | Erlang/OTP 24 [erts-12.0.2] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1]
web_1       |
web_1       | [info] Running HelloWeb.Endpoint with cowboy 2.9.0 at :::8000 (http)
web_1       | [info] Access HelloWeb.Endpoint at http://localhost:8000
web_1       | Interactive Elixir (1.12.1) - press Ctrl+C to exit (type h() ENTER for help)

Had to run it twice and now the directories are gone again this time.

When you run docker-compose --version, what version do you get back?

❯ docker-compose --version
docker-compose version 1.29.1, build c34c88b2

By the way running docker-compose down -v will remove all containers and volumes for this project, you don't always need to run a prune. If you run that and then up it afterwards, does it work?

And if it does work, what happens if you run just docker-compose down && docker-compose up which still removes the containers but it won't wipe out any volumes (in this case only PostgreSQL).

Tried both docker-compose down -v && docker-compose up and docker-compose down && docker-compose up but the assets are still not being generated. Sometimes the directories are there but still empty.

What happens if you comment out these lines and then up the project?

https://github.com/nickjj/docker-phoenix-example/blob/main/docker-compose.yml#L15-L16
https://github.com/nickjj/docker-phoenix-example/blob/main/docker-compose.override.yml.example#L14-L15 (do this in the real file, not the example)

My docker-compose.yml now:
x-app: &default-app
  build:
    context: "."
    target: "${MIX_ENV:-prod}"
    args:
      - "MIX_ENV=${MIX_ENV:-prod}"
      - "NODE_ENV=${NODE_ENV:-production}"
  depends_on:
    - "postgres"
  env_file:
    - ".env"
  restart: "${DOCKER_RESTART_POLICY:-unless-stopped}"
  stop_grace_period: "3s"
  tty: true
  # volumes:
  #   - "${DOCKER_WEB_VOLUME:-./priv/static:/app/priv/static}"

services:
  postgres:
    deploy:
      resources:
        limits:
          cpus: "${DOCKER_POSTGRES_CPUS:-0}"
          memory: "${DOCKER_POSTGRES_MEMORY:-0}"
    env_file:
      - ".env"
    image: "postgres:13.3"
    restart: "${DOCKER_RESTART_POLICY:-unless-stopped}"
    stop_grace_period: "3s"
    volumes:
      - "postgres:/var/lib/postgresql/data"

  web:
    <<: *default-app
    deploy:
      resources:
        limits:
          cpus: "${DOCKER_WEB_CPUS:-0}"
          memory: "${DOCKER_WEB_MEMORY:-0}"
    healthcheck:
      test: "${DOCKER_WEB_HEALTHCHECK_TEST:-curl localhost:8000/up}"
      interval: "60s"
      timeout: "3s"
      start_period: "5s"
      retries: 3
    ports:
      - "${DOCKER_WEB_PORT_FORWARD:-<IP_ADDRESS>:8000}:8000"

volumes:
  postgres: {}

I run docker-compose down -v && docker-compose up just to be sure. Looks like assets are being generated now
❯ tree priv/static
priv/static
├── 502.html
├── android-chrome-192x192.png
├── android-chrome-512x512.png
├── apple-touch-icon.png
├── browserconfig.xml
├── css
│   └── app.css
├── favicon.ico
├── images
│   └── phoenix.png
├── js
│   └── app.js
├── maintenance.html
├── manifest.json
├── mstile-150x150.png
├── robots.txt
└── safari-pinned-tab.svg

3 directories, 14 files

but
web_1       | [info] GET /images/phoenix.png
web_1       | [debug] ** (Phoenix.Router.NoRouteError) no route found for GET /images/phoenix.png (HelloWeb.Router)
web_1       |     (hello 0.1.0) lib/phoenix/router.ex:402: HelloWeb.Router.call/2
web_1       |     (hello 0.1.0) lib/hello_web/endpoint.ex:1: HelloWeb.Endpoint.plug_builder_call/2
web_1       |     (hello 0.1.0) lib/plug/debugger.ex:136: HelloWeb.Endpoint."call (overridable 3)"/2
web_1       |     (hello 0.1.0) lib/hello_web/endpoint.ex:1: HelloWeb.Endpoint.call/2
web_1       |     (phoenix 1.5.9) lib/phoenix/endpoint/cowboy2_handler.ex:65: Phoenix.Endpoint.Cowboy2Handler.init/4
web_1       |     (cowboy 2.9.0) /mix/deps/cowboy/src/cowboy_handler.erl:37: :cowboy_handler.execute/2
web_1       |     (cowboy 2.9.0) /mix/deps/cowboy/src/cowboy_stream_h.erl:306: :cowboy_stream_h.execute/3
web_1       |     (cowboy 2.9.0) /mix/deps/cowboy/src/cowboy_stream_h.erl:295: :cowboy_stream_h.request_process/3
web_1       |     (stdlib 3.15.1) proc_lib.erl:226: :proc_lib.init_p_do_apply/3

so page is still looking weird.

Something weird is going on then for sure. With those lines commented out there should be no assets in that directory if it were emptied before. The volume lines are what gets the files from the container to your dev box.

What does the RUN_MODE report back on the home page btw?

RUN_MODE = mix it says on the page.
Just tried docker-compose down -v, remove the directory, pull again, copy the two files, modify docker-compose.yml by commenting out but it still builds to priv/static.
Yesterday I also tried to prune everything, uninstall Docker and install again, but didn't change much. You think it's a Docker bug?

Oh yeah, you'll also need to comment these 2 lines in your docker-compose.override.yml file: https://github.com/nickjj/docker-phoenix-example/blob/main/docker-compose.override.yml.example#L14-L15 (not the example file but the real one).
So in the end you'll end up with 4 commented lines across 2 files. Does the outcome change then?
I'm not sure but after the above test you can try downgrading to a slightly older Docker version. I'm not able to reproduce this here btw.

I did see you initially mentioned both files but edited your comment so thought the second was maybe a mistake. Now both are commented out and nothing in priv/static is being generated.
I'll try to see if I can run an older version somehow, I'm on an M1 Mac so might be difficult, since before I used a preview build.

What version of Docker, docker-compose and OS are you running? I'll try to download same version.

Yeah my gut was right but then I backed out thinking it wasn't necessary but now that I think about it, that volume in the override file is for sure going to volume mount the files. It bind mounts everything.

What version of Docker, docker-compose and OS are you running? I'll try to download same version.

Just about the same as you. Docker Desktop 3.3.3 (64133) with Docker Compose 1.29.1.
The part that's bugging me is you say everything works 100% fine on the first run, it's only after that first run where the static files start to fail. What if you uncomment those lines and run: ./run clean and then docker-compose up? Any difference?

I'm also really confused how it works initially and then stops.
Tried to downgrade but doesn't seem to work, so uninstalled Docker now, installed 3.3.3 again, pruned everything, docker-compose up --build, everything works. I stop Ctrl-C, docker-compose up and it's broken. I run ./run clean, then docker-compose up but still broken.

I'd be curious to see if the Flask example app works at: https://github.com/nickjj/docker-flask-example
It follows the same steps as this app. Copy the 2 files, up the project with the same commands, etc. Does it work after the 1st run?

I think it generates assets, at least I get
❯ tree assets
assets
├── app
│   ├── app.css
│   └── app.js
├── package.json
├── postcss.config.js
├── static
│   ├── 502.html
│   ├── android-chrome-192x192.png
│   ├── android-chrome-512x512.png
│   ├── apple-touch-icon.png
│   ├── browserconfig.xml
│   ├── favicon-16x16.png
│   ├── favicon-32x32.png
│   ├── favicon.ico
│   ├── images
│   │   └── flask.png
│   ├── maintenance.html
│   ├── mstile-150x150.png
│   ├── robots.txt
│   ├── safari-pinned-tab.svg
│   └── site.webmanifest
├── tailwind.config.js
├── webpack.config.js
└── yarn.lock

but the page looks weird also. Missing styling too.

The assets/ directory has your source files, it's where Webpack looks to find your css, js, images, etc. The public/ directory in Flask is the same as the priv/static/ directory for Phoenix. It's what's served by the web server and is populated by Webpack. So in this case it looks like it's empty.
The only pattern I'm seeing here is that all assets that happen to be in a directory end up not being generated by Webpack for some reason. If you ls -la in the root of the project is the public/ directory owned by lauritz:staff?

What about the the directory itself?

That at least proves the volume is working as intended.
So now it's a problem of why do the static files in the root of the assets get copied over but not the css, js and images.
Try this, start with docker-compose down and then in the .env file:

Change both the FLASK_ENV and NODE_ENV to production
Change DOCKER_WEB_VOLUME to.:/tmp

Then mv docker-compose.override.yml docker-compose.override.yml.bak
Then docker-compose up --build
Does the page load normally then?

Yep! It does load correctly now, I see Flask logo and what I'd expect. It continues to look correct after Ctrl-C and docker-compose up.
I tried to follow the same instructions for the Phoenix repo, with export DOCKER_WEB_VOLUME=./priv/static:/app/priv/static and that also works correctly, even after stopping the services and uping them again.

Those steps for the Flask one build production ready static files (digested) and don't depend on Webpack to be running to produce new static files on the fly. It's not really too suitable for development since you'd have to stop everything, rebuild and wait before seeing changes to your CSS and JS. But it fully eliminates volumes from the picture.
The Phoenix equiv. would be setting MIX_ENV to prod and doing the same steps.
When it worked for you, did you keep it set to dev and also didn't do the mv command? So in the end all you did was disable the volume by using a different path with DOCKER_WEB_VOLUME?

I followed all the instructions (unfortunately), so changed both to prod/production, moved docker-compose.override.yml and then built again. It's still a problem during development, it is so weird. Also that we share the same Docker version (build). Are you on an M1 Mac too?
The best workaround for now is pruning everything, build from scratch and just don't shut it down, it's not often I switch project/reboot my computer anyway. It's just completely mind-boggling this issue to me.

Do the permissions of anything change between the first and 2nd run? No right?
I'm on Windows, so a wildly different environment.

Might be an OS thing, not sure.  Do you happen to know anyone else who has an M1 to see if it works for them?

I'm not sure actually, will ask around. I've planned to completely wipe my computer anyway, it's running low on storage, so a good time for a cleaning. I'll try that and see if that improves anything and report back, will probably get around to it Wednesday or something. Thanks for your time so far!

Nice work on figuring out a solution.
Personally I've never heard of anyone ever having to do this maybe it has something to do with the M1 being a total beast and can start things faster than most other set ups and this was never an issue with other hardware.
In any case, if I apply that patch to my end here things work the same.

maybe it has something to do with the M1 being a total beast and can start things faster than most other set ups

This is my theory at least, and I think it makes sense with what you mention in the README, that if styling is blank, then it's because Webpack is still running. Not sure how it can be so fast here though. I guess because it runs first, it emits files but then web mounts after and overwrites it somehow? It still doesn't completely make sense to me that order matters.
It's a bit more inconsistent with pruning since I cleaned my machine but when I'm able to build it correctly (without the two lines) I see
[emitted lots of Docker build lines]
=> exporting to image                                                                                1.0s
 => => exporting layers                                                                                1.0s
 => => writing image sha256:427f3f03e56e1b768c11fe18dcb04ba0fc8f2ce440864316c86542ef534fa522                                                              0.0s
 => => naming to docker.io/library/hellophoenix_web                                                                                0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
Creating hellophoenix_postgres_1 ... done
Creating hellophoenix_webpack_1  ... done
Creating hellophoenix_web_1      ... done
Attaching to hellophoenix_postgres_1, hellophoenix_webpack_1, hellophoenix_web_1
postgres_1  | The files belonging to this database system will be owned by user "postgres".
postgres_1  | This user must also own the server process.
postgres_1  |
postgres_1  | The database cluster will be initialized with locale "en_US.utf8".
postgres_1  | The default database encoding has accordingly been set to "UTF8".
postgres_1  | The default text search configuration will be set to "english".
postgres_1  |
postgres_1  | Data page checksums are disabled.
postgres_1  |
postgres_1  | fixing permissions on existing directory /var/lib/postgresql/data ... ok
postgres_1  | creating subdirectories ... ok
postgres_1  | selecting dynamic shared memory implementation ... posix
postgres_1  | selecting default max_connections ... 100
yarn run v1.22.5
warning package.json: No license field
$ /node_modules/webpack/bin/webpack.js --mode=development --progress --watch
postgres_1  | selecting default shared_buffers ... 128MB
postgres_1  | selecting default time zone ... Etc/UTC
postgres_1  | creating configuration files ... ok
postgres_1  | running bootstrap script ... ok
postgres_1  | performing post-bootstrap initialization ... ok
postgres_1  | syncing data to disk ... initdb: warning: enabling "trust" authentication for local connections
postgres_1  | You can change this by editing pg_hba.conf or using the option -A, or
postgres_1  | --auth-local and --auth-host, the next time you run initdb.
postgres_1  | ok
postgres_1  |
postgres_1  |
postgres_1  | Success. You can now start the database server using:
postgres_1  |
postgres_1  |     pg_ctl -D /var/lib/postgresql/data -l logfile start
postgres_1  |
postgres_1  | waiting for server to start....2021-06-10 12:07:16.279 UTC [48] LOG:  starting PostgreSQL 13.3 (Debian 13.3-1.pgdg100+1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
postgres_1  | 2021-06-10 12:07:16.280 UTC [48] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1  | 2021-06-10 12:07:16.284 UTC [49] LOG:  database system was shut down at 2021-06-10 12:07:16 UTC
postgres_1  | 2021-06-10 12:07:16.287 UTC [48] LOG:  database system is ready to accept connections
postgres_1  |  done
postgres_1  | server started
postgres_1  | CREATE DATABASE
postgres_1  |
postgres_1  |
postgres_1  | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
postgres_1  |
postgres_1  | 2021-06-10 12:07:16.508 UTC [48] LOG:  received fast shutdown request
postgres_1  | waiting for server to shut down....2021-06-10 12:07:16.512 UTC [48] LOG:  aborting any active transactions
postgres_1  | 2021-06-10 12:07:16.514 UTC [48] LOG:  background worker "logical replication launcher" (PID 55) exited with exit code 1
postgres_1  | 2021-06-10 12:07:16.514 UTC [50] LOG:  shutting down
postgres_1  | 2021-06-10 12:07:16.531 UTC [48] LOG:  database system is shut down
postgres_1  |  done
postgres_1  | server stopped
postgres_1  |
postgres_1  | PostgreSQL init process complete; ready for start up.
postgres_1  |
postgres_1  | 2021-06-10 12:07:16.624 UTC [1] LOG:  starting PostgreSQL 13.3 (Debian 13.3-1.pgdg100+1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
postgres_1  | 2021-06-10 12:07:16.624 UTC [1] LOG:  listening on IPv4 address "<IP_ADDRESS>", port 5432
postgres_1  | 2021-06-10 12:07:16.624 UTC [1] LOG:  listening on IPv6 address "::", port 5432
postgres_1  | 2021-06-10 12:07:16.626 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1  | 2021-06-10 12:07:16.630 UTC [76] LOG:  database system was shut down at 2021-06-10 12:07:16 UTC
postgres_1  | 2021-06-10 12:07:16.634 UTC [1] LOG:  database system is ready to accept connections
web_1       | Erlang/OTP 24 [erts-12.0.2] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1]
web_1       |
37% building 1/2 entries 9/9 dependencies 5/6 modules
webpack_1   | warn - You have enabled the JIT engine which is currently in preview.
webpack_1   | warn - Preview features are not covered by semver, may introduce breaking changes, and can change at any time.
assets by status 49.2 KiB [compared for emit]
webpack_1   |   assets by path *.png 29.8 KiB 4 assets
webpack_1   |   assets by path *.html 1.87 KiB
webpack_1   |     asset 502.html 1020 bytes [compared for emit] [from: static/502.html] [copied]
webpack_1   |     asset maintenance.html 892 bytes [compared for emit] [from: static/maintenance.html] [copied]
webpack_1   |   asset favicon.ico 14.7 KiB [compared for emit] [from: static/favicon.ico] [copied]
webpack_1   |   asset safari-pinned-tab.svg 1.9 KiB [compared for emit] [from: static/safari-pinned-tab.svg] [copied]
webpack_1   |   asset manifest.json 489 bytes [compared for emit] [from: static/manifest.json] [copied]
webpack_1   |   asset browserconfig.xml 246 bytes [compared for emit] [from: static/browserconfig.xml] [copied]
webpack_1   |   asset robots.txt 185 bytes [compared for emit] [from: static/robots.txt] [copied]
webpack_1   | assets by status 224 KiB [emitted]
webpack_1   |   asset js/app.js 173 KiB [emitted] (name: main)
webpack_1   |   asset images/phoenix.png 37.8 KiB [emitted] [from: static/images/phoenix.png] [copied]
webpack_1   |   asset css/app.css 13.4 KiB [emitted] (name: main)
webpack_1   | Entrypoint main 186 KiB = css/app.css 13.4 KiB js/app.js 173 KiB
webpack_1   | runtime modules 937 bytes 4 modules
webpack_1   | cacheable modules 159 KiB
webpack_1   |   modules by path ../../node_modules/ 158 KiB
webpack_1   |     ../../node_modules/phoenix_html/priv/static/phoenix_html.js 2.21 KiB [built] [code generated]
webpack_1   |     ../../node_modules/phoenix/priv/static/phoenix.js 38.9 KiB [built] [code generated]
webpack_1   |     ../../node_modules/topbar/topbar.min.js 4 KiB [built] [code generated]
webpack_1   |     ../../node_modules/phoenix_live_view/priv/static/phoenix_live_view.js 113 KiB [built] [code generated]
webpack_1   |   modules by path ./app/ 1020 bytes
webpack_1   |     ./app/app.css 50 bytes [built] [code generated]
webpack_1   |     ./app/app.js 966 bytes [built] [code generated]
webpack_1   | css ../../node_modules/css-loader/dist/cjs.js!../../node_modules/postcss-loader/dist/cjs.js!./app/app.css 13.4 KiB [code generated]
webpack_1   | webpack 5.36.1 compiled successfully in 2351 ms
web_1       | Compiling 16 files (.ex)
assets by status 273 KiB [cached] 14 assets
webpack_1   | Entrypoint main 186 KiB = css/app.css 13.4 KiB js/app.js 173 KiB
webpack_1   | cached modules 159 KiB (javascript) 13.4 KiB (css/mini-extract) 937 bytes (runtime) [cached] 10 modules
webpack_1   | ./app/app.css 50 bytes [built]
webpack_1   | webpack 5.36.1 compiled successfully in 129 ms
web_1       | Generated hello app
web_1       | [info] Running HelloWeb.Endpoint with cowboy 2.9.0 at :::8000 (http)
web_1       | [info] Access HelloWeb.Endpoint at http://localhost:8000
web_1       | Interactive Elixir (1.12.1) - press Ctrl+C to exit (type h() ENTER for help)
web_1       | iex(1)> [info] GET /
web_1       | [debug] Processing with HelloWeb.PageController.home/2
web_1       |   Parameters: %{}
web_1       |   Pipelines: [:browser]
web_1       | [info] Sent 200 in 11ms

It does look like that web is the last service to run? However, the files are there and the page loads fine. I then shut it down, run docker-compose up and now the order is consistently this
❯ docker-compose up
Starting hellophoenix_webpack_1  ... done
Starting hellophoenix_postgres_1 ... done
Starting hellophoenix_web_1      ... done
Attaching to hellophoenix_postgres_1, hellophoenix_webpack_1, hellophoenix_web_1
postgres_1  |
postgres_1  | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres_1  |
postgres_1  | 2021-06-10 12:10:30.144 UTC [1] LOG:  starting PostgreSQL 13.3 (Debian 13.3-1.pgdg100+1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
postgres_1  | 2021-06-10 12:10:30.144 UTC [1] LOG:  listening on IPv4 address "<IP_ADDRESS>", port 5432
postgres_1  | 2021-06-10 12:10:30.144 UTC [1] LOG:  listening on IPv6 address "::", port 5432
postgres_1  | 2021-06-10 12:10:30.146 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1  | 2021-06-10 12:10:30.148 UTC [26] LOG:  database system was shut down at 2021-06-10 12:10:24 UTC
postgres_1  | 2021-06-10 12:10:30.151 UTC [1] LOG:  database system is ready to accept connections
yarn run v1.22.5
warning package.json: No license field
$ /node_modules/webpack/bin/webpack.js --mode=development --progress --watch
37% building 1/2 entries 9/9 dependencies 5/6 modules
webpack_1   | warn - You have enabled the JIT engine which is currently in preview.
webpack_1   | warn - Preview features are not covered by semver, may introduce breaking changes, and can change at any time.
assets by path *.png 29.8 KiB
webpack_1   |   asset android-chrome-512x512.png 12.5 KiB [compared for emit] [from: static/android-chrome-512x512.png] [copied]
webpack_1   |   asset android-chrome-192x192.png 7.42 KiB [compared for emit] [from: static/android-chrome-192x192.png] [copied]
webpack_1   |   asset apple-touch-icon.png 7.16 KiB [compared for emit] [from: static/apple-touch-icon.png] [copied]
webpack_1   |   asset mstile-150x150.png 2.71 KiB [compared for emit] [from: static/mstile-150x150.png] [copied]
webpack_1   | assets by chunk 186 KiB (name: main)
webpack_1   |   asset js/app.js 173 KiB [compared for emit] (name: main)
webpack_1   |   asset css/app.css 13.4 KiB [compared for emit] (name: main)
webpack_1   | assets by path *.html 1.87 KiB
webpack_1   |   asset 502.html 1020 bytes [compared for emit] [from: static/502.html] [copied]
webpack_1   |   asset maintenance.html 892 bytes [compared for emit] [from: static/maintenance.html] [copied]
webpack_1   | 6 assets
webpack_1   | Entrypoint main 186 KiB = css/app.css 13.4 KiB js/app.js 173 KiB
webpack_1   | runtime modules 937 bytes 4 modules
webpack_1   | cacheable modules 159 KiB
webpack_1   |   modules by path ../../node_modules/ 158 KiB
webpack_1   |     ../../node_modules/phoenix_html/priv/static/phoenix_html.js 2.21 KiB [built] [code generated]
webpack_1   |     ../../node_modules/phoenix/priv/static/phoenix.js 38.9 KiB [built] [code generated]
webpack_1   |     ../../node_modules/topbar/topbar.min.js 4 KiB [built] [code generated]
webpack_1   |     ../../node_modules/phoenix_live_view/priv/static/phoenix_live_view.js 113 KiB [built] [code generated]
webpack_1   |   modules by path ./app/ 1020 bytes
webpack_1   |     ./app/app.css 50 bytes [built] [code generated]
webpack_1   |     ./app/app.js 966 bytes [built] [code generated]
webpack_1   | css ../../node_modules/css-loader/dist/cjs.js!../../node_modules/postcss-loader/dist/cjs.js!./app/app.css 13.4 KiB [code generated]
webpack_1   | webpack 5.36.1 compiled successfully in 1601 ms
assets by status 273 KiB [cached] 14 assets
webpack_1   | Entrypoint main 186 KiB = css/app.css 13.4 KiB js/app.js 173 KiB
webpack_1   | cached modules 159 KiB (javascript) 13.4 KiB (css/mini-extract) 937 bytes (runtime) [cached] 10 modules
webpack_1   | ./app/app.css 50 bytes [built]
webpack_1   | webpack 5.36.1 compiled successfully in 120 ms
web_1       | Erlang/OTP 24 [erts-12.0.2] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1]
web_1       |
web_1       | [info] Running HelloWeb.Endpoint with cowboy 2.9.0 at :::8000 (http)
web_1       | [info] Access HelloWeb.Endpoint at http://localhost:8000
web_1       | Interactive Elixir (1.12.1) - press Ctrl+C to exit (type h() ENTER for help)

With a complete new build, it seems that one message from webpack service gets mixed with web service. On shut down and new up, they are completely separated. I recorded a little session to show how it looks for me

That's so interesting that it starts webpack first for you, even to the point where it blocks the web service from starting until webpack is 100% fully up after the first run. That never happens here. On my end web begins to start, webpack begins to start and then webpack kicks in and eventually compiles after a few seconds. During those few seconds if I access the page the CSS is all messed up because it's not fully compiled but eventually all of the files get written out correctly and reloading the browser works.
The compile speed for webpack isn't drastically different on my box btw. Mine finishes in about 2.8 seconds where as yours is 1.6 seconds. That is a big difference but it's not like it takes 8 seconds on mine but 1 for yours, etc..
Yeah I think a 2 line PR would be good for the example override file. Just please make sure it's alphabetized by putting it after the command property.

It's very weird, not sure why the order is different on M1 Macs or my machine at least. It was a tricky one to solve but at least the solution is simple and I guess good to know about in the future if a similar issue happens. Created the PR and made sure it's after command.
I realize my dad has an M1 machine too, so I'll try it on his, just to see if it's an M1 issue or my machine that's weird.

Can you check that out on his machine before we merge the PR?


[QUESTION] List of engines

By what criteria are engines added to this list?

I think if they are generally NOTABLE in some way. Doesn't need to be "famous" or anything, just like, a generally something interesting for people to poke into!
From The Awesome Manifesto

try to only include actual awesome stuff in your list. After all, it's a curation, not a collection.

Research if the stuff you're including is actually awesome. Only put stuff on the list that you or another contributor can personally recommend.

i would suggest adding Kade Engine Fork by BoloVEVO, i think it's pretty cool.


App rotation on request

Hi !
Just curious: Is it right that the app somehow forget that it sent a sign-in request if the user turn the screen of the device after pushing the button ? The http request get cancelled and loading sign get removed.
How is your way to handle this when the current state (loading) can barely get stored in DB ?

Hi, it's definitely not ideal that the request gets cancelled on screen rotation however it's usually not a major problem for the user. This app doesn't handle that case. One possible way to achieve this would be to keep presenter instances across configuration changes so the same presenter instance can be re-attached to the new view.


adding 30 day cap for invalid drops

Description of the change:
Reason for the change:
Link to original source:

Closes #

Hello @Whatthefoxsays,Thanks again for the PR! We appreciate your contribution and look forward to                                              continued collaboration. Thanks!                                             Team SendGrid DX


Deployment link of my Portfolio

Live demo to my Portfolio project :
https://otmane-echchafyky.github.io/Portfolio/

Hi @OTMANE-ECHCHAFYKY
Your project is complete!  There is nothing else to say other than... it's time to merge it
Congratulations!

Highlights
✔ Portfolio correctly deployed 💯
Optional suggestions
Every comment with the [OPTIONAL] prefix won't stop the approval of this PR. However, I strongly recommend you to take them into account as they can make your code better. Some of them were simply missed by the previous reviewer and addressing them will really improve your application.
Cheers and Happy coding!:aplauso::aplauso::aplauso:
Feel free to leave any questions or comments in the PR thread if something is not 100% clear.
Please, remember to tag me in your question so I can receive the notification.
As described in the Code reviews limits policy  you have a limited number of reviews per project (check the exact number in your Dashboard). If you think that the code review was not fair, you can request a second opinion using this form.._)

Thank you @BregornOriginal


TaskBook.json does not save generated default new file

A default TaskBook.json file is generated whenever a file is invalid. However, this newly created file cannot be found within the data folder after starting and closing the program (without running any commands). Only when we run a command (eg. add), a new json file will be created and stored.

I think the behavior of "write on change" is valid and should be kept as it is as it does not affect the User experience.
For the second point the recovery step is simple (delete the save file). Will add to UG as FAQ.
Lowered severity to low as it won't affect normal operations.


feat: add option to parallelize pdfs

We aren’t planning to bring server side parallel mode back, but this can be a game changer for big documents. We should have the client optionally break a pdf into pages and send these off async. See this example.
We need to see if this it's possible to hook into the autogenerated code for something like this.

This should likely be the default. IMO, https://github.com/Unstructured-IO/unstructured/issues/2461 is a blocker.

It's not easy with new client parameter split_pdf_page (boolean), because the python client is now generated based on the openapi.yaml , which in turn is generated based on code by unstructured-api. So, if we add there a new param, it will be also visible in the Swagger preview (like http://<IP_ADDRESS>:8000/general/docs) of endpoints, misleading users that backend API takes such parameter. But it's not, as clearly the param is for client only.
I looked for ways of hiding the endpoint param in Openapi spec, but it's not supported even for the more basic case of endpoints https://github.com/OAI/OpenAPI-Specification/issues/433 Perhaps some clever hacking would work here, but it might not be pretty. So I wanted to ask if it's ok for now (or forever), if we have a parameter which will work only on client side, and will be ignored backend side? We could rename it to client_split_pdf_page.

Yeah, that's a good question. Adding something on the server for this does feel messy. I wonder if we could establish a better pattern for configuring this sorts of things on the client - some sort of Configuration object that we pass in to the sdk constructor. We can see what Speakeasy thinks about this, or figure out how to support the behavior on our own.
I think, for now, we can add the logic and assume that it will be the default behavior. We could add configuration via env variable if users really want to be able to turn it off.

Ok I asked the Speakeasy support and they suggested a way for adding the parameter only on client side, through overlay. This is relatively simple and looks like the best solution. The draft implementation is already in PR.


[PDI-19064] - The Append Option in JSON Output will create new files when the number of rows exceed the number of blocs

@ssamora
@smmribeiro
@moraesvc

:white_check_mark: Build finished in 26m 20s
Build command:
mvn clean verify -B -e -Daudit -amd -pl core,plugins/json/core

:ok_hand: All tests passed!
Tests run: 5124, Failures: 0, Skipped: 7    Test Results

:information_source:  This is an automatic message

:white_check_mark: Build finished in 34m 11s
Build command:
mvn clean verify -B -e -Daudit -amd -pl core,engine,plugins/json/core

:ok_hand: All tests passed!
Tests run: 5127, Failures: 0, Skipped: 7    Test Results

:information_source:  This is an automatic message


"Borrowing" the code

Hi!
Thank you for the script. It works great.
I want to integrate your script into mine. I will comment there in the code that I used your code and provide a link to your GitHub.
I hope that it is fine.
Is there any other way I can cite your work?

Awesome! I encourage you to check out my krust repository as that's a Rust implementation that might also be interesting.
I'd love to see your combined work once it's out in the open. Thanks for the heads up.


Models support for v. 2.0 and basic amount of tests added

added flat models support for API v. 2.0, output fields and marshal_with supported (as shown in swg_app_20 example)
added support for nested models
detect_producers function moved inside BaseProducer class
basic test set added (import, integration and swagger schema validation for swagger API 2.0)

That's a lot of work! Awesome!


[5.5] Respect SWIFT_TOOLCHAIN_MACOS_DEPLOYMENT_TARGET when set (such as during Swift toolchain build)

This is the Swift 5.5 nomination of fixes for @available() annotations merged to mainline in #220.
(cherry picked from commit 21a79185b2ea8f89b9253ed8cdf2338bf564c499)

@swift-ci please test


Customizable fastscroller

Modified FastScroller to make it easier to use with existing adapters:

moved Animator classes to a separate file to make FastScroller focused on fast scrolling
added FastScrollerDelegate and FastScrollerAdapterInterface to make using FastScroller more straightforward
removed dependency to Utils.java since this is defined under the flexibleadapter package
removed custom attribute for bubbleAndHandleColor since this will conflict with the color attribute in the drawables

Other changes:

fixed app crash in the sample app
ignored touch events when RecyclerView is not scrollable
added custom attribute to enable/disable the bubble

I've reviewed what done until now, in general, you have done a very good job! :1st_place_medal:

@davideas i'm done with the PR :)
i added a new option to ignore touches outside the handle (similar to google docs)

Merged, really many thanks! please check my last commit in this branch.

@arpinca, do you know how to get the handle shape like this when released?
https://github.com/turing-tech/MaterialScrollBar

Can you be more specific?

@arpinca, you can see it opening the video from that home page or see the 2nd image here, the handle is rounded and stick out a little bit. I think he does that in the class Handle.java not from xml.

@davideas sorry for the late reply >.<
Hmm, how about via layer-list? As much as possible, I prefer creating drawables via xml. I'll try to check this on the weekend.

@arpinca, xml is ok. Cool.