Should we re-consider zinc's incremental compilation?

Given zinc 1.0 (http://scala-lang.org/blog/2017/11/03/zinc-blog-1.0.html)
I'm wondering if we should take another look at if/how to integrate zinc's incremental compiler into rules_scala.
@johnynek @gkossakowski wdyt?

I think this is an interesting question. I think the problem is we would need to basically implement a build system in memory to do it.
If we hashed all the jar inputs to a rule, you could use that as a key to a mutable in memory filesystem that keeps all the state for zinc. Then zinc could run statefully with that Key.
That circumvents the intent of bazel: stateless compilers with all state managed by bazel.
Alternatively we could push to add a way for bazel to track compiler state. I have contemplated this in the past where rules have optional state with invalidation policies. When the rule runs bazel passes it the state. The rule can say things like a dependency change invalidates state, etc...

Let's try and discuss this at the conference with Damien/Ulf/Dmitry
On Sat, Nov 4, 2017 at 5:14 PM P. Oscar Boykin<EMAIL_ADDRESS>wrote:

I think this is an interesting question. I think the problem is we would
need to basically implement a build system in memory to do it.
If we hashed all the jar inputs to a rule, you could use that as a key to
a mutable in memory filesystem that keeps all the state for zinc. Then zinc
could run statefully with that Key.
That circumvents the intent of bazel: stateless compilers with all state
managed by bazel.
Alternatively we could push to add a way for bazel to track compiler
state. I have contemplated this in the past where rules have optional state
with invalidation policies. When the rule runs bazel passes it the state.
The rule can say things like a dependency change invalidates state, etc...
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/rules_scala/issues/328#issuecomment-341904423,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIF3eBDL9FclgzGqIYwScr9oKYEsBqks5szH8_gaJpZM4QR_9h
.

I agree that rules (or actions to be precise) not being able to carry an
internal state is problematic.
However, one could use zinc purely within one target and in-memory (in
worker mode). See this discussion about bazel+zinc
https://gitter.im/sbt/zinc-contrib?at=59f725fdb20c6424296673a1
On 4 November 2017 at 09:12, Ittai Zeidman<EMAIL_ADDRESS>wrote:

Let's try and discuss this at the conference with Damien/Ulf/Dmitry
On Sat, Nov 4, 2017 at 5:14 PM P. Oscar Boykin<EMAIL_ADDRESS>wrote:

I think this is an interesting question. I think the problem is we would
need to basically implement a build system in memory to do it.
If we hashed all the jar inputs to a rule, you could use that as a key to
a mutable in memory filesystem that keeps all the state for zinc. Then
zinc
could run statefully with that Key.
That circumvents the intent of bazel: stateless compilers with all state
managed by bazel.
Alternatively we could push to add a way for bazel to track compiler
state. I have contemplated this in the past where rules have optional
state
with invalidation policies. When the rule runs bazel passes it the state.
The rule can say things like a dependency change invalidates state,
etc...
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<https://github.com/bazelbuild/rules_scala/issues/
328#issuecomment-341904423>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/
ABUIF3eBDL9FclgzGqIYwScr9oKYEsBqks5szH8_gaJpZM4QR_9h>
.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/rules_scala/issues/328#issuecomment-341908978,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAQeUzrQnfo5LbpRDmt7K6ETQ_Fw_2vks5szIzrgaJpZM4QR_9h
.

--
gkk

@lihaoyi you had been looking into this...did anything come of it?
I plan on getting Zinc working in the next month or so, and don't want to duplicate any work.

we are pretty happy with how well caching is working for us currently, so I still stand by the feeling that getting hermetic builds + caching and remote execution is the scalable way forward.
If zinc can be a part of that, great, but I personally don't want to give up hermetic builds.

FWIW I have a prototype of Zinc based rules with the intention of applying learnings back to rules_scala. I'm not currently concerned with incremental compilation and the primary goal was to get access to the analysis output of Zinc for downstream analysis.

we are pretty happy with how well caching is working for us currently,

Good to know. My belief remains that we cannot get near-sbt build times, without creating significant burden on creating fine grained DAGs.

If zinc can be a part of that, great, but I personally don't want to give up hermetic builds.

I assume that adding a opt-in worker strategy that persists and re-uses analysis is acceptable.
Worker strategies in general are trust-me hermeticity. This is a riskier (but still low risk, IMO) trust-me approach.

FWIW I have a prototype of Zinc based rules

@andyscott exciting. What's the status of that?

It works, but it's early stage. And full disclosure: It's a separate repository written mostly from scratch. The intention is to use the latest and greatest Bazel patterns, experiment with solving some of the key issues faced in this repo, and then incrementally apply the learnings back here. I also needed a way to really dive into Bazel without being encumbered with lots of existing code. If you're curious to dig through my mess... https://github.com/andyscott/rules_scala_annex.

Is there a way (or is it the default) for Zinc to guarantee that the incremental compile result is identical to a non-incremental compile result? I've speculated pushing for Bazel to allow 'principled use' of an incremental compiler if Bazel had flags to run non-incremental variants of the job as a 'check' on the incremental compile.
Scala (with Zinc) was going to be my test case; however, I've never really had a chance to look deeply at the reliability of the incremental compile.

@gkossakowski might be able to weigh in here

Is there a way (or is it the default) for Zinc to guarantee that the incremental compile result is identical to a non-incremental compile result?

Not generally, because there are many possible inputs to an "incremental compile". So, you could check a select number of state transitions but not all. And certainly zinc has tests for that purpose.
In any worker strategy, it's possible to be nonhermetic. In fact, I believe scalac 2.12 had precisely this problem; statics were polluting subsequent compilation runs. The difference here is that we are purposefully "polluting" runs with past state rather than accidentally doing it.

Actually I think Gregg turned it off for 2.12 because he found that it’s
problematic
On Sat, 7 Apr 2018 at 3:51 Paul Draper<EMAIL_ADDRESS>wrote:

Is there a way (or is it the default) for Zinc to guarantee that the
incremental compile result is identical to a non-incremental compile result?
Not generally, because there are many possible inputs to an "incremental
compile". So, you could check a select number of state transitions but not
all. And certainly zinc has tests for that purpose.
In any worker strategy, it's possible to be nonhermetic. In fact, I
believe scalac 2.12 had precisely this problem; statics were polluting
subsequent compilation runs. The difference here is that we are
purposefully "polluting" runs with past state rather than accidentally
doing it.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/rules_scala/issues/328#issuecomment-379420240,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIFwh8O2m8bkVveONXZdUZCSAGfVUJks5tmA14gaJpZM4QR_9h
.

I've been following this for a while, and have a question. Is Zinc a possibility to use, at least for developer only iterative development builds, in the near term? We have several units of compilation that if changed, trigger 100+ second rebuilds for our developers that are non-trivial to break into smaller DAG's. Momentum is building to switch our build back to sbt, because allowing developers to iterate several orders of magnitude faster is more important than the faster centralized builds that occur as part of our release process. We get a lot of benefit from bazel and I'd like to keep it, but the developer productivity is really suffering because of these long build times.

You might want to take a look at this:
https://github.com/andyscott/rules_scala_annex
they are using zinc there.
I would like to see us write a tool to break large targets up using https://github.com/lightbend/scala-sculpt so that you could automatically split large targets. This might not be very hard, but we haven't started the project yet. Would someone on your end be interested in contributing that?
At stripe, we use caching (and are testing caches readable and writable from laptops). I don't have faith (I have seen evidence against and no strong evidence for the case) that zinc is reproducible. So, using it with caching is not a prudent plan.
I hear you about the long compilation times. We have a few bad targets in our repo as well.  I think the main difference between these rules and annex are that we have been focused on meeting bazel's model of speed through reproducibility and caching. Giant targets hinder that.
cc @pauldraper @jvican who I think have visions closer to what you want.

@bpatters there is a functioning version of what you are describing at https://github.com/andyscott/rules_scala_annex .
You can opt-in to incremental Zinc compilation by providing a local file path for workers to keep analysis files. I no longer work at Lucid, but I know that their developers are currently using this mode. Or justification in creating was:
(1) Practically it is hard to get good incremental builds in Scala. (a) Scala is already slow. (b) There are limitations on ijar-like improvements due to complex types and macro introspection. (c) Even with small targets, the transitive graphs are still very large.
(2) rules_scala allow you to disable transitive dependencies, but IMO that is very cumbersome to work with, especially relative to the savings I've seen. Google's choice for Java:

Bazel always passes the entire transitive classpath to javac, not only the direct classpath. This frees the user from having to aggregate their transitive dependencies manually. In other words, javac never fails because of a missing symbol, as long as every rule specifies its direct dependencies.
https://blog.bazel.build/2017/06/28/sjd-unused_deps.html

(3) Creating enormous numbers of targets is unappealing because I'd rather have developers work with logical dependencies graphs. Oscar's wording of this perspective in 2016.

I know the canonical answer to this is to make finer grained targets, but to me this is not a complete solution. First, it is a bit painful to create one target for each item in the logical dependency graph just to speed up compilation. Second it conflates the real logical blocks of your system, which people can understand, with the nodes in the strict dependency graph, which might be enormous.
https://groups.google.com/forum/#!topic/bazel-discuss/3iUy5jxS3S0

(4) Worker strategies already state sharing across subsequent invocations; in fact, sharing state is the entire point of having a worker. It's essentially impossible to prove them correct -- whether JVM JIT or Zinc caching, and in fact, both of these have had bugs. The question is: is the state sharing correct enough? Based on considerable recent experience, I say yes to both, but again, it's not a proof.

A little off-topic, but based on learning from rules_scala_annex, even outside incremental compilation I do think that rules_scala has good reason to adopt zinc: (a) the dependency detection (for strict deps and usused deps) is far more precise (b) it makes test discovery for every Scala framework easy.
If that were done, opt-in Zinc caching for workers would be a simple -- if controversial -- addition at that point.

Re your last point of dependency detection- I’d really wish this was a
module part of zinc we could depend on...
On Fri, 31 Aug 2018 at 22:27 Paul Draper<EMAIL_ADDRESS>wrote:

@bpatters https://github.com/bpatters there is a functioning version of
what you are describing at https://github.com/andyscott/rules_scala_annex
.
You can opt-in to incremental Zinc compilation by providing a local file
path for workers to keep analysis files. I no longer work at Lucid, but I
know that their developers are currently using this mode. Or justification
in creating was:
(1) Practically it is hard to get good incremental builds in Scala. (a)
Scala is already slow. (b) There are limitations on ijar-like improvements
due to complex types and macro introspection. (c) Even with small targets,
the transitive graphs are still very large.
(2) rules_scala allow you to disable transitive dependencies, but IMO that
is very cumbersome to work with, especially relative to the savings I've
seen. Google's choice for Java:
Bazel always passes the entire transitive classpath to javac, not only the
direct classpath. This frees the user from having to aggregate their
transitive dependencies manually. In other words, javac never fails because
of a missing symbol, as long as every rule specifies its direct
dependencies.
https://blog.bazel.build/2017/06/28/sjd-unused_deps.html
(3) Creating enormous numbers of targets is unappealing because I'd rather
have developers work with logical dependencies graphs. Oscar's wording of
this perspective in 2016.
I know the canonical answer to this is to make finer grained targets, but
to me this is not a complete solution. First, it is a bit painful to create
one target for each item in the logical dependency graph just to speed up
compilation. Second it conflates the real logical blocks of your system,
which people can understand, with the nodes in the strict dependency graph,
which might be enormous.
https://groups.google.com/forum/#!topic/bazel-discuss/3iUy5jxS3S0
(4) Worker strategies already state sharing across subsequent invocations;
in fact, sharing state is the entire point of having a worker. It's
essentially impossible to prove them correct -- whether JVM JIT or Zinc
caching, and in fact, both of these have had bugs. The question is: is the
state sharing correct enough? Based on considerable recent experience, I
say yes to both, but again, it's not a proof.
A little off-topic, but based on learning from rules_scala_annex, even
outside incremental compilation I do think that rules_scala has good
reason to adopt zinc: (a) the dependency detection (for strict deps and
usused deps) is far more precise (b) it makes test discovery for every
Scala framework easy.
If that were done, opt-in Zinc caching for workers would be a simple -- if
controversial -- addition at that point.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/rules_scala/issues/328#issuecomment-417766999,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIF0BgGtXv5Dxa5hEBhwL2MGmboip2ks5uWY45gaJpZM4QR_9h
.

I would welcome bazel adding the ability to make dynamic internal targets that could automatically split up big target mudballs, but I don't have the time to try to keep pushing bazel devs on this and they have not been super excited. Even then, I'm not sure it would be faster since just building that internal graph and keeping it accurate is the part about zinc (I'm told) that causes the reproducibility problems.
We have about 60-70 developers editing maybe 500k lines of scala at Stripe and the non-transitive classpaths along with the recently enabled unused-deps errors seems to work pretty well. Our CIs are very fast (several minutes for each pull), remote caching hit rate is pretty good generally above 80% and very often above 90%.
If scala compiled as fast as java, I would probably prefer to put the transitive classpath for compilation, but since scala is so much slower, I think the occasional pain of having to think about where a depencency comes from (which can be solved by using strict-deps=warn) is well worth the win in isolation from rebuilds. We should probably document more this approach: when the deps are set up, iterate without strict-deps warn, when they are not, use that mode until you don't have warnings and it will tell you what targets to add.
I disagree that sharing state is the point of the worker, the point of the worker is to share the jitting and classloading, but still to have a stateless pure function for build. We have had very few bugs around that, and we have a test (which I think zinc does not) that our builds are reproducible.
To me, the most compelling reason to use zinc is to get the test framework support. I am not opposed to using zinc if it can pass our reproducibility tests and doesn't cause a migration pain for existing production users. I am skeptical our users would be better off investing in zinc vs building a tool to split large targets automatically, which could then leverage a fully distributed cache and remote execution, something we preclude as soon as non-reproducibility enters the picture. Trading remote execution or the ability to share artifacts built by other laptops in a large org, seems like it would be a net negative.
In any case, annex is there today for those who may have different needs. I think we should stay focused on support for big code bases, large teams, and caching/remote execution efficiency. For folks not in that boat, I'm not sure I can say that bazel + scala is there today.
We welcome contributors, and would love contributions to improve any of these aspects. But frankly, if you are not here for reproducibility, sbt and/or bloop will probably be your better bet.

@jvican would you be interested in making a .md or google doc on a design proposal? It is hard for me to see what the design space looks like when both bazel and and some other tool want to own a cache. How do you invalidate that cache? Once we have a design, we can see if it would seems like a PR to consider.
I think we should be pushing towards buildfarm and remote caching, and I wonder how bloop fits into that. Since bazel is parallel and incremental at the target level, it seems the main thing we get is the ability to make giant targets, but that is an anti-pattern in bazel.
I still lean towards the highest leverage thing being a tool to split BUILD files up automatically (preserving existing public targets but minimizing the internal DAG).
Seems like a fun problem. Does no one really want to solve it!? Sculpt + some dag minimization seems like fun.

@jvican would you be interested in making a .md or google doc on a design proposal? It is hard for me to see what the design space looks like when both bazel and and some other tool want to own a cache. How do you invalidate that cache? Once we have a design, we can see if it would seems like a PR to consider.

Sure thing, would be happy to do that. But as that would take me some time, I would prefer to wait a bit to see more responses in the thread. It's also important that we set expectations and that we discuss tradeoffs, as that will shape much of the design.

I think we should be pushing towards buildfarm and remote caching, and I wonder how bloop fits into that. Since bazel is parallel and incremental at the target level, it seems the main thing we get is the ability to make giant targets, but that is an anti-pattern in bazel.

Remote compilation is not at odds with bloop. You can disable incremental compilation in the CI, while still get all the performance benefits that bloop provides over scalac even for small targets. Build pipelining (the ability to run compilation of projects before the compilations of their dependencies are finished) will give you a significant compilation speedup if you have a parallel build graph.
In fact, given how important remote caching is for bazel, I would strongly suggest that you should not enable incremental compilation in remote servers, you should only use it for local development and local caches.
Despite the better compile times you would get instead of using scalac by default, Bloop/zinc shine on speeding up incremental workflows. For example, I'm currently tweaking bloop to provide sub-second incremental compiler executions -- with the goal of providing users with instant output from their test suite or running program for each change they make in a target(s). This is an example of why a bloop integration would mean providing a more productive setting for your users.
As I see it, these would be my three goals with the Bloop integration I propose:

Making sure that bazel is still reliable and reproducible by not contaminating the remote caches.
Increasing the compilation throughput in your bazel remote servers.
Providing the fastest edit-compile-test workflow for local development.

We currently want to use the the remote execution and caching for local development. How does that interact with bloop?
We don't want to have to locally build the whole repo, especially for users that pop into a big repo rarely. We want them to fetch the cached artifacts, and not spend 20 minutes rebuilding the first go.

My concern with Bloop is that it seems to be solving a very similar problem as Bazel workers. It also introduces a dependency on running the Bloop server, which seems incompatible with remote execution.
From: https://scalacenter.github.io/bloop/docs/bloop-basics/

Bloop is an build-tool-agnostic build server that makes hot compilers accessible to all your toolchain — IDEs, build tools and terminals get peak performance at all times.

This is largely what Bazel workers do.
If Bloop provides a both build server and a better API for Zinc, it would be nice to see those concerns separated. That way you could write a worker using the stable and improved API for Zinc.

I would like to see us write a tool to break large targets up using https://github.com/lightbend/scala-sculpt so that you could automatically split large targets. This might not be very hard, but we haven't started the project yet. Would someone on your end be interested in contributing that?

We have bounced around ideas about doing something like this at Lucid. We have almost 1m lines of Scala to break down into smaller dependencies. A tool to take Scala code and break it into smaller projects would be great. However, I'm somewhat concerned about the number of circular dependencies we would find in our existing codebase inside of our projects.
We would like to do so something like this in the long term, but our Scala build team is a bit short staffed at the moment. We're currently hiring more people to work on this, but hiring + getting familiar with the domain takes time.

Our CIs are very fast (several minutes for each pull), remote caching hit rate is pretty good generally above 80% and very often above 90%.

Remote caching is fantastic. It is one of my favorite parts of Bazel so far. Internally, using Bazel + annex, CI populates an http cache in S3. We have proxies that proxy that s3 cache in a closet in the office and on developer's laptops. We see pretty good cache rates in dev. Here's an example of a local build using the closet cache on a commit that was previously built in CI. It takes several minutes without caching.
Target //user-service:user-service up-to-date:
  bazel-bin/user-service/user-service.jar
  bazel-bin/user-service/user-service-bin
INFO: Elapsed time: 6.145s, Critical Path: 3.17s
INFO: 625 processes: 625 remote cache hit.
INFO: Build completed successfully, 956 total actions

We run in dev and CI with strict_deps and unused_deps set to cause build errors. We added support to ibazel for running buildozer commands based on bazel output. This has been merged back upstream: https://github.com/bazelbuild/bazel-watcher/pull/134 This + Annex's buildozer messages makes maintaining a minimal classpath quite ergonomic. ibazel automatically presents a [y/N] prompt to execute a buildozer command when you stop using a direct dependency or start directly using a transitive dependency.
This does rely on a fork of bazel-deps, which adds transitivity: deps support for bazel-deps. https://github.com/lucidsoftware/bazel-deps/tree/java-import-external Sadly, we haven't had a chance to update our work and start the process of getting things merged back upstream. It's on our backlog, though.

I am not opposed to using zinc if it can pass our reproducibility tests and doesn't cause a migration pain for existing production users. I am skeptical our users would be better off investing in zinc vs building a tool to split large targets automatically, which could then leverage a fully distributed cache and remote execution, something we preclude as soon as non-reproducibility enters the picture.

In annex there is a goal of making incremental compilation configurable. We have several engineers internally who value correctness over speed, and would like to disable  incremental compilation. We have a bug internally in our tools/bazel, which always enables incremental compilation in dev, so we sadly cannot comment on how deterministic annex is with incremental compilation disabled. We should know more soon, though.
I do believe supporting incremental compilation in some fashion will be necessary if we want Bazel + Scala to be widely adopted. I imagine a typical migration path will be sbt -> Bazel with large projects + optional incremental compilation -> incrementally decrease the size of build targets with the sculpt based tool and some elbow crease -> Bazel with small projects and no incremental compilation (or at least opt in incremental compilation).
It's really hard to migrate a large codebase to Bazel + break down build targets at the same time. Until build targets are small enough to be fast, you either support sbt + Bazel or developer productivity suffers. Incremental compilation provides a nice intermediate step. Even if it doesn't play nicely with Bazel.
I know that the Swift folks are at least considering incremental compilation as well: https://github.com/bazelbuild/rules_swift/issues/48

So if you have issues with the current implementation, I propose you file tickets about the limitations and a compiler engineer could have a look at them or provide pointers to fix them.

We started with https://github.com/scalacenter/classpath-shrinker for strict/unused dependency analysis. However, we found Zinc's analysis to be better. Considering annex is also compiling with Zinc, this works well. A long term compiler based solution to this issue would be great.  However, it's hard to give up the better dependency analysis in the short term.

It's a real shame we can't agree on vision given everyone has people working on related things...
Maybe it is good, but we are certainly doing the competition thing in scala build tooling and very little cooperation: sbt/zinc, bloop, mill, fury, pants, bazel (rules_scala, annex)... all trying to build some next generation tooling.
Maybe in a few years a clear winner will shake out, but in the mean time we will have a ton of losers and a huge amount of effort spent on the losing forks.

Thinking on this more, I can see a few things we lack agreement:

simplified build definitions (bazel, fury?) vs full turing complete programming environments (sbt, pants, mill?)
does correctness + caching ultimately yield better performance that approximate, but incorrect caches (e.g. scalac caching jars by name, not content, or the cache bugs with zinc/sbt).
implicitness vs explicitness (bazel requires explicit dependency DAG, vs sbt finding implicit dependencies inside a module).

So maybe it is fine that we are exploring the space...
But here, for this repo, I think the most relevant question is: do we buy bazel's argument that correctness leads to better performance via caching and remote execution. We keep having this debate about using less correct (e.g. known non-reproducible builds) yet arguing this can lead to better performance. I think this can be true for a small enough build, but not for a build big enough to be relevant industrially.
If you do have such a small build, I think using sbt is not a bad option (or maybe try mill, fury etc...)
If you don't care about reproducibility in a large repo, I'd suggest pants. It's actually really usable and uses zinc caching. I didn't like all the weird bugs we hit with refactoring code due to buggy caching, but others seem to not mind.
My interest in bazel is to explore the space of reproducible (in other words, correct) builds, and I believe the bazel hype: if the builds are reproducible, caching makes them fast.
Of course, this requires small enough targets so each build can be O(seconds). We generally do that at Stripe, although some folks do get lazy and make mega targets with globs which take 1 minute to compile. I think this is a problem to tackle, but the way I'd like to tackle it is with tooling to split the targets.
@ittaiz can weigh in, but I hope the vision of these rules (and I hope Stripe's approach to build) is to keep the focus on reproducibility. annex, mill, pants, etc... exist for people who don't want to work towards the reproducibility-enables-speed vision (or the correctness-is-its-own-reward vision if you don't care about the speed part of the argument).
We keep hearing this "use non-reproducible builds for local dev" but I don't see how to integrate that with bazel and bazel's remote execution and cache easily. But, in any case, I think annex is very open to this idea, so maybe you can show the way and Lucid would be interested in sharing their results and we can all learn from it.

First array of comments answering @jjudd's feedback.

My concern with Bloop is that it seems to be solving a very similar problem as Bazel workers. It also introduces a dependency on running the Bloop server, which seems incompatible with remote execution.

I don't understand your concern. In the early days, I did a bloop integration in rules_scala_annex that was never upstreamed where Bloop was itself a remote worker. You can use bloop as a remote worker in bazel.

If Bloop provides a both build server and a better API for Zinc, it would be nice to see those concerns separated. That way you could write a worker using the stable and improved API for Zinc.

You can already do so. However, let me reiterate that as a maintainer and active developer of both I want bloop to become the standard way of doing compilation in build tools because it has a much better design and allows the core team (me and other compiler engineers) to fix compilation correctness and performance issues at once for all the clients.

so we sadly cannot comment on how deterministic annex is with incremental compilation disabled

Both rules_scala and annex should be as deterministic as the underlying compiler is when incremental compilation is disabled. 2.13.0-M5 is the first release that makes some promises with regards to deterministic compilation.

I do believe supporting incremental compilation in some fashion will be necessary if we want Bazel + Scala to be widely adopted. I imagine a typical migration path will be sbt -> Bazel with large projects + optional incremental compilation -> incrementally decrease the size of build targets with the sculpt based tool and some elbow crease -> Bazel with small projects and no incremental compilation (or at least opt in incremental compilation).

If you're interested in the sbt -> bazel migration problem, I've left some thoughts in https://github.com/stripe/sbt-bazel/issues/4. The ticket didn't get much attention by the Stripe team, but I think it's an interesting possibility if you're time constrained and value correctness of the migration over creating your own solution.

We started with scalacenter/classpath-shrinker for strict/unused dependency analysis. However, we found Zinc's analysis to be better.

How is it better? Have you used the last improvements that went into unused dependency analysis to detect macros?

Considering annex is also compiling with Zinc, this works well. A long term compiler based solution to this issue would be great. However, it's hard to give up the better dependency analysis in the short term.

My recommendation still stands there. Though I see the value in using it in the short term, this is probably not the best solution long-term. If something breaks or we decide not to follow some dependencies in Zinc internals, you'll be affected by that.

We keep hearing this "use non-reproducible builds for local dev" but I don't see how to integrate that with bazel and bazel's remote execution and cache easily. But, in any case, I think annex is very open to this idea, so maybe you can show the way and Lucid would be interested in sharing their results and we can all learn from it.

If you keep yourself open to the idea of merging a bloop integration (which again would not necessarily add only incremental compilation, but better compilation performance over the board with no risk to correctness), I can draft a design document and we can discuss more about how I see this playing with remote execution and caching. I do have solutions to some of the concerns you've raised.
That being said, I would be happy to work with the Lucid guys too (which have more experience in the bazel-zinc field). But, ultimately, I think we should not be splitting up our efforts. It does not motivate me at all to create a bloop integration for a small part of the Bazel community. If you cannot agree internally about what the best approach is (or allow the integrations to live in the main repo), then I prefer to invest my time in other build tools where my work is more impactful.

Jorge, I use this tooling and generally have a few second cycle.
I’m not speculating. I actually use this all day. You are making a non-existence claim, but I actually use this system to do my work in a medium size code base (several repos linked as bazel external source dependencies totaling about 500k lines of code).
If you worked in such a repo, that was fully reproducible, and well written targets took 1-5 seconds, do you really think you would want to trade reproducibility and have an unclear impact on your caching to get the same order of performance? Also you have to do a bunch of work to test and deploy that to 60 engineers when the current system seems to be working well. I don’t think you have actually compared to a real system doing as I propose, yet you also don’t trust our experience report that small targets with caching has done very well (our second largest repo completes the CI before you can finish making the PR, the largest has some large docker images that prevent that).
Basically: lots of people are trying the wager that non-reproducibility is a trade to make (actually I think virtually everyone is). Why must every tool do this? Why isn’t pants or annex the right tool for people who don’t care about reproducibility?

I don’t think you have actually compared to a real system doing as I propose, yet you also don’t trust our experience report that small targets with caching has done very well

I don't doubt your experience report at all! Please don't get me wrong, I have not worked in such a system and I wouldn't dare to say I understand how working with bazel in big projects is like because I do not have access to these repositories.
However, I'll say that the fact that @jjudd et al have experimented with Zinc proves there's interest in a faster workflow than 5 seconds per compilation per target. I've also talked to people working on big bazel projects and from what I've learned they miss incremental compilation. In their experience, making changes to several targets becomes slower.
Ultimately the numbers depend on the granularity of your targets. Maybe at Stripe you're incredibly disciplined at splitting up the targets and containing the large compilation times. But I don't think every user of Bazel will because (1) it is time consuming, (2) changes to the dependencies of a target can often create merge conflicts with upstream (which slow down development) and (3) correctly organizing a build is a challenging task.
I'll let the contributors of annex address this point since they are more qualified than I am to judge the status quo.

Why must every tool do this? Why isn’t pants or annex the right tool for people who don’t care about reproducibility?

Because I believe a build tool is all about its ecosystem and community, more than about different technical properties. Pants is a great build tool and it has a different community of contributors and rules for languages. Annex competes with rules_scala for the same community/ecosystem, and that will cause confusion in Bazel users.

I'll dive into this conversation later this week and try to add my 2c.
I love the very fact this discussion is happening, I think it's important  and I hope that through it we can better understand how we can collaborate.
I might be wrong but I'd like to suggest taking a bit more time to respond. It feels both of you are, to some extent, replying very fast and that can get us in a loop. I hope I'm not offending anyone by saying so.
In general, and again without diving into the details, I agree with @johnynek about favoring reproducibility over (local) speed. We have a setup where devs use the exact same cache as CI and given we have >1M LOC of scala over a few tens of repos if people start using "clean" from time to time it can kill their productivity. This is not to mean we can't collaborate (disable incremental compilation?) and in fact I think we need to see how we can collaborate.

If anyone is interested in helping make a tool to split large targets I started this PR:
https://github.com/lightbend/scala-sculpt/pull/61

If scala compiled as fast as java

That is what it comes down to. If Google used Scala, Swift, Haskell, or Rust I think opinions would differ.

I disagree that sharing state is the point of the worker, the point of the worker is to share the jitting and classloading, but still to have a stateless pure function for build.

"Pure" in the sense that same input gives the same output, yes. But have a risk of being impure in that code may improperly keep state (JIT, statics).

If you don't care about reproducibility in a large repo, I'd suggest pants.

Pants is indeed a good example of when the worldbuilders had Scala as a primary language.

We should probably document more this approach: when the deps are set up, iterate without strict-deps warn, when they are not, use that mode until you don't have warnings and it will tell you what targets to add.

FYI, by default rules_scala_annex runs in a mode where indirect and unused checks are part of a non-default output group. Not fundamentally different; just a little experiment.

I don't see how to integrate that with bazel and bazel's remote execution and cache easily.

Remote execution doesn't work with workers.

But frankly, if you are not here for reproducibility

We keep hearing this "use non-reproducible builds for local dev"

I'm here for reproducibility. Same input, same output. The place where you and I differ @johnynek is where/how reproducibility is enforced/tested.
Bazel optionally allows users to sandbox actions. This enormously magnifies I/O latency (or at least used to). So it can be turned off or on. Users who turn it off are not asking for unreproducibility. They are opting out of a reproducibility mechanism due to performance overhead. Workers allow for state sharing. Users who use workers are not asking for unreproducibility. They are opting out of a reproducibility enforcement mechanism due to performance overhead.

Re your last point of dependency detection- I’d really wish this was a
module part of zinc we could depend on...

To me, the most compelling reason to use zinc is to get the test framework support.

I hope the vision of these rules (and I hope Stripe's approach to build) is to keep the focus on reproducibility. annex, mill, pants, etc... exist for people who don't want to work towards the reproducibility-enables-speed vision

How does that stand with https://github.com/bazelbuild/rules_scala/issues/125#issuecomment-279471684 ?

I think we should stay focused on support for big code bases, large teams, and caching/remote execution efficiency.

To be clear....Zinc can be used without stateful incrementality. In fact, non-stateful is default of rules_scala_annex.
The point of the Zinc project is to understand the dependencies of inputs and outputs while compiling Scala. If you have that, then you can do a number of things, including incremental builds but also including indirect deps detection, unused deps detection, and test dectection.
Using in stateful and non-stateful modes is as easy as whether you provide it the analysis from last time. That's it.

Wow great discussion. To clarify the scenarios I'm talking about with Developers, caching is not relevant. The scenario is I have a freshly completed build and I'm not writing code to develop a new feature. As simplistic, but realistic, scenario is I make changes to one file to fix a bug in a previous commit where I had a typo in a column name making for a slick table. This changes Node A in the DAG. I then write an integration test to verify it in node T. In order to compile T  bazel has to build Node A, B, C, D, E and then T.
This complete DAG build for this 1 line change can take over 2 minutes in our situation. This makes iteration time for developers adding functionality to the product huge compared to SBT, which takes about 2-3 seconds to compile.
Breaking up our build into smaller nodes is part of our long term plan, though restructuring our build to the nano level so that bazel works well with it is not ideal, but smaller targets than we have is certainly possible. However; there is no guarantee that smaller targets makes the DAG wider, and not just deeper, thus not improving the build time at all.

It took me a minute to parse that. "To clarify the scenarios I'm talking about with Developers, distributed caching is not relevant".
And that's true. The compile/change one line/compile workflow isn't helped at all by distributed caching. (Distributed execution, on the other hand, can potentially help, depending on the number of targets to be built, the network latency, and size of the cluster.)

I think it is a bit more subtle: when you change one line, you don't want to invalidate all your other cache. Next, due to non-reproducibility, users get trained to clean to fix issues that happen (this happened when I worked with pants and it happens still to me periodically with sbt). By contrast I recall zero times in the past year or so at Stripe that a bazel clean fixed a build issue (bazel clean --expunge has fixed things since there seem to be some lurking thread bugs with downloads sometimes failing and blocking build, the last scala related bazel cleans were related to macro non-reproducibility and we think we have gotten rid of all the instances of that we know of at Stripe).
Training users to clean hammers performance in a big repo unless, again, your cache story is really good: clean is only touching local state (which it does), and the remote cache is well populated. However, who populates the remote cache if the users are building non-reproducibly? You can imagine some side system you deploy to have workers populating a cache, or you can use build-farm (which is not a side system). Then, again, the question comes which things you submit to build-farm vs not.
If you imagine building some system to switch to local non-reproducible builds, you also have to that in a way that doesn't change cache-keys which, again, would wipe out the cache you depend on for inputs into the target that you are iterating on.
I want to comment on this because it is not simple or clear, how to either implement this or teach users a model for how to work with this.
My conjecture is that working on a tool that automatically factors large targets, but then just use normal fully reproducible builds using caching/buildfarm, is easier and more ergonomic, and in the end, offers great performance.
We could make it even more ergonomic by making a bazel proposal and working on the code to build an API to express internal dags of large targets. @pauldraper had a nice proposal on that in the past, but I don't think anyone has yet had time to implement his proposal as a sketch. I think waiting for bazel to solve this may be a long wait since the team responsible for bazel has not been highly prioritizing solving this problem.

I think it is a bit more subtle: when you change one line, you don't want to invalidate all your other cache.

Yes, that is a good way of putting it. On a practical level, I think it is very hard to take a typical Scala code base and create enough targets so that the Bazel cache invalidation is even close to the Zinc cache invalidation. I.e. for @bpatters to enjoy the 2-3s incremental compilation he had with sbt.
This is really a lot "well MY code base". It would be interesting to write a Bazel build for a well-structured, large-ish OSS Scala code base and test this out. Like https://github.com/playframework/playframework .

@pauldraper had a nice proposal on that in the past, but I don't think anyone has yet had time to implement his proposal as a sketch.

For reference: https://docs.google.com/document/d/16iogGwUlISoN2WLha2TAaUdpYCjRiVQ2sRQ7--INxkg It has a lot of interesting uses. E.g. testing with a automatically determined number of shards. Or quickly scanning file dependencies of TypeScript and automatically creating a granular action graph. It's really useful if you have a short-ish action that uses file contents to inform the structure or number of of long-ish actions. I'm skeptical it could work for specifically for Scala compilation, but it'd be great if it could.

I think waiting for bazel to solve this may be a long wait since the team responsible for bazel has not been highly prioritizing solving this problem.

Yeah. Bazel would solve this and a dozen other things tomorrow if C/C++ (which make extensive use of dependency pruning and other advanced features) were in Skylark. Ah well.

By contrast I recall zero times in the past year or so at Stripe that a bazel clean fixed a build issue

I've had several times I needed to bazel clean, related to bugs in workers (i.e. one of the ways to remove reproducibility protections) -- which, admittedly, I was at the time actively developing and so it was "my fault".

However, who populates the remote cache if the users are building non-reproducibly?

CI or build farm. Lucid upload from CI. Google upload from build farm. Uploading from dev environments is dangerous business; https://github.com/bazelbuild/bazel/issues/3360 . "Ulf: As such, using a user-writable shared cache is something that we'll warn people against."

Maybe I can make my point more quantitatively.
Pclean: the probability you run a clean before each build.
Tclean: the time to run a clean build of a target.
Ttarget: the time to run a scalac/non-incremental build of a given target
Tzinc: the time to run a zinc build for a given target

What we are debating is if the following predicate is true, in which case imperfect caching via zinc is a win:
Pclean * Tclean + (1 - Pclean) Tzinc < Ttarget

If this is true, a noisy cache that you sometimes clean is still a win on average. The condition for this is:
Pclean < (Ttarget - Tzinc) / (Tclean - Tzinc)

We can readily evaluate these: at Stripe, most Ttarget values are something like 1-10sec, but we have metrics on it. The worst case are some very fat ill-maintained targets that people keep throwing junk in that have gotten up to 200sec. At Stripe in our largest repo, Tclean ~ 20 min (1200sec). Let’s say Tzinc ~ 1s.
So, plugging this in we need:
Pclean < (10 sec - 1sec) / (1200sec - 1sec) = 0.75%

This is very rare:  you need to keep your confidence for a period of 133 builds.
That’s assuming 10sec average Ttarget. If you assume Tzinc = 0, you need Pclean < Ttarget/Tclean. Since in a monorepo, Tclean gets very large Pclean needs to be going to zero for this to be a win.
If, with smaller targets you can get your build time down to 2 seconds, that expands up to cleaning only once every 1199 builds ((2-1)/(1200-1)). Furthermore, an expert scala user may have an intuition for when to clean, but once clean is common in your workflow, neophytes are running clean every time anything looks remotely fishy, so they get particularly hammered and see even worse performance than experts who can more accurately guess which is a likely to be an actual build error vs an error in the user code. I have seen this play out both at Stripe and Twitter. We actively tell people to file a JIRA if bazel clean fixes anything for you since it never should. We try to talk people out of ever thinking they should even try that. At twitter, with zinc-enabled-pants we had a script called wash-pants which nuked all possible state it could find and rebuilt the repo: we had to tool up cache cleaning because it was frequent enough.
Now, some people may be in a world where Ttarget = Tclean, e.g. they are effectively buidling 1 target. Hopefully it is clear why bazel is bad here: Pclean approaches 1 when the approximate rebuild solution is still better. It sounds like @bpatters is in this regime, so sbt probably is a lot better.
My case is this: we should be optimizing for the case where Tclean is huge (say 20-30 minutes), if you don't expect to be in this regime, you should probably use sbt. Next, the targets need to be small (10 seconds or less), again, if you can't dedicate the time to factor your build, bazel is going to be a bad time now.
If we redirect the energy of this thread to a tool to factor the build we could get some real advances here (e.g. take https://github.com/lightbend/scala-sculpt/pull/61 and make an optional output of the rules a list of targets to factor itself into N smaller targets).
I want to help make it easier to adopt bazel. I understand that big targets are painful to convert to rules_scala, but I think we need to directly solve that in tooling and not try to paper over the issue by encouraging large targets enabled by a non-reproducible compiler.
Lastly, even if you use a non-reproducible compiler locally, no one on the thread is arguing to make CI non-reproducible. With large targets even with caching, CI will be slow because you will tend to see invalidations.
To me, the only solution here is better tooling to split large targets. After we have that, we can consider pushing again to expose bazel's support for dynamic build graphs to allow bazel to automatically split targets based on discovered dependencies so it can manage the caching (i.e. @pauldraper 's proposal).
Personally, I'm not interested in yet-another-non-reproducible-build-tool. I think we have plenty of those. I want to help advance the state of the art and move us to a world where we expect reproducibility and we leverage it for amazing caching. I want small targets to enable amazing parallelism in my build. I hope people who are excited about this vision help bazel and rules_scala get there.

In all of this discussion, I still believe that togglable and defaulted off incremental compilation is best path forward for Bazel + Scala. It provides a migration path for large repos that currently always use Zinc while preserving reproducibility for those whom it is a requirement.
At Lucid our sbt build currently has similar reproducibility issues like the ones you mentiond with Pants. We have had a few cutely named commands over the years that are equivalent to wash-pants. New engineers have a hard time telling apart Zinc reproducibility problems from actual compile errors, so they tend to clean too frequently. We have a monorepo, so clean builds are slow without Bazel's caching.
That being said, if a productive and fast dev workflow in Bazel required us to, upfront, refactor all of our Scala codebase follow the 1:1:1 rule, then we would have likely never successfully migrated to Bazel. Maintaining two build systems for months at a time is challenging and expensive.
We are currently breaking up our TypeScript targets into smaller build targets, and it is a large amount of work. It is going to take a similar amount of effort to refactor our Scala code. It is just too easy to create circular dependencies within build targets. A tool would majorly help with this, but I doubt it would decrease the effort enough to make blocking the migration worth it.
Togglable incremental compilation enables a large Zinc compiled Scala codebase to migrate large build targets to Bazel and maintain their current developer speed and reproducibility issues. Then, over time, they can migrate targets to follow the 1:1:1 rule and stop using incremental compilation. In the meantime, users who value reproducibility over speed can disable incremental compilation using a local and gitignored .bazelrc.
Over time, Bazel may improve the power of Starlark as they migrate native rules, thus decreasing the need for incremental compilation. Perhaps Zinc will continue to improve and the probability of reproducibility issues will decrease, thus making incremental compilation more attractive. Perhaps we write a tool that is so powerful it is capable of migrating the entire Lucid codebase to the 1:1:1 rule with minimal effort.
In the meantime, togglable incremental compilation seems like a compromise that defaults to reproducibility while enabling migration of large Zinc compiled Scala codebases.
I do not think this effort is in opposition to breaking down builds into smaller targets using Starlark/Bazel or with a tool. I view it as complementary. I see an opportunity to grow the Bazel + Scala community if we can provide an easy migration to Bazel by converting sbt targets to rules_scala targets using optional incremental compilation and https://github.com/stripe/sbt-bazel. This requires little refactoring at the start. Then codebases can use our to-be-built tool and some elbow grease to incrementally migrate to the 1:1:1 rule.
Overall, I see Bazel as a potentially very attractive build tool for Scala codebases of a certain size. I would like to find a path forward for rules_scala + Annex in such a way that meets the needs of everyone and positions Bazel + Scala for success as a next generation Scala build tool. I believe togglable incremental compilation is key to making that happen.

Aside: in Annex incremental compilation is defaulted to off. It is toggled using a worker_extra_flag. As far as I understand, worker flags are not part of the cache key. Thus, even when using incremental compilation, you still get cache hits from CI. The disadvantage of this is that it enables or disables incremental compilation for all builds. It would be really neat to somehow toggle incremental compilation at a target level, but I am not sure how to do that off the top of my head while preserving cache hits. Perhaps we should use some kind of force_incremental_compilation_off flag per target that modifies the command line, but that is ok because the command line will be the same in CI and dev.

If you have distributed caching (correctly populated, reproducible artifacts, no https://github.com/bazelbuild/bazel/issues/3360 or similar), then the actual time to build the repo is Tclean' = Ttarget + (Tclean - Ttarget) * Rdownload, where Rdownload is the ratio between cache download time and compile time.
Say Rdownload = 0.1, i.e. it takes one tenth the time to download a jar as to compile it from source. (I could see it been far lower depending on your cache and network setup.)
Pclean < (10 sec - 1sec) / (10sec + (1200sec - 10sec) / 10 - 1sec) = 7%

So you need to edit, compile, edit, compile, edit, compile, etc. 14 times before cleaning.

True, though I'm fine with every one line change causing an extra 60s of CI time. I'm less fine with everu one line change typically causing an 60s of developer time.

Personally, I'm not interested in yet-another-non-reproducible-build-tool.

Personally, I am also not interested in such a tool.
@johnynek you hold as a fundamental truth that "Zinc is non-reproducible*". That is, that given the same inputs, Zinc will give different outputs [because of other, internal state]. That principle underlies every argument, every point, every discussion, and every formula.
*In a 99.95% sense...as you said, even without Zinc, the Scala macros can be non-reproducible. Even Bazel itself can be non-reproducible https://github.com/bazelbuild/bazel/issues/1162 https://github.com/bazelbuild/bazel/issues/3360 https://github.com/bazelbuild/bazel/issues/4770 . Again, in the "except for this one rare bug" sense.
And yet, though it forms the underpinnings of every discussion, not once has a non-reproducible case been volunteered.
I believe Zinc is reproducible. And I use it, with Bazel. And other people want to use it, with Bazel. Even 2016 Oscar wanted to use it, with Bazel.
FWIW, even Google internally (not in the OSS version) uses a persistent Java compiler. https://groups.google.com/d/msg/bazel-discuss/3iUy5jxS3S0/smqhBOa0AQAJ

we've done (hundreds of?) thousands of builds with the incremental Java compiler, and we haven't encountered any correctness issues recently.

Google, the owner of the greatest Tclean in the world uses a stateful compiler for the language it really cares about.

I don't think it's possible for me to convince you that Zinc is reproducibly correct.
Given that, does it make sense for the "Zinc is non-reproducible" users to rules_scala or rules_scala_annex, and the "Zinc is reproducible users" to use rules_scala_annex?

I'll echo @pauldraper here a bit. Incremental compilation provides benefits not only as a migration path, but once Zinc reaches a certain threshold of reproducibility, it becomes an attractive end state. I'm not sure how much effort is required to reach that threshold, and I'm not suggesting it should be the end state for everyone. However, it is nice for those who want to use it. The choice/toggle seems essential here.

I hope everyone has had a chance to record their thoughts here.
I don't know if any minds have been changed, but it is good we have laid out the arguments. I am happy to link to annex or any fork of this repo for users who want to use zinc's caching, but I'm not going to add stateful zinc caching to these rules (while I am involved with the project).
If @ittaiz disagrees and the users want to go in another direction I'm happy to get out of the way and will wish you all the best. In the mean time, I'm more interested in tooling to split large targets, perhaps in sbt-bazel https://github.com/stripe/sbt-bazel/issues/9
I'm open to a PR to use zinc non-statefully similarly to how we create the workers here in order to have a better compiler API, support for the test API, assuming this can be done with zero (or very minimal) breakage to existing users and that we have no evidence or belief that it is less reproducible that allocating a new compiler on each run of the worker as we do now.

And yet, though it forms the underpinnings of every discussion, not once has a non-reproducible case been volunteered.

@pauldraper : We should take the discussion elsewhere, but: I can point you to a few. Ping me on Twitter.

I don't think people actually try to keep track of this, but I just saw this:
[info]   IncompatibleClassChangeError was thrown during property evaluation.
[info]     Message: Found class cats.data.ContT, but interface was expected
[info]     Occurred when passed generated values (
[info]       arg0 = FromFn(AndThen$83029018),
[info]       arg1 = org.scalacheck.GenArities$$Lambda$33539/163349956@7ce03c1,
[info]       arg2 = org.scalacheck.GenArities$$Lambda$33539/163349956@315ab84a
[info]     )

when changing a trait to an abstract class with sbt 1.1.6. I see bugs like this periodically. I think no one using sbt assumes that using clean is a bug, rather than a normal part of using the tool.
It would be interesting to instrument sbt so that it logs every time a user needs gets a compilation error that is fixed with a clean compile. It would be cool if it could suck up all state and ship the before and after state on an opt in basis. Maybe then zinc could collect enough data to fix these bugs.
Alternatively, they could build a test harness that applies diffs from github and then sees if each clean compile matches the incremental compile. If zinc built such a test harness and had evidence that things worked, I'd be thrilled to reevaluate. I don't think this has so far been a priority for the zinc/sbt team.

@stuhood I will.

I think no one using sbt assumes that using clean is a bug, rather than a normal part of using the tool.

That could be true. And I've seen an sbt bug, but it turned out to be due to bad ivy/maven dep caching instead. With sbt, cbt, mill, scala-maven-plugin, Pants, etc. using zinc, I'd hope that someone would fill an issue.
FWIW, there are currently no open reproducibility issues with zinc.
And Bazel has several reproducibility issues open for  months (particularly on Mac). Yet overall, it still has "very high" (and very useful) reproducibility. And both projects look to fix any issues that come up.

I’m part of the problem I guess because I didn’t report this one.
It’s probably repeatable by converting traits to abstract class in the cats build.

when changing a trait to an abstract class with sbt 1.1.6. I see bugs like this periodically. I think no one using sbt assumes that using clean is a bug, rather than a normal part of using the tool.

I don't think this is true. clean is only used exceptionally. I use myself sbt more than 8 hours a day and the errors with regards to incremental compilation don't happen, by any means, "periodically". I think this is an exaggeration. Also note that you're not using the last Zinc and sbt versions, which has critical fixes.

It would be interesting to instrument sbt so that it logs every time a user needs gets a compilation error that is fixed with a clean compile. It would be cool if it could suck up all state and ship the before and after state on an opt in basis. Maybe then zinc could collect enough data to fix these bugs.

https://github.com/sbt/zinc/pull/585
The goal is to make bug reporting in Zinc effortless, and that's already merged in master.

I don't think this has so far been a priority for the zinc/sbt team.

My priority has been to solve all the issues with regards to correctness possible. But, generally, no, getting a fully reproducible zinc has not been the priority because the users of tools like Bazel and Pants are negligible compared to the majority of users we support.

Alternatively, they could build a test harness that applies diffs from github and then sees if each clean compile matches the incremental compile. If zinc built such a test harness and had evidence that things worked, I'd be thrilled to reevaluate.

I think I already registered this idea in the Zinc issue tracker somewhere. It's not possible to carry it out until we have a fully reproducible compiler, which will happen with 2.12.7 or 2.13.0.

I'm unsubscribing from this thread, if somebody wants to talk to me with regards to Zinc, drop by the sbt/zinc gitter channel or talk to me in private.


Vertical rather than horizontal tabs

The Windows front-end has always had horizontal tabs for the main tabs (Story, Index, etc.) but the OSX and Linux front-ends have vertical tabs, with the actual tabs on the right.
Investigate making the Windows front-end have vertical tabs too, possibly optionally.

Done, finished by commit ce61ae4030fb529baf4fcbad5c20b2896c9807a0


Support let-defined methods in Minitest rewriter

Input
→ View on sorbet.run
# typed: true

class Minitest::Spec
  module DSL
    T::Sig::WithoutRuntime.sig { params(name: Symbol, block: T.proc.returns(T.anything)).void }
    def let(name, &block)
      T.unsafe(self).define_method(name) do
        @_memoized ||= {}
        @_memoized.fetch(name) { |k| @_memoized[k] = instance_eval(&block) }
      end
    end
  end

  extend DSL
end

class MyTest < Minitest::Spec
  let(:my_method) { 0 }
  # From a typing perspective, ^ this is as if you did this:
  #
  #     def my_method; 0; end
  #
  # I think that Sorbet would also trivially support writing
  # a `sig` for this in `# typed: strict` test files,
  # like how it works for `attr_reader` methods.

  describe 'example' do
    it 'calls method defined with let' do
      my_method
    end
  end
end

Observed output
editor.rb:29: Method my_method does not exist on MyTest::<describe 'example'> https://srb.help/7003
    29 |      my_method
              ^^^^^^^^^
  Did you mean method? Use -a to autocorrect
    editor.rb:29: Replace with method
    29 |      my_method
              ^^^^^^^^^
    https://github.com/sorbet/sorbet/tree/master/rbi/core/kernel.rbi#L559: Defined here
     559 |  def method(arg0); end
            ^^^^^^^^^^^^^^^^
Errors: 1

Expected behavior

Sorbet should be able to know that the let method defines a method with the given name.

I believe that RSpec has the same concept of let-defined methods, with the same API. It is slightly worrying, because there wouldn't be an easy way for Sorbet to detect whether the let call comes from Minitest/RSpec or from some other DSL. If it becomes a problem, we might be able to limit the rewriter to only make this transformation inside describe blocks? I'm not sure how limiting that is in practice or whether it's needed in the first place (or for that matter, whether it's actually required because you're not allowed to use let outside of describe blocks).

This is done, though not exactly in the form presented in the ticket. For the time being, it's limited to only those let calls which are inside describe blocks, to avoid accidentally stepping on any other DSL called let which doesn't behave the same way that minitest/RSpec's does.


Fix the Pagination cog

When using pagination in the Reddit cog, clicking the LAST_EMOJI would
raise an exception like this:
TypeError: unsupported operand type(s) for -: 'list' and 'int'
This was resolved by taking the subtraction out of the len() function.
Relevant Issues
Closes #352
Description

Reasoning

Screenshots

Additional Details

Did you:

[ x ] Join the Python Discord Community?
[ x ] If dependencies have been added or updated, run pipenv lock?
[ x ] Lint your code (pipenv run lint)?
[ x ] Set the PR to allow edits from contributors?

@sco1 Migrated, waiting for your approval

@sco1 Is everything ready or there are still changes needed?


Colour functions

Added hex value to the RGB objects, I'm not sure if this is the normal way of doing pull requests, I think i was probably supposed to make a new branch instead of using the same one I used last time

I'm not sure. It should be fine though


[BUG] Fix compatibility issue when WeakRef is not supported

ember-ref-bucket generates an exception on browsers that don't support WeakRef.
Tests were not passing on Safari 13

@cyril-sf thank you for finding and fix! v5.0.6 published


Edit Migration Toasts

Purpose
Stop toasting on linode_migrate events with a status of started because they have a percentage, meaning that a toast would appear for each percentage update.

Coverage increased (+0.007%) to 26.379% when pulling 5d91dba3a1971c7a4ac29d303ea369025c1279bf on martinmckenna:remove-migrations-toasts into 52b3bb841b400f8f27171266d37f9de50edf1088 on linode:feature/migrations.


What is this?

I am sorry but I seem to be either quite confused or missing something.  It's never been breached.
It hides all the text in crypto blank spaces only chosen can see.

lol, my bad. I bow down to the creator and his messenger. May I be worthy of noCode privilege one day and gather enough wisdom to finally see ...


Internal server errror when sending authorization header

Any authenticated request (by authenticated I mean sending authorization: Bearer 1231...) crashes the request. I just get Internal Server Error with this error showing in Node.
[timestamp:2019-05-05T16:47:04.384Z] [level:info] [message:API ready] [port:3000]
  TypeError: this.knex is not a function
      at UserProcessor.getQuery (/Users/betocantu93/Documents/prototypal/untitled-chat-project-api/node_modules/@ebryn/jsonapi-ts/src/processors/knex-processor.ts:93:17)
      at UserProcessor.get (/Users/betocantu93/Documents/prototypal/untitled-chat-project-api/node_modules/@ebryn/jsonapi-ts/src/processors/knex-processor.ts:116:46)
      at UserProcessor.identify (/Users/betocantu93/Documents/prototypal/untitled-chat-project-api/src/processors/userProcessor.ts:11:18)
      at UserProcessor.execute (/Users/betocantu93/Documents/prototypal/untitled-chat-project-api/node_modules/@ebryn/jsonapi-ts/src/processors/operation-processor.ts:27:56)
      at Application.executeOperation (/Users/betocantu93/Documents/prototypal/untitled-chat-project-api/node_modules/@ebryn/jsonapi-ts/src/application.ts:74:36)
      at process._tickCallback (internal/process/next_tick.js:68:7)
(node:53824) Warning: a promise was rejected with a non-error: [object Object]
(node:53824) Warning: a promise was rejected with a non-error: [object Object]
JSONAPI-TS:  TypeError: (intermediate value) is not iterable
    at handleJsonApiEndpoint (/Users/betocantu93/Documents/prototypal/untitled-chat-project-api/node_modules/@ebryn/jsonapi-ts/src/middlewares/json-api-koa.ts:104:30)
    at process._tickCallback (internal/process/next_tick.js:68:7)
[timestamp:2019-05-05T16:47:13.108Z] [level:info] [message:IncomingRequest] [url:/users/49] [queryParams:undefined] [body:{}] [headers:{"client":"catalog","cache-control":"no-cache","postman-token":"536a257d-b177-49c1-9200-0e7ec8dc19f2","user-agent":"PostmanRuntime/7.6.0","accept":"*/*","host":"localhost:3000","accept-encoding":"gzip, deflate","connection":"keep-alive"}]
[timestamp:2019-05-05T16:47:13.109Z] [level:error] [message:OutgoingResponse] [status:500] [body:{"errors":[{"status":500,"code":"unhandled_error"}]}] [headers:{"client":"catalog","cache-control":"no-cache","postman-token":"536a257d-b177-49c1-9200-0e7ec8dc19f2","user-agent":"PostmanRuntime/7.6.0","accept":"*/*","host":"localhost:3000","accept-encoding":"gzip, deflate","connection":"keep-alive"}]
{ status: 401, code: 'unauthorized' }
  TypeError: this.knex is not a function
      at UserProcessor.getQuery (/Users/betocantu93/Documents/prototypal/untitled-chat-project-api/node_modules/@ebryn/jsonapi-ts/src/processors/knex-processor.ts:93:17)
      at UserProcessor.get (/Users/betocantu93/Documents/prototypal/untitled-chat-project-api/node_modules/@ebryn/jsonapi-ts/src/processors/knex-processor.ts:116:46)
      at UserProcessor.identify (/Users/betocantu93/Documents/prototypal/untitled-chat-project-api/src/processors/userProcessor.ts:11:18)
      at UserProcessor.execute (/Users/betocantu93/Documents/prototypal/untitled-chat-project-api/node_modules/@ebryn/jsonapi-ts/src/processors/operation-processor.ts:27:56)
      at Application.executeOperation (/Users/betocantu93/Documents/prototypal/untitled-chat-project-api/node_modules/@ebryn/jsonapi-ts/src/application.ts:74:36)
      at process._tickCallback (internal/process/next_tick.js:68:7)
e

This has been fixed... I believe yesterday. If you are referring to chat-api, I haven't released the fix there yet


Fix NotImplemented does not derive from BaseException

Creating an instance of Validator with a schema that requires a missing validator rule raises a TypeError because NotImplemented is not the correct error. It should be NotImplementedError.
from cerberus import Validator
schema = {'amount': {'validator': 'oddity'}}
v = Validator(schema)
...
TypeError: exceptions must derive from BaseException

erm, to be honest. these were just placeholders while re-massaging the implementation.
so, these parts should get implemented (or be removed). still, i have no clue why the tests succeed so far.

@funkyfuture OK, I will let you decide whether to implement or remove. Please feel free to close this PR as you see fit.

Thanks!


Rename Object.notification to notify

DO NOT EXPECT THIS PR TO BE MERGED BEFORE STABLE 4.0.
This was mostly a proof of test.
Closes https://github.com/godotengine/godot-proposals/issues/5668
See https://github.com/godotengine/godot-proposals/issues/5668 for reasoning. To make it brief, Object.notification() and Object._notification() is outstandingly confusing and makes the whole ordeal seem more complicated than it actually is. It doesn't help that the class reference sometimes refers to it as "notify", and there exist methods such as SceneTree.notify_group(), yet at the same time the documentation is discouraged from calling it "notify" because that's not how the original method is called.
This PR renames Object.notification() to notify(), across the whoooole codebase.

I do not expect this to make the cut, but it is worth a shot. It would really improve user accessibility for notifications and make the explanation of EVERY other notification in the engine so much less confusing.

After discussing this internally, we decided to reject this rename. While we're partial to this change in principle, the name happens to sit right in between two conventions. On the one hand, as a method name it's not great, breaks the expected pattern of being a verb (or a verb and a subject). On the other hand, this does follow the pattern for pairs of public/private methods that we have in the core and Object API.
All those methods are implemented as multilevel calls and all of them have this same pattern of the names being the same with the private counterpart having an underscore. Though the majority of them still respects the verb or verb + subject structure. So the rename would make sense here, if we also could rename _notification, the private one. But that's not going to happen. It's used everywhere, it would be too much of breaking change. And its name is kind of fitting too, as you receive notifications with it. It's like a signal handler.
So as you can see, pulling this in either direction breaks one expectation or convention, or it breaks another. So ultimately, it doesn't seem to be worth it to rename it this way. Sorry!


flutter run android,ProcessException: ProcessException: Permission denied.

command
flutter run
exception
ProcessException: ProcessException: Permission denied
Command: /Users/huanghuiqun750/Desktop/PingAn Works/EPV/EPMonitor-Flutter/android/gradlew -v
#0      runAsync (package:flutter_tools/src/base/process.dart:242:35)
<asynchronous suspension>
#1      runCheckedAsync (package:flutter_tools/src/base/process.dart:329:34)
<asynchronous suspension>
#2      _initializeGradle (package:flutter_tools/src/android/gradle.dart:297:9)
<asynchronous suspension>
#3      _ensureGradle (package:flutter_tools/src/android/gradle.dart:278:37)
<asynchronous suspension>
#4      _readGradleProject (package:flutter_tools/src/android/gradle.dart:192:31)
<asynchronous suspension>
#5      _gradleAppProject (package:flutter_tools/src/android/gradle.dart:112:37)
<asynchronous suspension>
#6      getGradleAppOut (package:flutter_tools/src/android/gradle.dart:106:29)
<asynchronous suspension>
#7      AndroidApk.fromAndroidProject (package:flutter_tools/src/application_package.dart:164:23)
<asynchronous suspension>
#8      ApplicationPackageFactory.getPackageForPlatform (package:flutter_tools/src/application_package.dart:46:32)
<asynchronous suspension>
#9      FlutterDevice.runHot (package:flutter_tools/src/resident_runner.dart:359:56)
<asynchronous suspension>
#10     HotRunner.run (package:flutter_tools/src/run_hot.dart:252:39)
<asynchronous suspension>
#11     RunCommand.runCommand (package:flutter_tools/src/commands/run.dart:460:37)
<asynchronous suspension>
#12     FlutterCommand.verifyThenRunCommand (package:flutter_tools/src/runner/flutter_command.dart:490:18)
#13     _asyncThenWrapperHelper.<anonymous closure> (dart:async-patch/async_patch.dart:71:64)
#14     _rootRunUnary (dart:async/zone.dart:1132:38)
#15     _CustomZone.runUnary (dart:async/zone.dart:1029:19)
#16     _FutureListener.handleValue (dart:async/future_impl.dart:137:18)
#17     Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:678:45)
#18     Future._propagateToListeners (dart:async/future_impl.dart:707:32)
#19     Future._completeWithValue (dart:async/future_impl.dart:522:5)
#20     _AsyncAwaitCompleter.complete (dart:async-patch/async_patch.dart:30:15)
#21     _completeOnAsyncReturn (dart:async-patch/async_patch.dart:288:13)
#22     RunCommand.usageValues (package:flutter_tools/src/commands/run.dart)
#23     _asyncThenWrapperHelper.<anonymous closure> (dart:async-patch/async_patch.dart:71:64)
#24     _rootRunUnary (dart:async/zone.dart:1132:38)
#25     _CustomZone.runUnary (dart:async/zone.dart:1029:19)
#26     _FutureListener.handleValue (dart:async/future_impl.dart:137:18)
#27     Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:678:45)
#28     Future._propagateToListeners (dart:async/future_impl.dart:707:32)
#29     Future._completeWithValue (dart:async/future_impl.dart:522:5)
#30     _AsyncAwaitCompleter.complete (dart:async-patch/async_patch.dart:30:15)
#31     _completeOnAsyncReturn (dart:async-patch/async_patch.dart:288:13)
#32     AndroidDevice.isLocalEmulator (package:flutter_tools/src/android/android_device.dart)
#33     _asyncThenWrapperHelper.<anonymous closure> (dart:async-patch/async_patch.dart:71:64)
#34     _rootRunUnary (dart:async/zone.dart:1132:38)
#35     _CustomZone.runUnary (dart:async/zone.dart:1029:19)
#36     _FutureListener.handleValue (dart:async/future_impl.dart:137:18)
#37     Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:678:45)
#38     Future._propagateToListeners (dart:async/future_impl.dart:707:32)
#39     Future._completeWithValue (dart:async/future_impl.dart:522:5)
#40     Future._asyncComplete.<anonymous closure> (dart:async/future_impl.dart:552:7)
#41     _rootRun (dart:async/zone.dart:1124:13)
#42     _CustomZone.run (dart:async/zone.dart:1021:19)
#43     _CustomZone.runGuarded (dart:async/zone.dart:923:7)
#44     _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:963:23)
#45     _microtaskLoop (dart:async/schedule_microtask.dart:41:21)
#46     _startMicrotaskLoop (dart:async/schedule_microtask.dart:50:5)
#47     _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:116:13)
#48     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:173:5)

flutter doctor
[✓] Flutter (Channel master, v1.9.8-pre.79, on Mac OS X 10.14.4 18E226, locale zh-Hans-CN)
    • Flutter version 1.9.8-pre.79 at /Users/huanghuiqun750/development/flutter
    • Framework revision 72cacb4040 (3 weeks ago), 2019-09-03 18:00:45 -0700
    • Engine revision e7f9ef6aa0
    • Dart version 2.5.0 (build 2.5.0-dev.4.0 36985859e4)

[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
    • Android SDK at /Users/huanghuiqun750/Library/Android/sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-28, build-tools 28.0.3
    • ANDROID_HOME = /Users/huanghuiqun750/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1343-b01)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 10.2.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 10.2.1, Build version 10E1001
    • CocoaPods version 1.7.0.beta.3

[✓] Android Studio (version 3.4)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 38.2.1
    • Dart plugin version 183.6270
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1343-b01)

[✓] VS Code (version 1.27.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 2.21.1

[✓] Connected device (1 available)
    • Android SDK built for x86 • emulator-5554 • android-x86 • Android 7.0 (API 24) (emulator)

• No issues found!

Hi, you are using a pretty old flutter version on master.
Could you please upgrade?
Does this also happen with a newly created project?

Mine is new and same problem.
It's getting really boring constantly running into problems that are clearly configuration or compatibility. The flutter team would do themselves a HUGE favor by testing this... keep in mind that flutter is meant to deploy on multiple platforms, ideally without changes, therefore the shim layer between ios and flutter and android and flutter should have an ideal configuration or starting point.
[✓] Flutter (Channel stable, v1.9.1+hotfix.4, on Mac OS X 10.14.6 18G95, locale en-US)
    • Flutter version 1.9.1+hotfix.4 at /Users/rbucker/dev/flutter
    • Framework revision cc949a8e8b (9 days ago), 2019-09-27 15:04:59 -0700
    • Engine revision b863200c37
    • Dart version 2.5.0

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.2)
    • Android SDK at /Users/rbucker/Library/Android/sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-29, build-tools 29.0.2
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 11.0)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 11.0, Build version 11A420a
    • CocoaPods version 1.8.1

[✓] Android Studio (version 3.5)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 40.0.2
    • Dart plugin version 191.8423
    • Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)

[✓] VS Code (version 1.38.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.4.1

[✓] Connected device (1 available)
    • Pixel 3a • 98BAY13YKL • android-arm64 • Android 10 (API 29)

• No issues found!

for good measure I did a flutter upgrade etc... and still no joy.
[✓] Flutter (Channel stable, v1.9.1+hotfix.4, on Mac OS X 10.14.6 18G95, locale en-US)
    • Flutter version 1.9.1+hotfix.4 at /Users/rbucker/dev/flutter
    • Framework revision cc949a8e8b (9 days ago), 2019-09-27 15:04:59 -0700
    • Engine revision b863200c37
    • Dart version 2.5.0

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.2)
    • Android SDK at /Users/rbucker/Library/Android/sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-29, build-tools 29.0.2
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 11.0)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 11.0, Build version 11A420a
    • CocoaPods version 1.8.1

[✓] Android Studio (version 3.5)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 40.0.2
    • Dart plugin version 191.8423
    • Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)

[✓] VS Code (version 1.38.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.4.1

[✓] Connected device (1 available)
    • Pixel 3a • 98BAY13YKL • android-arm64 • Android 10 (API 29)

• No issues found!

Hi, I just resolved the same problem by running chmod +x android/gradlew
The relative issue can be find at https://stackoverflow.com/questions/17668265/gradlew-permission-denied/17669566

flutter upgrade . Flutter 1.7.8+hotfix.4 • channel stable
resolved the problem

Olá, Acabei de resolver o mesmo problema executando chmod +x android/gradlew
O problema relativo pode ser encontrado em https://stackoverflow.com/questions/17668265/gradlew-permission-denied/17669566

It fix to me


No support for ReplicationFactor in Graph and Collection

In order to use ArangoDB Cloud (Oasis), a Replication Factor of at least 3 needs to be specified in the Graph and Collection objects.
I have made the following change to make it work.
https://gist.github.com/crajah/7861653fa20e0776ad7a2327bfd11192

Would you be interested in submitting a PR for this?

Absolutely. I don't have access to submit a PR though.
Chandan Rajah
Chief Executive Officer
<EMAIL_ADDRESS>+44 7711 060 452
This e-mail is confidential and may contain privileged information or
otherwise be protected. If you are not the intended recipient (or have
received this e-mail in error) please notify Parallel AI immediately by
e-mail at<EMAIL_ADDRESS>and permanently delete this e-mail from your
system. Any unauthorised copying, disclosure or distribution of the
material in this e-mail is strictly prohibited. Email communications cannot
be guaranteed to be secure or error free as information can, inter alia, be
corrupted, intercepted, lost, contain viruses or otherwise be altered.
Parallel AI may monitor outgoing and incoming messages for reasons of
security, to protect its respective businesses and to ensure compliance
with legal and regulatory obligations and its internal policies. In
communicating with us by email you understand and accept these risks, and
Parallel AI accepts no liability or responsibility for such matters or
their respective consequences.
On Mon, 9 Nov 2020 at 17:05, Matt Hicks<EMAIL_ADDRESS>wrote:

Would you be interested in submitting a PR for this?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/outr/scarango/issues/19#issuecomment-724145657, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ABARBL37KT4EKQUJJBIBJVTSPAONZANCNFSM4TOSZPQQ
.

Absolutely. I don't have access to submit a PR though. :-(

What do you mean?  All you have to do in GitHub to submit a PR is to fork the project, commit your changes to your fork, and then submit that fork as a PR. No access is required except for having a GitHub account.

My mistake. I was attempting to create a branch. :-). I'll send a PR shortly.

No worries, let me know if you have any trouble.


fix: fix node version

:tada: This PR is included in version 2.2.0 :tada:
The release is available on:

GitHub release
npm package (@latest dist-tag)

Your semantic-release bot :package::rocket:


Cannot pair to sony camera but works when using Bluedroid

Hi h2zero,
Hello, I encountered a problem with connecting esp32 and Sony ZV-1 camera, and I would like to ask you for advice.
I modified the example program of Nimble-Arduino library a bit, and I wanted to use esp32 to control my ZV-1 camera, but when esp32 and the camera paired, the program got stuck at D NimBLEClient: >> secureConnection(), and then reported an error E NimBLEClient: secureConnection: failed rc=13.
I know that ZV-1 would pop up a confirm pairing window, when it's in pairing mode, and the connected device has sent any requests to it. I can successfully simulate this process with the nrf connect app on Android.
I saw on GitHub that someone was also testing how to use Nimble or Bluedroid to control Sony cameras. I tried his program and found that, using nimble-arduino would also get stuck at D NimBLEClient: >> secureConnection(), and disconnect after 10 seconds;
However, using Bluedroid can pair and control normally. And after pairing successfully with Bluedroid, I changed to flash the Nimble-Arduino program,and this time esp32 could also pair with the camera, but the device name displayed on the camera was Bluedroid program's.
I tried changing the settings of NimBLEDevice::setSecurityAuth() or NimBLEDevice::setSecurityIOCap(), but it didn't work.  Am I doing anything wrong or is there a bug in the Nimble-Arduino library?
Thank you for your time and help.

All I can see in the log is this:
secureConnection: failed rc=13

Which is

0x0d
BLE_HS_ETIMEOUT
Operation timed out.

This may be due to the configuration of the security settings in NimBLEDevice::setSecurityIOCap() and NimBLEDevice::setSecurityAuth().

All I can see in the log is this:
secureConnection: failed rc=13

Which is
0x0d	BLE_HS_ETIMEOUT	Operation timed out.
This may be due to the configuration of the security settings in NimBLEDevice::setSecurityIOCap() and NimBLEDevice::setSecurityAuth().

Hi h2zero,
It seems that the camera only requires no io capacity and bonding to be functional.
so I've tried to set my esp32 with:
NimBLEDevice::setSecurityIOCap(BLE_HS_IO_NO_INPUT_OUTPUT);
NimBLEDevice::setSecurityAuth(BLE_SM_PAIR_AUTHREQ_BOND |  BLE_SM_PAIR_AUTHREQ_SC);

but this doesn't help. I'm still getting the same error. I also tried other combinations and they don't help too.
Then I found another one using NRF52840 the chip and bluefruit library to control a sony camera. seems in the code he is able to manually issue an SMP Pairing Request or an SMP Security Request. I don't know how to achieve this with an esp32 so I'm just assuming ble libraries would do this automatically. so I just tried to duplicate the default GAP security parameters of the bluefruit library:
// default is Just Work
static const ble_gap_sec_params_t _sec_param_default =
{
  .bond         = 1,
  .mitm         = 0,
  .lesc         = LESC_SUPPORTED,
  .keypress     = 0,
  .io_caps      = BLE_GAP_IO_CAPS_NONE,
  .oob          = 0,
  .min_key_size = 7,
  .max_key_size = 16,
  .kdist_own    = { .enc = 1, .id = 1},
  .kdist_peer   = { .enc = 1, .id = 1}
};

And I'm doing like this:
  ble_hs_cfg.sm_bonding=1;
  ble_hs_cfg.sm_mitm=0;
  ble_hs_cfg.sm_sc=1;
  ble_hs_cfg.sm_keypress=0;
  ble_hs_cfg.sm_io_cap=BLE_SM_IO_CAP_NO_IO;
  ble_hs_cfg.sm_oob_data_flag=0;
  ble_hs_cfg.sm_our_key_dist=BLE_SM_PAIR_KEY_DIST_ENC|BLE_SM_PAIR_KEY_DIST_ID;
  ble_hs_cfg.sm_their_key_dist=BLE_SM_PAIR_KEY_DIST_ENC|BLE_SM_PAIR_KEY_DIST_ID;

Unfortunately this doesn't make a difference either. I'm still getting the same error.
I'm still new to ble, and don't know why Nimble and Bluedroid behave differently, although they seem to be using similar interfaces. the Bluedroid code just sets encryption level with BLEDevice::setEncryptionLevel(ESP_BLE_SEC_ENCRYPT); and all the thing goes flawlessly; but I just don't know what kind of magic is happening here, and what to do to get Nimble working like that. Could you please give me some more advice on that?
Best reguards

I think this may have something to do with BLE_SM_PAIR_AUTHREQ_SC or ble_hs_cfg.sm_sc=1; being enabled. Try removing this, by setting ble_hs_cfg.sm_sc=0;

@Lemin2  Any updates?


api.go doesn't respect nsswitch.conf

Due to the implementation of api.go which reads only /etc/group, it is impossible to rely on other auth mechanisms (e.g. LDAP, NIS).
groups, err := ioutil.ReadFile("/etc/group")
if err != nil {
        return err
}
re := regexp.MustCompile("(^|\n)docker:.*?:([0-9]+)")
if gidMatch := re.FindStringSubmatch(string(groups)); gidMatch != nil {
        gid, err := strconv.Atoi(gidMatch[2])
        if err != nil {
                return err
        }
        utils.Debugf("docker group found. gid: %d", gid)
        if err := os.Chown(addr, 0, gid); err != nil {
                return err
        }
}

Instead, the getgrnam syscall should be used to determine, whether a docker group exists or not.
The implemenation of lookup_unix.go in the user Go package might serve as inspiration on how to do this.

seems like there's a new patch but still in review since ~6 months https://codereview.appspot.com/101310044

Any news on this ?
Group lookup is still done in the staticaly defined /etc/group file without any support to NSS lookups.
Any chance to have Docker support basic Linux features ?
Thanks.

+1

USER POLL
The best way to get notified of updates is to use the Subscribe button on this page.
Please don't use "+1" or "I have this too" comments on issues. We automatically
collect those comments to keep the thread short.
The people listed below have upvoted this issue by leaving a +1 comment:
@rmrf

Given the activity level on this issue, I'm going to close it as it's either fixed, a duplicate or not a request anymore. If you think I'm mistaken, feel free to discuss it there 😉

Not sure why this would be closed, this bug is still there. See docker/for-linux#186 and analysis by @mjameswh.

Please fix before closing !

@vdemeester Could you please consider reopening this issue?
I think the PR I submitted yesterday should fix it, but I'm afraid that at present, it won't get any attention because it has been closed a long time ago...


FAILURE: Build failed with an exception.

What went wrong:
Could not resolve all dependencies for configuration ':forgeGradleUserDevPackage'.

Could not resolve net.minecraftforge:forge:1.11.2-<IP_ADDRESS>8.
Required by:
com.microsoft.MalmoMod:Minecraft:0.35.6
Could not resolve net.minecraftforge:forge:1.11.2-<IP_ADDRESS>8.
Could not get resource 'https://files.minecraftforge.net/maven/net/minecraftforge/forge/1.11.2-<IP_ADDRESS>8/forge-1.11.2-<IP_ADDRESS>8.pom'.
Could not HEAD 'https://files.minecraftforge.net/maven/net/minecraftforge/forge/1.11.2-<IP_ADDRESS>8/forge-1.11.2-<IP_ADDRESS>8.pom'.
sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Possibly a temporary networking issue but could you say what OS you are using and which Java version you have installed (java -version should say 8), thanks?

Hello. I'm seeing the same issue with Malmo 0.36.1 on Windows 10 with Java 8.

Hi, thanks for that info!