[Mac] - Scrolling momentum has disappeared from Listener scroll signal

In super_editor we use a Listener widget to implement scrolling in our documents.
Before Flutter 3.3, when scrolling with a Magic Mouse on Mac, the Listener widget reported events that implicitly included momentum. In other words, if we simply added event.scrollDelta.dy to our scroll offset, we would get scroll momentum for free.
In Flutter 3.3, that momentum is gone. This has made our scrolling experience unacceptably bad.
https://user-images.githubusercontent.com/7259036/193917727-df76fcd6-8fb9-466f-9209-b81510c459fb.mov
I don't have a full repro, but here are the relevant pieces:
void _scrollOnMouseWheel(PointerSignalEvent event) {
    if (event is PointerScrollEvent) {
      widget.autoScroller.jumpBy(event.scrollDelta.dy);
    }
  }

@override
  Widget build(BuildContext context) {
    return Listener(
      onPointerSignal: _scrollOnMouseWheel,
      // Note: in our actual code we have a bunch of stuff between the Listener and the SingleChildScrollView
      child: SingleChildScrollView(
        controller: _scrollController,
        physics: const NeverScrollableScrollPhysics(),
        child: child,
      ),
    );
  }

Flutter version:
Flutter 3.3.3 • channel stable • https://github.com/flutter/flutter.git
Framework • revision 18a827f393 (6 days ago) • 2022-09-28 10:03:14 -0700
Engine • revision 5c984c26eb
Tools • Dart 2.18.2 • DevTools 2.15.0

Thanks for the report. I cloned super_editor and ran it on latest stable and observed that scrolling isn't smooth.
https://user-images.githubusercontent.com/67046386/194001058-3588bff6-e4e3-4b30-95b4-e6af7ac93140.mov

 stable flutter doctor -v

[✓] Flutter (Channel stable, 3.3.3, on macOS 12.2.1 21D62 darwin-x64, locale
    en-GB)
    • Flutter version 3.3.3 on channel stable at
      /Users/dhs/documents/fluttersdk/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 18a827f393 (2 days ago), 2022-09-28 10:03:14 -0700
    • Engine revision 5c984c26eb
    • Dart version 2.18.2
    • DevTools version 2.15.0

[!] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    ! Flutter recommends a minimum Xcode version of 13.
      Download the latest version or update via the Mac App Store.
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (5 available)
    • SM G975F (mobile)       • RZ8M802WY0X • android-arm64   • Android 11 (API 30)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 98.0.4758.80

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.

Magic mouse is actually treated as a trackpad.  Does this also happen with a trackpad?
This could be due to adding support for trackpad gestures, where the events presumably take a different path.
cc @dkwingsmt

Yeah, this is a breaking change from the trackpad support. Trackpad scrolling used to be simulated as mouse scrolling, represented as PointerSignalEvent. Now they're sent directly as PointerPanZoomEvent, which doesn't trigger Listener.onSignal. Currently Listener has no way of receiving pan-zoom events. We can add a onPanZoom to Listener, although you'll still have to write your own callback.
cc @moffatman

@moffatman we purposefully don't defer to the SingleChildScrollView. Also, generally, if developers were processing interaction events from the platform, the new answer shouldn't be "add a widget". Because a widget is a fundamentally different approach to a problem than handling platform events. I believe that we still want to handle each scrolling event.
This then raises the question: for those who were handling scroll events, and depending on momentum, what must we do to recover momentum in our event handling? Can one of you please add such an example to the breaking changes document? Maybe that means tracking velocity and putting it through a simulation. I'm not sure. But, I would suggest that the document is incomplete without a demonstration to recover the behavior that was lost. And again, the behavior that was lost was related to individual event handling, not widget behavior.

Sure, if you want to do it that way you have the right idea:

Use Listener onPointerPanZoomUpdate to get trackpad pan deltas
Use each pan delta in your custom scrolling thing
Add each pan position to a velocity tracker
Use Listener onPointerPanZoomEnd to know when the user lifted their finger off the trackpad
Create a BallisticScrollSimulation using the final velocity of your velocity tracker
Poll the simulation at some rate, and use the delta in your custom scrolling thing
(Flutter 3.4+) Listen for onPointerSignal's PointerScrollInertiaCancel event, and stop your simulation if received

This is to handle the user resting their finger back on the trackpad

@moffatman can you add a relevant example to the migration guide in the breaking change document? There could be any number of developers who need the same advice.

@matthew-carroll
I have an example below, but I haven't put it in the migration guide as I don't recommend using it. To be honest, there is no real migration for removal of momentum at the platform event level, you're not going to be able to ever recreate the events exactly. This example is more of a workaround than a migration, as even with it you will still be missing some features when compared to using the native scrolling widget (retained momentum between swipes, over-scroll rubber-banding at top and bottom).

Faking momentum example
import 'dart:async';

import 'package:flutter/cupertino.dart';
import 'package:flutter/gestures.dart';
import 'package:flutter/material.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatefulWidget {
  const MyApp({Key? key}) : super(key: key);

  @override
  createState() => _MyAppState();
}

class _MyAppState extends State<MyApp> with TickerProviderStateMixin {
  DateTime trackingStartTime = DateTime.now();
  VelocityTracker? tracker;
  Simulation? simulationX;
  Simulation? simulationY;
  double simulationTime = 0;
  Timer? timer;
  static const simulatedInputHz = 120;
  static const duration = Duration(milliseconds: 1000 ~/ simulatedInputHz);
  Offset offset = Offset.zero;
  Color color = Colors.blue;
  bool isTrackpad = false;

  void _tick(Timer t) {
    final x0 = simulationX!.x(simulationTime);
    final y0 = simulationY!.x(simulationTime);
    simulationTime += duration.inMicroseconds / 1e6;
    final x1 = simulationX!.x(simulationTime);
    final y1 = simulationY!.x(simulationTime);
    final diff = Offset(x1 - x0, y1 - y0);
    if (diff == Offset.zero && simulationTime > 0) {
      t.cancel();
      setState(() {
        color = Colors.blue;
      });
    }
    else {
      setState(() {
        offset += diff;
      });
    }
  }

  @override
  Widget build(BuildContext context) {
    return CupertinoApp(
      title: 'Inertia Demo',
      home: CupertinoPageScaffold(
        child: Listener(
          onPointerSignal: (event) {
            // Flutter 3.4+ only
            if (event is PointerScrollInertiaCancelEvent) {
              timer?.cancel();
              setState(() {
                color = Colors.blue;
              });
            }
          },
          onPointerPanZoomStart: (event) {
            isTrackpad = event.kind == PointerDeviceKind.trackpad;
            if (!isTrackpad) return;
            timer?.cancel();
            tracker = VelocityTracker.withKind(event.kind);
            trackingStartTime = DateTime.now();
            setState(() {
              color = Colors.red;
            });
          },
          onPointerPanZoomUpdate: (event) {
            if (!isTrackpad) return;
            setState(() {
              offset += event.panDelta;
            });
            tracker?.addPosition(DateTime.now().difference(trackingStartTime), event.pan);
          },
          onPointerPanZoomEnd: (event) {
            if (!isTrackpad) return;
            final metrics = FixedScrollMetrics(
              minScrollExtent: 0,
              maxScrollExtent: 99999,
              pixels: 50000,
              viewportDimension: 99999,
              axisDirection: AxisDirection.down
            );
            const physics = BouncingScrollPhysics(
              // Flutter 3.4+ only
              decelerationRate: ScrollDecelerationRate.fast
            );
            simulationX = physics.createBallisticSimulation(
              metrics,
              tracker!.getVelocity().pixelsPerSecond.dx
            );
            simulationY = physics.createBallisticSimulation(
              metrics,
              tracker!.getVelocity().pixelsPerSecond.dy
            );
            simulationTime = 0;
            timer?.cancel();
            timer = Timer.periodic(duration, _tick);
            setState(() {
              color = Colors.orange;
            });
          },
          child: Container(
            color: Colors.green,
            child: Center(
              child: Transform.translate(
                offset: offset,
                child: Container(
                  color: color,
                  width: 50,
                  height: 50
                )
              )
            )
          )
        )
      ),
    );
  }

  @override
  void dispose() {
    super.dispose();
    timer?.cancel();
  }
}

This breakage only affects people reimplementing their own scrolling solely for desktop usage, because if you ever need to handle touch, you are already unable to rely system scrolling momentum. I would guess the number of affected packages is nearly zero, I'm happy to help with contributions to any of them including your own in transitioning to the new system.
I tried the following change in your DocumentScrollable and it seems to work fine:
  Widget _buildScroller({
    required Widget child,
  }) {
    return ScrollConfiguration(
      behavior: ScrollConfiguration.of(context).copyWith(
        dragDevices: {PointerDeviceKind.trackpad}
      ),
      child: SingleChildScrollView(
        controller: _scrollController,
        child: child,
      )
    );
  }

This example is more of a workaround than a migration, as even with it you will still be missing some features when compared to using the native scrolling widget (retained momentum between swipes, over-scroll rubber-banding at top and bottom)

I don't understand this point. How is the rest of Flutter implementing scrolling momentum? Are you saying that there's some special behavior locked away in some Flutter widget, and that behavior isn't available more broadly?

Nothing special, just that the ScrollView / ScrollPhysics has a lot of logic to wire up. I guess I could have put it in the example but it gets more and more like copying chunks of code from the framework scrolling classes. If you use that workaround, you could get trackpad scrolls to work as they did before in your widget. But Flutter trackpad scrolling got better in recent release by treating it more like a touch drag rather than a lot of discrete mouse events. Let me rephrase: my example is a workaround to use the previous behaviour, rather than a migration to new behaviour to better meet expectations of how trackpad scrolling works in other apps.

just that the ScrollView / ScrollPhysics has a lot of logic to wire up

This is an important statement to keep in mind. For those of us who can't use out-of-the-box widgets, we're stuck with whatever set of responsibilities come with separating scrolling from gestures. This is probably a good opportunity to improve the framework. Why is there so much to wire up for scrolling? Could Flutter reduce that burden for those of us with more complicated use-cases?

it gets more and more like copying chunks of code from the framework scrolling classes

This is a good indication that Flutter has done a poor job of decomposing the scrolling problem. Instead, it sounds like Flutter probably implemented a number of logical scrolling solutions within pre-packaged widgets. Thus, the problem we're facing with super_editor.

like a touch drag rather than a lot of discrete mouse events

I'd love to take advantage of new and improved behavior. But scrolling is fundamentally not a touch gesture. So when you say that it's treated as a touch drag, does that include a gesture recognizer taking control of the gesture arena? Does it include a visual area that absorbs taps? Because that's what we have to avoid. super_editor is probably one of the most complex combinations of layout, scrolling, and touch use-cases in the Flutter ecosystem. To reach our goals, we need to separate scroll position/movement from user interaction. Is that possible with this new trackpad system? Is it possible with the magic mouse?

my example is a workaround to use the previous behaviour, rather than a migration to new behaviour to better meet expectations of how trackpad scrolling works in other apps

My goal certainly isn't to implement a temporary hack. My point with this thread is that we had a behavior in super_editor that was broken by Flutter. I haven't seen anything to suggest to me that we were somehow misusing or abusing a Flutter behavior. We were only guilty of separating touch input from logical scrolling, which I believe was necessary, and I believe it's a completely legitimate approach. So if there's an intentional migration to recover scrolling behavior, without completely breaking the rest of our input system, I'm all ears. By all means, let's see a recommended migration guide for the problems we're dealing with.

I think the use case is so specific that a general migration guide will be a waste of time rather than fixing the specific package. I still don't really know all the constraints you have, but I found a better way to resolve this which fits in your design. I basically copied how it was done for the super_editor touch versions, and it works well.
https://github.com/moffatman/super_editor/commit/d484c1375f96af8affd181f87e050c03b66c93a9
Now it behaves like it would before the update, you are not at feature parity with letting the scrollview handle it, but it's quite close. You are missing overscrolling, but never had that to begin with, and actually don't even have it on touch devices. So that is a larger rewrite for another day...

Now it behaves like it would before the update, you are not at feature parity with letting the scrollview handle it, but it's quite close. You are missing overscrolling, but never had that to begin with, and actually don't even have it on touch devices. So that is a larger rewrite for another day...

Nothing special, just that the ScrollView / ScrollPhysics has a lot of logic to wire up. I guess I could have put it in the example but it gets more and more like copying chunks of code from the framework scrolling classes.

For what it's worth, this has disrupted more than just super_editor and Matthew's effort to help the Flutter community. It's affected us greatly too with no clear pathway forward.
I'm still wrapping my head around the refactored framework code -- but it seems like what was done is good and bad.
Good: The full scrolling experience has been moved into Flutter (no scroll simulation).
Bad: I'm just looking into this still, so please help me if I'm wrong -- but it kind of looks like all the trackpad scrolling logic and behaviors were rewritten to be tightly coupled to ScrollView with no clear pathway forward to use that logic or behavior in a sustainable way.
I'm concerned because I'm trying to figure out how to move forward and the only two options I see right now are:

Refactor the whole app to use ScrollView which doesn't cater to our use case
Essentially fork the Flutter framework by copy-pasting 100s of lines

Is there no non-tightly coupled way to separate touch input from logical scrolling moving forward with Flutter?

it doesn't stop scrolling when I touch a macOS touchpad which is expected behavior before Flutter 3.3

With the approach suggested in https://github.com/flutter/flutter/issues/112880#issuecomment-1272150287 this is true for the scroll wheel, too.
Because the momentum control has been removed from Flutter's signals, we're responsible for adding momentum, we are also responsible for stopping momentum. As a result, I assume we need to add a tag gesture recognizer that cancels momentum on tap down. This can get tricky in the case of mouse scroll wheels because we're trying to keep our scrolling system from taking control of touch gestures, but now we need to add a touch recognizer to stop the scroll momentum, which will intercept all such touch events. I'm gonna try to turn that recognizer on/off based on whether there's scroll momentum, but I don't know how well that's going to work.

Momentum was moved to the framework level as the platform/engine momentum was only usable for linear-scrolling lists. When widgets have control over momentum, you it enables smooth over-scrolling animations, pull-to-refresh, PageView-type scrolling (i.e. smoothly aligning to discrete pages), and custom non-scrolling trackpad gestures (ex: swiping left/right on a message in Mail.app).
Nothing is tightly-coupled. If you want to reimplement ScrollView yourself, of course to have feature-parity you need to implement every feature... The features that I mentioned above as being missing in my momentum simulation example are features that were never present on trackpad scrolling pre-3.3. I used them as examples of things you would not have if using an approach of recreating the old events and old mechanics.
Let me know if I can take a look at your use case @tannermeade, I think the problem will be easier to solve than you think.
Sorry that we don't have everything in 3.3, as the release version comes much later than the code changes, I didn't have a good connection of what was landing together. I would have rather everything come at once....

Not sure what is meant above in the discussion about mouse scroll wheels, those should continue to work as in previous version in flutter (mouse wheel turns being discrete scroll increments, with no momentum). Maybe you are just meaning Magic Mouse?
For Flutter 3.3, you can use the onPointerHover callback of Listener to cancel momentum, no need to interact with the gesture arena. And in 3.4+, you will have the PointerScrollInertiaCancel event in onPointerSignal instead, which will fire even when the the trackpad is touched without any tap or movement.

Maybe you are just meaning Magic Mouse?

Yes, I'm using the term interchangeably. I don't actually have a moues with a physical scroll wheel.

For Flutter 3.3, you can use the onPointerHover callback of Listener to cancel momentum

Unfortunately, this isn't the case. I applied your code and as the document scrolls I'm able to tap to place a caret while the document continues to scroll. There's also a really weird, rare situation where somehow I'm able to get into a tap-to-drag scrolling mode. I have no idea where that's coming from.

BTW @moffatman can you explain why your example code only applies to ScrollPositionWithSingleContext?
What are the situations where a different ScrollPosition is used, and why are those situations not candidates for the scrolling behavior in SuperEditor?

What I ended up with is:
Listener(
      onPointerSignal: _scrollOnMouseWheel,
      onPointerHover: (event) => _cancelScrollMomentum(),
      onPointerDown: (event) => _cancelScrollMomentum(),
      child: ...,
    )

I'm finding that scrolling is janky. I don't know what's causing that. I checked the performance monitor and we're not skipping any frames. I think the jank is too minor to show in a video, but it's noticeable when interacting with SuperEditor.

@moffatman we're still working on a fix for super_editor. We're having a discussion over at https://github.com/superlistapp/super_editor/pull/820#discussion_r993865559
The question we're dealing with right now is how to respond to a single finger making contact with the trackpad. @tannermeade has suggested that a single finger placed on a trackpad should stop scrolling. However, we haven't found any place to receive such a signal. There doesn't seem to be any callback in Listener that notifies us of a single finger touch. Even an onTapDown in a GestureRecognizer isn't triggered.
Do you know how we're expected to respond to a single finger making contact with a trackpad?

@matthew-carroll
Yeah that single finger tap/touch is not a real click, it doesn't map to anything besides cancelling scrolling, so no way to get it in Flutter 3.3. That's the purpose of PointerScrollInertiaCancelEvent in Flutter 3.4+.

Ok. I merged the current work in super_editor and filed a ticket for 3.4: https://github.com/superlistapp/super_editor/issues/824
You mentioned that this signal will come as a PointerScrollInertiaCancelEvent. Does this mean that Flutter isn't forwarding the touch event, and instead Flutter is interpreting the meaning of the touch before it gets to developers? Or is there a touch signal, and on top of that there's also an interpreted event to cancel scrolling inertia?

There won't be any touch event, only the PointerScrollInertiaCancelEvent. I think the official/HIG name for it in AppKit is a "resting touch", but documentation is very scarce. It shouldn't have any meaning other than cancelling the scroll inertia. See the implementation here, I had to add some new handlers to the view-controller, it's not a reinterpretation of existing data: https://github.com/flutter/engine/pull/34451
Note that you will receive PointerScrollInertiaCancelEvent in other cases too, like before a trackpad click or another scroll. So the onPointerHover, onPointerDown, and onPointerPanZoomStart handlers won't be needed then.

It shouldn't have any meaning other than cancelling the scroll inertia

That's an application decision, not a toolkit decision. Unless Mac OS literally reports a "cancel inertial event", then I don't think it's appropriate for Flutter to draw that conclusion.
If, as you say, it's really a "resting touch", then Flutter should let us know that a "resting touch event" has occurred. Flutter's scrolling widgets can then interpret the event to mean "cancel inertia". Developers with other use-cases can interpret that same event in ways that might be more appropriate for their application use-cases.

The PointerScrollInertiaCancelEvent is generated with very different methods in different platforms. Only macOS has a clean way to get the resting touch events. On other platforms, I had to use some tricks to figure out when scroll momentum should be cancelled which can’t be used at other times. So PointerScrollInertiaCancelEvent needs to stay as a scrolling-specific event. But we can add some new events if you think it’s useful to know about that resting touch on its own on macOS. The macOS trackpad two-finger double-tap could also be added which does have a generally expected application meaning (zoom-to-fit).

But we can add some new events if you think it’s useful to know about that resting touch on its own on macOS

In general, my philosophy towards Flutter and platforms is that Flutter should never hide a platform signal that Flutter receives. It's fine for other areas of Flutter to build on top. It's fine if Flutter wants to also dispatch a PointerScrollInertiaCancelEvent for those specific use-cases. But if Flutter gets to receive a RestingTouchEvent, then Flutter should not prevent that information from broadcasting to packages and apps, which may have any number of reasons to react to it.

I was able to create a generic widget that "resembles" the old behavior with the code at the bottom of this comment.
It is based on the example that @moffatman gave in https://github.com/flutter/flutter/issues/112880#issuecomment-1272091219.
However, the momentum does not feel native, and I can't wrap my head around the correctness of that example because there appears to be so many edge cases that need to be handled across all platforms, something that, I suppose, only somebody that has worked on that part of Flutter could truly reason about in a sound way.
Scrolling has always felt slightly off with Flutter, especially on Flutter Web.
Instead of letting everybody reinvent the wheel, I think, it would be great to simply ship a general purpose widget with the semantics of "This widgets tries to simulate the scrolling behavior that users expect for each platform" and to work on improving that so that everybody can benefit from it.
What's preventing us from having a first party general purpose widget (and/or other low level mechanisms) that exposes a "momentum callback"?
class MomentumScrolled extends StatefulWidget {
  final void Function(Offset) addDelta;
  final Widget child;

  const MomentumScrolled({
    required this.addDelta,
    required this.child,
  });

  @override
  _MomentumScrolledState createState() => _MomentumScrolledState();
}

class _MomentumScrolledState extends State<MomentumScrolled> with TickerProviderStateMixin {
  DateTime trackingStartTime = DateTime.now();
  VelocityTracker? tracker;
  Simulation? simulationX;
  Simulation? simulationY;
  double simulationTime = 0;
  Timer? timer;
  static const simulatedInputHz = 120;
  static const duration = Duration(milliseconds: 1000 ~/ simulatedInputHz);
  bool isTrackpad = false;

  void _tick(
    final Timer t,
  ) {
    if (simulationX != null && simulationY != null) {
      final x0 = simulationX!.x(simulationTime);
      final y0 = simulationY!.x(simulationTime);
      simulationTime += duration.inMicroseconds / 1e6;
      final x1 = simulationX!.x(simulationTime);
      final y1 = simulationY!.x(simulationTime);
      final diff = Offset(x1 - x0, y1 - y0);
      if (diff == Offset.zero && simulationTime > 0) {
        t.cancel();
      } else {
        widget.addDelta(diff);
      }
    }
  }

  @override
  Widget build(
    final BuildContext context,
  ) {
    return Listener(
      behavior: HitTestBehavior.translucent,
      onPointerSignal: (final event) {
        print("YO SIGNAL $event");
        // Flutter 3.4+ only
        if (event is PointerScrollInertiaCancelEvent) {
          timer?.cancel();
        }
      },
      onPointerPanZoomStart: (final event) {
        print("YO panzoomstart $event");
        isTrackpad = event.kind == PointerDeviceKind.trackpad;
        if (!isTrackpad) return;
        timer?.cancel();
        tracker = VelocityTracker.withKind(event.kind);
        trackingStartTime = DateTime.now();
      },
      onPointerPanZoomUpdate: (final event) {
        print("YO panzoomupdate $event");
        if (!isTrackpad) return;
        setState(() {
          widget.addDelta(event.panDelta);
        });
        tracker?.addPosition(DateTime.now().difference(trackingStartTime), event.pan);
      },
      onPointerPanZoomEnd: (final event) {
        print("YO panzoomend $event");
        if (!isTrackpad) return;
        final metrics = FixedScrollMetrics(
            devicePixelRatio: MediaQuery.of(context).devicePixelRatio,
            minScrollExtent: 0,
            maxScrollExtent: 99999,
            pixels: 50000,
            viewportDimension: 99999,
            axisDirection: AxisDirection.down
        );
        const physics = BouncingScrollPhysics(
          // Flutter 3.4+ only
            decelerationRate: ScrollDecelerationRate.fast
        );
        simulationX = physics.createBallisticSimulation(
            metrics,
            tracker!.getVelocity().pixelsPerSecond.dx
        );
        simulationY = physics.createBallisticSimulation(
            metrics,
            tracker!.getVelocity().pixelsPerSecond.dy
        );
        simulationTime = 0;
        timer?.cancel();
        timer = Timer.periodic(duration, _tick);
      },
      child: widget.child,
    );
  }

  @override
  void dispose() {
    super.dispose();
    timer?.cancel();
  }
}


Create dfbench, split up tpch benchmark runner into modules

Note this looks like a big PR but it just moves code around. I will move the other runners into this new format as follow on PRs
Which issue does this PR close?
Part of https://github.com/apache/arrow-datafusion/issues/7052
related to https://github.com/apache/arrow-datafusion/issues/6994
Rationale for this change
I want to have a nice home for clickbench tests https://github.com/apache/arrow-datafusion/issues/6994 but the current situation in the benchmark runners is a mess
What changes are included in this PR?

Create dfbench entrypoint
split up tpch benchmark runner code into moudles

This PR also retain the old entrypoints for backwards compatibility, so the following two commands do the same thing
Old
cargo run  --bin tpch -- benchmark datafusion --iterations 5 -m --format parquet -q 1 --path data/tpch_sf1

New
cargo run  --bin dfbench -- tpch --iterations 5 -m --format parquet -q 1 --path data/tpch_sf1

Are these changes tested?
existing coverage
Are there any user-facing changes?

cc @Dandandan  / @andygrove  in case you have time to review this code. I have big plans but this one just moves things around

Looks good to me :)


Complete Documentation

Our project has become so large in terms of lines and code / number of files, and also in terms of how many (new) developers we have working together, so we require high quality documentation in order for this project to live on. This is what I try to provide here.
I'd really love it if you guys could give me a thorough code review, both in content but also grammar etc. as clarity of writing makes a big difference in how helpful the documentation is.
Look forward to having this in our project!

Just pushed a pretty significant update as I realized I had missed a few things, if you had already looked and want to check just the changes see here: https://github.com/thegazelle-ad/gazelle-server/pull/419/files/da18ff7..f809ece

Okay then... I couldn't help myself and documented our directory structure haha https://github.com/thegazelle-ad/gazelle-server/pull/419/files/f809ece634b81757db82733d271b44e85c5db92b..1b14f49f72351e0f783e7d7f0b3ab0db0959657d#diff-3f26975f3b01109ec051c12794a52722

Yeah thanks @navyasuri ! That was really helpful. I added in one of your resources that I really liked and deleted some of the less helpful ones for Promises / async/await. In terms of videos, yeah of course people learn differently, so feel free to add some videos to the documentation later. I personally prefer written content myself as I feel it's faster.
If you can approve it (or give more comments) I think I'll just merge it. Also it's just documentation so it's not as important that it's perfect(as compared to code that might break in production) as it can be iterated on

I think it's good to go!
The best part about documentation is we can always add to it when we find something that could be useful to everone else!

The best part about documentation is we can always add to it when we find something that could be useful to everone else!

Yes exactly, and I hope to encourage a culture of doing that, especially with the PR template.
@navyasuri could you just officially "approve" it through the PR interface? Then I don't need to use my admin privileges to override our Github settings to merge it :)


Exponential backoff for sleeping during update()

This seems to fix issue #3.

Looks great; thanks!


Published projects - spectra display is missing from NMRium

In the production environment, in the published projects, the spectra are not displayed in NMRium. Example: https://nmrxiv.org/S81


Path Manipulation in writeFile

The filename argument passed to the writeFile() function in file-util.js may be manipulated to write arbitrary directories on the filesystem. The writeFile() function attempts to sanitize the untrusted filename, but calls the mkdirRecursive() function on the untrusted filename before the filename is sanitized (https://github.com/google/traceur-compiler/blob/master/src/node/file-util.js#L56)
For example, passing the filename "../test/target.txt" on a *nix system will create the directory "test" in the parent directory. An attacker can use the dot-dot-slash path manipulation technique to create directories anywhere on the filesystem.
function writeFile(filename, contents) {
  // Compute the output path
  var outputdir = fs.realpathSync(process.cwd());
  mkdirRecursive(path.dirname(filename));
  var filedir = fs.realpathSync(path.dirname(filename));
  filedir = removeCommonPrefix(outputdir, filedir);
  outputdir = path.join(outputdir, filedir);

  mkdirRecursive(outputdir);
  var outputfile = path.join(outputdir, path.basename(filename));
  fs.writeFileSync(outputfile, contents, 'utf8');
}

Not sure what you are suggesting here?
I think it is important to allow paths starting with .. (or / for that matter). If you are accepting paths from an untrusted source you might want to consider checking the path before handing it to Traceur.

Maybe I should clarify. If I submit the filename "../test/target.txt" and my current directory is "/tmp/traceur", the file is written to "/tmp/traceur/test/target.txt". However, the writeFile() function ALSO creates the empty directory "/tmp/test". This appears to be a bug that allows the user to write this "duplicate" directory structure anywhere on the filesystem, when the function is attempting to only write to the current directory. Regardless, this "duplicate" directory is unintended, is it not?

I see. Yeah. That seems like a bug. The intent is to create the directory where the output files/dir are going.

Can someone please check and help merge below PR

https://github.com/google/traceur-compiler/pull/2154


Add Document Symbol Provider sample

This is a forgotten one but is very necessary to help extension developers.

Thanks @dhonx @jrieken
I can confirm that it would be very nice to have an example.
I have tried to use the DocumentSymbol type but to no avail.
Using the old SymbolInformation worked well but of course lacks hierarchy.
There are a few resources on the topic including this one but it did not work for me (and I did not manage to debug it even by logging extra info in the console).

I need this sample very badly too. the vscode.executeDocumentSymbolProvider currently do not work as I expected when it's not responding when I return it in provideDocumentSymbols :(

hi all, is there any chance we have this example please :( I need this for hightlighting our own keyword. thanks a lot


checkov: 3.2.231 -> 3.2.233

Diff: https://github.com/bridgecrewio/checkov/compare/refs/tags/3.2.231...3.2.233
Changelog: https://github.com/bridgecrewio/checkov/releases/tag/3.2.233
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 336010 run on x86_64-linux 1

2 packages built:

checkov
checkov.dist


⚠ Bofrost.at has degraded performance

In 8f23689, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1763 ms

Resolved: Bofrost.at performance has improved in 33dbf35 after 6 minutes.


BUG: _compare_version backward incompat

cc @hoechenberger:
>>> import mne
>>> mne.fixes._compare_version('1.0.dev0', '>=', '1.0')
False
>>> from distutils.version import LooseVersion
>>> LooseVersion('1.0.dev0') >= LooseVersion('1.0')
<stdin>:1: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
True

This is a problem because we usually want the second result :(
The git grep check_version | wc -l  count is 67 and git grep _compare_version | wc -l is 26, I'd rather not have to change all of these to allow for .dev0 extensions. Plus other submodules like mne-nirs that already use this (where I saw the problem actually!) will not work properly...
I'm tempted to change this to cut off any .dev from the end to make life easy, but then we depart from standard. But it seems a bit like the lesser of two evils here.

I believe there were some relatively recent changes by @cbrnr if I'm not mistaken (cannot check right now), could you have a look at the history of that function and see if this problem maybe only started showing up after these changes?

This is a bit unfortunate and maybe even a bug? They are explicitly recommending to switch to packaging.version.parse, but it doesn't behave identically. I've open an issue.

I have to say that IMHO the new behavior is more correct than the old one though. 1.0 is newer than 1.0dev0. So the version comparison should be >=1.0dev0, no? I think this is how conda behaves too if I'm not mistaken.

Technically yes, dev0 is a prerelease, so it is less than the final release. However, speaking of semantic versioning, I think we're not using those prerelease tags correctly, because they should be separated by a hyphen (e.g. 1.0.0-dev0).

As @hoechenberger suspected this is expected behavior, and distutils.version was doing its own thing. I'd change the comparisons in our code. Alternatively, we could vendor the old versioning code, but I'd rather not do that.

Let's change our code to make it more consistent with my thinking 😅🥴😁

I'd rather add a loose=True that prunes the dev. Most of the time if a person is using 1.0.dev0, they probably have what's needed and if they don't they know how to update. Plus it's backward compatible this way. And it's much easier than changing 100+ places we use (and think about) these comparisons.

So this would basically affect equality comparisons, right? Using loose=True would mean that 1.0.dev0 >= 1.0, 1.0.dev0 <= 1.0, and 1.0.dev0 == 1.0 are all True. Does this make sense? I don't really understand why we rely on such strange comparisons.

I can document this inaccuracy with == and loose=True. In practice we (almost?) never use ==. To me practicality -- and backward compat for other libs and even users that make use of check_version and have silently been getting the wrong result, at least according to how this has behaved for the last 9 years -- beats purity here by a long shot.

The most important thing is that it restores the behavior of check_version. This uses _compare_version under the hood with a < operator to say if something is not okay:
https://github.com/mne-tools/mne-python/blob/6608b720b214529732542cb2aac11b401770eddf/mne/utils/check.py#L100
So for example:
>>> LooseVersion('1.0.dev0') < LooseVersion('1.0')
False
>>> mne.fixes._compare_version('1.0.dev0', '<', '1.0')
True

So I think it should work. So maybe we should add loose=True to check_version, do the stripping there, and not change _compare_version (and just fix the way fewer of those we use, where needed).

So maybe we should add loose=True to check_version, do the stripping there, and not change _compare_version (and just fix the way fewer of those we use, where needed).

This sounds good!


Create #658 - WinGet Download.md

[x] I have signed the Contributor License Agreement.
[x] This pull request is related to an issue.

Microsoft Reviewers: Open in CodeFlow

I would think that the ability to download only would make more sense in the context of cache management. This discussion around cache management even includes the Download only request. I think we should assess if cache management makes more sense here to decrease code complexity

We had some internal discussions about this. Since we already moved installer download to temp, there's not a lot more benefit of doing cache management. For customers wanting to do offline installation, we think explicit winget download is the way to go. We had customer asks that they need to manage a group of devices with limited network access, so winget download would be an option for this.

By the way, I just thought of something, do we deal with dependency packages for download? If yes, what will be the behavior and layout (folder structure, filename, etc)

By the way, I just thought of something, do we deal with dependency packages for download? If yes, what will be the behavior and layout (folder structure, filename, etc)

We will need to handle dependency packages.

Need to include an option for only downloading the apps and not the license file.
Does having a similar flow for License file only make sense?

Does having a similar flow for License file only make sense?

No, licensing requires a specific package (content id), licensing call must come after download call.

Does having a similar flow for License file only make sense?

No, licensing requires a specific package (content id), licensing call must come after download call.

I was thinking about the scenario of when the app downloads but the license file fails to download. Should the administrator then re-download the app again to get the license file.

Thinking about the use of --download-directory, would it make more sense for us to use "--Location", or "--Path"?

Another thought, what if we include a new parameter "--continueOnError" that when applied would continue the download of the main and dependencies even after a single dependency fails.

I'd propose a more generic -ErrorAction like PowerShell's implementation where WinGet would respect stop, continue, silentlyContinue, inquire, ignore and maybe even suspend. This would probably make the PowerShell cmdlet implementation and could also be useful in other areas too. It could replace the IgnoreUnavailable argument for the import flow, could be used to set the behavior of upgrade all, could be used to continue when depenency installation fails, and would be a great option as a generic parameter for scripting

Another thought, what if we include a new parameter "--continueOnError" that when applied would continue the download of the main and dependencies even after a single dependency fails.

I'd propose a more generic -ErrorAction like PowerShell's implementation where WinGet would respect stop, continue, silentlyContinue, inquire, ignore and maybe even suspend. This would probably make the PowerShell cmdlet implementation and could also be useful in other areas too. It could replace the IgnoreUnavailable argument for the import flow, could be used to set the behavior of upgrade all, could be used to continue when depenency installation fails, and would be a great option as a generic parameter for scripting

I like this idea of having a parameter that can provide a deterministic experience for the user, that allows the user to determine what they want the experience to be.

I like this idea of having a parameter that can provide a deterministic experience for the user, that allows the user to determine what they want the experience to be.

If we are to replace existing parameters with new ErrorAction parameter, it will be a breaking change. And needs to go through all existing workflows to ensure the experience is accurate. As well as all Com and Powershell work. I would suggest making this a different work item not attached to this download work. For download, I think using --force or a specific flag is good for the purpose.
We also need to think about each different error case by case, some errors are hard stop (installer corrupt, etc), some errors (like dependency download failure for winget download) may let the workflow continue.
By the way, the reason I would like dependency download to fail the workflow is for consistency. Looks like in winget install flow, a dependency download failure will cause main package to fail to install. In winget download flow, a dependency download failure may be ok if user desires.

I'll make a separate issue for the proposed ErrorAction

I guess the download command got implemented, and the spec never made to the repo.

Yep, we just noticed yesterday and Ryan is working on updating it to match the final implementation. Thanks.


Revisit client side cluster related code

One more thing: this part does not look right to me.... Why are we generating things for client sides of clusters here?
Let's leave this as-is for now, but do a followup to not generate this... and probably not generate ClusterRevision for client clusters either, since it's not observable...
Originally posted by @bzbarsky-apple in https://github.com/project-chip/connectedhomeip/pull/14299#discussion_r794022110
ZAP will generate meta for client cluster, but the client should not have cluster data per spec, should revisit related code in code base.

We need some metadata for client clusters, because Descriptor Cluster needs to be able to show info about them existing, right?
What we should not be generating are:

ClusterRevision attributes for client clusters, with space in the attr store and all that.
CommandList metadata bits for client clusters.

And I guess we could try to handle the Descriptor bits via some mechanism other than EmberAfCluster entries, but that will take a bit more surgery, whereas not generating the above two is an easy space win.

Fixed in  #14677


[feat] Windows Virtual Desktop

Describe the problem
Switched to a new desktop, unable to bring the Tauri application over.
Describe the solution you'd like
_
Alternatives considered
No response
Additional context
No response

That is needed to be done manually by the user.

That is needed to be done manually by the user.

Anyways, your issue lacks enough information so I will close this for now but feel free to ping me with more info if you feel I missed something

I've actually written a floating window, and I hope it can be displayed on all virtual desktops. @amrbashir

I see, are you using skipTaskbar: true ? it should work when you enable that option.

I'm sorry, I forgot that. I'm sorry to disturb you.


⚠ Bofrost.at has degraded performance

In 59b912f, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1591 ms

Resolved: Bofrost.at performance has improved in f406789.


ETCD2 Won't start on coreos latest

Jun 07 12:46:24 x35b sudo[2483]:     core : TTY=pts/0 ; PWD=/home/core ; USER=root ; COMMAND=/bin/systemctl status etcd2.service
Jun 07 12:46:29 x35b sudo[2484]:     core : TTY=pts/0 ; PWD=/home/core ; USER=root ; COMMAND=/bin/systemctl restart etcd2.service
Jun 07 12:46:29 x35b polkitd[1589]: Registered Authentication Agent for unix-process:2484:88033 (system bus name :1.43 [/usr/bin/pkttyagent --notify-fd 5 --fallback], object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8
Jun 07 12:46:29 x35b systemd[1]: Stopped etcd2.
-- Subject: Unit etcd2.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit etcd2.service has finished shutting down.
Jun 07 12:46:29 x35b systemd[1]: Starting etcd2...
-- Subject: Unit etcd2.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit etcd2.service has begun starting up.
Jun 07 12:46:29 x35b etcd2[2489]: recognized and used environment variable ETCD_ADVERTISE_CLIENT_URLS=http://<IP_ADDRESS>:2379
Jun 07 12:46:29 x35b etcd2[2489]: recognized and used environment variable ETCD_DATA_DIR=/var/lib/etcd2
Jun 07 12:46:29 x35b etcd2[2489]: recognized and used environment variable ETCD_INITIAL_ADVERTISE_PEER_URLS=http://<IP_ADDRESS>:2380
Jun 07 12:46:29 x35b etcd2[2489]: recognized and used environment variable ETCD_LISTEN_CLIENT_URLS=http://<IP_ADDRESS>:2379
Jun 07 12:46:29 x35b etcd2[2489]: recognized and used environment variable ETCD_LISTEN_PEER_URLS=http://<IP_ADDRESS>:2380
Jun 07 12:46:29 x35b etcd2[2489]: recognized and used environment variable ETCD_NAME=f4320093d8b741fe8afbab7f40242066
Jun 07 12:46:29 x35b etcd2[2489]: etcd Version: 2.3.1
Jun 07 12:46:29 x35b etcd2[2489]: Git SHA: 2b67f52
Jun 07 12:46:29 x35b etcd2[2489]: Go Version: go1.5.3
Jun 07 12:46:29 x35b etcd2[2489]: Go OS/Arch: linux/amd64
Jun 07 12:46:29 x35b etcd2[2489]: setting maximum number of CPUs to 16, total number of available CPUs is 16
Jun 07 12:46:29 x35b etcd2[2489]: listening for peers on http://<IP_ADDRESS>:2380
Jun 07 12:46:29 x35b etcd2[2489]: listening for client requests on http://<IP_ADDRESS>:2379
Jun 07 12:46:29 x35b etcd2[2489]: stopping listening for client requests on http://<IP_ADDRESS>:2379
Jun 07 12:46:29 x35b etcd2[2489]: stopping listening for peers on http://<IP_ADDRESS>:2380
Jun 07 12:46:29 x35b etcd2[2489]: --initial-cluster must include f4320093d8b741fe8afbab7f40242066=http://<IP_ADDRESS>:2380 given --initial-advertise-peer-urls=http://<IP_ADDRESS>:2380
Jun 07 12:46:29 x35b etcd2[2489]: forgot to set --initial-cluster flag?
Jun 07 12:46:29 x35b etcd2[2489]: if you want to use discovery service, please set --discovery flag.
Jun 07 12:46:29 x35b systemd[1]: etcd2.service: Main process exited, code=exited, status=1/FAILURE
Jun 07 12:46:29 x35b systemd[1]: Failed to start etcd2.
-- Subject: Unit etcd2.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit etcd2.service has failed.
--

i have tried changing the discovery url and everything i have no clue why this isn't starting

@0x4139 Can you please provide your etcd configuration? It seems that discovery-token is not provided from the log.

I'm not sure if this is related or not but I also cannot start etcd2 on 1010.5.0.
journalctl -u etcd2
-- Reboot --
Jun 11 13:12:08 spengler.ghostbusters.sheltek.us systemd[1]: Starting etcd2...
Jun 11 13:12:08 spengler.ghostbusters.sheltek.us systemd[1]: etcd2.service: Control process exited, code=exited status=2
Jun 11 13:12:08 spengler.ghostbusters.sheltek.us systemd[1]: Failed to start etcd2.
Jun 11 13:12:08 spengler.ghostbusters.sheltek.us systemd[1]: etcd2.service: Unit entered failed state.
Jun 11 13:12:08 spengler.ghostbusters.sheltek.us systemd[1]: etcd2.service: Failed with result 'exit-code'.
Jun 11 13:12:18 spengler.ghostbusters.sheltek.us systemd[1]: etcd2.service: Service hold-off time over, scheduling restart.
Jun 11 13:12:18 spengler.ghostbusters.sheltek.us systemd[1]: Stopped etcd2.
Jun 11 13:12:18 spengler.ghostbusters.sheltek.us systemd[1]: Starting etcd2...
Jun 11 13:12:18 spengler.ghostbusters.sheltek.us systemd[1]: etcd2.service: Control process exited, code=exited status=2
Jun 11 13:12:18 spengler.ghostbusters.sheltek.us systemd[1]: Failed to start etcd2.
Jun 11 13:12:18 spengler.ghostbusters.sheltek.us systemd[1]: etcd2.service: Unit entered failed state.
Jun 11 13:12:18 spengler.ghostbusters.sheltek.us systemd[1]: etcd2.service: Failed with result 'exit-code'.
Jun 11 13:12:29 spengler.ghostbusters.sheltek.us systemd[1]: etcd2.service: Service hold-off time over, scheduling restart.
Jun 11 13:12:29 spengler.ghostbusters.sheltek.us systemd[1]: Stopped etcd2.
Jun 11 13:12:29 spengler.ghostbusters.sheltek.us systemd[1]: Starting etcd2...
Jun 11 13:12:29 spengler.ghostbusters.sheltek.us systemd[1]: etcd2.service: Control process exited, code=exited status=2
Jun 11 13:12:29 spengler.ghostbusters.sheltek.us systemd[1]: Failed to start etcd2.
Jun 11 13:12:29 spengler.ghostbusters.sheltek.us systemd[1]: etcd2.service: Unit entered failed state.
Jun 11 13:12:29 spengler.ghostbusters.sheltek.us systemd[1]: etcd2.service: Failed with result 'exit-code'.
Jun 11 13:12:39 spengler.ghostbusters.sheltek.us systemd[1]: etcd2.service: Service hold-off time over, scheduling restart.
Jun 11 13:12:39 spengler.ghostbusters.sheltek.us systemd[1]: Stopped etcd2.
Jun 11 13:12:39 spengler.ghostbusters.sheltek.us systemd[1]: Starting etcd2...
Jun 11 13:12:39 spengler.ghostbusters.sheltek.us systemd[1]: etcd2.service: Control process exited, code=exited status=2
Jun 11 13:12:39 spengler.ghostbusters.sheltek.us systemd[1]: Failed to start etcd2.
Jun 11 13:12:39 spengler.ghostbusters.sheltek.us systemd[1]: etcd2.service: Unit entered failed state.
Jun 11 13:12:39 spengler.ghostbusters.sheltek.us systemd[1]: etcd2.service: Failed with result 'exit-code'.
Jun 11 13:12:49 spengler.ghostbusters.sheltek.us systemd[1]: etcd2.service: Service hold-off time over, scheduling restart.
Jun 11 13:12:49 spengler.ghostbusters.sheltek.us systemd[1]: Stopped etcd2.
Jun 11 13:12:49 spengler.ghostbusters.sheltek.us systemd[1]: Starting etcd2...
Jun 11 13:12:49 spengler.ghostbusters.sheltek.us systemd[1]: etcd2.service: Control process exited, code=exited status=2
Jun 11 13:12:49 spengler.ghostbusters.sheltek.us systemd[1]: Failed to start etcd2.
Jun 11 13:12:49 spengler.ghostbusters.sheltek.us systemd[1]: etcd2.service: Unit entered failed state.
Jun 11 13:12:49 spengler.ghostbusters.sheltek.us systemd[1]: etcd2.service: Failed with result 'exit-code'.
```

and cloud-config looks like this:

coreos:
etcd2:
name: spengler
discovery: https://discovery.etcd.io/4c7ad265bd34ff638a021e401a85218f
advertise-client-urls: http://<IP_ADDRESS>:2379,http://<IP_ADDRESS>:4001
initial-advertise-peer-urls: http://<IP_ADDRESS>:2380
listen-client-urls: http://<IP_ADDRESS>:2379,http://<IP_ADDRESS>:4001
listen-peer-urls: http://<IP_ADDRESS>:2380
units:
- name: etcd2.service
command: start
drop-ins:
- name: 50-network-config.conf
content: |
[Service]
ExecStartPre=/usr/bin/etcdctl set /coreos.com/network/config '{"Network":"<IP_ADDRESS>/16", "Backend": {"Type": "vxlan"}}'

Starting etcd2 by hand (or passing the parameters in by hand, rather) goes like this..

spengler ~ # !18
ETCD_DATA_DIR=/var/lib/etcd2 etcd2 --discovery https://discovery.etcd.io/0e7c08f5671e126e67b503df2adde3be --advertise-client-urls http://<IP_ADDRESS>:2379,http://<IP_ADDRESS>:4001 --initial-advertise-peer-urls http://<IP_ADDRESS>:2380 --listen-client-urls http://<IP_ADDRESS>:2379,http://<IP_ADDRESS>:4001 --listen-peer-urls http://<IP_ADDRESS>:2380 --name spengler
2016-06-12 02:30:34.040694 I | flags: recognized and used environment variable ETCD_DATA_DIR=/var/lib/etcd2
2016-06-12 02:30:34.040847 I | etcdmain: etcd Version: 2.3.1
2016-06-12 02:30:34.040857 I | etcdmain: Git SHA: 2b67f52
2016-06-12 02:30:34.040863 I | etcdmain: Go Version: go1.5.3
2016-06-12 02:30:34.040870 I | etcdmain: Go OS/Arch: linux/amd64
2016-06-12 02:30:34.040876 I | etcdmain: setting maximum number of CPUs to 16, total number of available CPUs is 16
2016-06-12 02:30:34.040935 N | etcdmain: the server is already initialized as member before, starting as etcd member...
2016-06-12 02:30:34.040988 I | etcdmain: listening for peers on http://<IP_ADDRESS>:2380
2016-06-12 02:30:34.041025 I | etcdmain: listening for client requests on http://<IP_ADDRESS>:2379
2016-06-12 02:30:34.041055 I | etcdmain: listening for client requests on http://<IP_ADDRESS>:4001
2016-06-12 02:30:34.057009 W | etcdserver: discovery token ignored since a cluster has already been initialized. Valid log found at "/var/lib/etcd2/member/wal"
2016-06-12 02:30:34.057039 I | etcdserver: name = spengler
2016-06-12 02:30:34.057048 I | etcdserver: data dir = /var/lib/etcd2
2016-06-12 02:30:34.057056 I | etcdserver: member dir = /var/lib/etcd2/member
2016-06-12 02:30:34.057064 I | etcdserver: heartbeat = 100ms
2016-06-12 02:30:34.057076 I | etcdserver: election = 1000ms
2016-06-12 02:30:34.057084 I | etcdserver: snapshot count = 10000
2016-06-12 02:30:34.057092 I | etcdserver: discovery URL= https://discovery.etcd.io/0e7c08f5671e126e67b503df2adde3be
2016-06-12 02:30:34.057116 I | etcdserver: advertise client URLs = http://<IP_ADDRESS>:2379,http://<IP_ADDRESS>:4001
2016-06-12 02:30:34.120314 I | etcdserver: restarting member c23abd567f2f8c7e in cluster e73568347adcdecd at commit index 6956
2016-06-12 02:30:34.120543 I | raft: c23abd567f2f8c7e became follower at term 2
2016-06-12 02:30:34.120564 I | raft: newRaft c23abd567f2f8c7e [peers: [], term: 2, commit: 6956, applied: 0, lastindex: 6956, lastterm: 2]
2016-06-12 02:30:34.123676 I | etcdserver: starting server... [version: 2.3.1, cluster version: to_be_decided]
2016-06-12 02:30:34.124044 E | etcdmain: failed to notify systemd for readiness: No socket
2016-06-12 02:30:34.124060 E | etcdmain: forgot to set Type=notify in systemd service file?
2016-06-12 02:30:34.136309 N | etcdserver: added member 950c7a0679d18d1 [http://<IP_ADDRESS>:2380] to cluster e73568347adcdecd
2016-06-12 02:30:34.136922 N | etcdserver: added member 2cf696424304f954 [http://<IP_ADDRESS>:2380] to cluster e73568347adcdecd
2016-06-12 02:30:34.150970 N | etcdserver: added member 97757695ec6e6be0 [http://<IP_ADDRESS>:2380] to cluster e73568347adcdecd
2016-06-12 02:30:34.151108 N | etcdserver: added local member c23abd567f2f8c7e [http://<IP_ADDRESS>:2380] to cluster e73568347adcdecd
2016-06-12 02:30:34.151556 N | etcdserver: set the initial cluster version to 2.3
2016-06-12 02:30:35.423983 I | raft: c23abd567f2f8c7e is starting a new election at term 2
2016-06-12 02:30:35.424079 I | raft: c23abd567f2f8c7e became candidate at term 3
2016-06-12 02:30:35.424094 I | raft: c23abd567f2f8c7e received vote from c23abd567f2f8c7e at term 3
2016-06-12 02:30:35.424131 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 950c7a0679d18d1 at term 3
2016-06-12 02:30:35.424145 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 2cf696424304f954 at term 3
2016-06-12 02:30:35.424160 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 97757695ec6e6be0 at term 3
2016-06-12 02:30:36.723939 I | raft: c23abd567f2f8c7e is starting a new election at term 3
2016-06-12 02:30:36.723986 I | raft: c23abd567f2f8c7e became candidate at term 4
2016-06-12 02:30:36.724003 I | raft: c23abd567f2f8c7e received vote from c23abd567f2f8c7e at term 4
2016-06-12 02:30:36.724020 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 950c7a0679d18d1 at term 4
2016-06-12 02:30:36.724036 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 2cf696424304f954 at term 4
2016-06-12 02:30:36.724071 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 97757695ec6e6be0 at term 4
2016-06-12 02:30:38.023935 I | raft: c23abd567f2f8c7e is starting a new election at term 4
2016-06-12 02:30:38.023982 I | raft: c23abd567f2f8c7e became candidate at term 5
2016-06-12 02:30:38.024007 I | raft: c23abd567f2f8c7e received vote from c23abd567f2f8c7e at term 5
2016-06-12 02:30:38.024037 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 2cf696424304f954 at term 5
2016-06-12 02:30:38.024057 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 97757695ec6e6be0 at term 5
2016-06-12 02:30:38.024078 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 950c7a0679d18d1 at term 5
2016-06-12 02:30:39.223873 I | raft: c23abd567f2f8c7e is starting a new election at term 5
2016-06-12 02:30:39.223918 I | raft: c23abd567f2f8c7e became candidate at term 6
2016-06-12 02:30:39.223933 I | raft: c23abd567f2f8c7e received vote from c23abd567f2f8c7e at term 6
2016-06-12 02:30:39.223952 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 950c7a0679d18d1 at term 6
2016-06-12 02:30:39.223971 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 2cf696424304f954 at term 6
2016-06-12 02:30:39.223989 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 97757695ec6e6be0 at term 6
2016-06-12 02:30:40.523901 I | raft: c23abd567f2f8c7e is starting a new election at term 6
2016-06-12 02:30:40.523969 I | raft: c23abd567f2f8c7e became candidate at term 7
2016-06-12 02:30:40.523982 I | raft: c23abd567f2f8c7e received vote from c23abd567f2f8c7e at term 7
2016-06-12 02:30:40.523996 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 950c7a0679d18d1 at term 7
2016-06-12 02:30:40.524008 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 2cf696424304f954 at term 7
2016-06-12 02:30:40.524020 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 97757695ec6e6be0 at term 7
2016-06-12 02:30:41.124121 E | etcdserver: publish error: etcdserver: request timed out
2016-06-12 02:30:41.353078 E | etcdhttp: got unexpected response error (etcdserver: request timed out)
2016-06-12 02:30:41.923952 I | raft: c23abd567f2f8c7e is starting a new election at term 7
2016-06-12 02:30:41.924012 I | raft: c23abd567f2f8c7e became candidate at term 8
2016-06-12 02:30:41.924030 I | raft: c23abd567f2f8c7e received vote from c23abd567f2f8c7e at term 8
2016-06-12 02:30:41.924051 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 950c7a0679d18d1 at term 8
2016-06-12 02:30:41.924064 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 2cf696424304f954 at term 8
2016-06-12 02:30:41.924074 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 97757695ec6e6be0 at term 8
2016-06-12 02:30:43.223947 I | raft: c23abd567f2f8c7e is starting a new election at term 8
2016-06-12 02:30:43.223994 I | raft: c23abd567f2f8c7e became candidate at term 9
2016-06-12 02:30:43.224010 I | raft: c23abd567f2f8c7e received vote from c23abd567f2f8c7e at term 9
2016-06-12 02:30:43.224025 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 950c7a0679d18d1 at term 9
2016-06-12 02:30:43.224037 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 2cf696424304f954 at term 9
2016-06-12 02:30:43.224049 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 97757695ec6e6be0 at term 9
2016-06-12 02:30:43.304928 E | etcdhttp: got unexpected response error (etcdserver: request timed out)
2016-06-12 02:30:44.524055 I | raft: c23abd567f2f8c7e is starting a new election at term 9
2016-06-12 02:30:44.524120 I | raft: c23abd567f2f8c7e became candidate at term 10
2016-06-12 02:30:44.524136 I | raft: c23abd567f2f8c7e received vote from c23abd567f2f8c7e at term 10
2016-06-12 02:30:44.524152 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 950c7a0679d18d1 at term 10
2016-06-12 02:30:44.524176 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 2cf696424304f954 at term 10
2016-06-12 02:30:44.524196 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 97757695ec6e6be0 at term 10
2016-06-12 02:30:45.040727 E | etcdhttp: got unexpected response error (etcdserver: request timed out) [merged 1 repeated lines in 1.74s]
2016-06-12 02:30:45.823899 I | raft: c23abd567f2f8c7e is starting a new election at term 10
2016-06-12 02:30:45.823959 I | raft: c23abd567f2f8c7e became candidate at term 11
2016-06-12 02:30:45.823977 I | raft: c23abd567f2f8c7e received vote from c23abd567f2f8c7e at term 11
2016-06-12 02:30:45.823996 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 950c7a0679d18d1 at term 11
2016-06-12 02:30:45.824014 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 2cf696424304f954 at term 11
2016-06-12 02:30:45.824033 I | raft: c23abd567f2f8c7e [logterm: 2, index: 6956] sent vote request to 97757695ec6e6be0 at term 11
^C2016-06-12 02:30:46.173021 N | osutil: received interrupt signal, shutting down...

Any suggestions where to start? Params match the cloud-config file...

Thanks.

@0x4139 I am closing this out. We need more logging, reproduce steps to help you more. Feel free to reopen if this is still an issue for you and you have more information. Thanks!


Feature/nextflow rna partial fix

Previously an error with an invalid config file with nf, it would kill refresh. Now it continues, but because we can't parse the config, we cannot parse metadata, make a tool page, or make a DAG.

Coverage decreased (-1.1%) to 68.39% when pulling c4f0145dc8ec03251165659ab7d0bc0e2ea4ad3f on feature/nextflow_rna_partial_fix into 5ad004f196a35c03b7e24496ab3534da31de48e5 on develop.


No such file or directory @ rb_sysopen on Xcode 8.3 - Gym

New Issue Checklist

[X] Updated fastlane to the latest version
[X] I have read the Contribution Guidelines

Issue Description
gym started to fail after Xcode (Version 8.3 (8E162)) installation
+------------------+----------+
|        Lane Context         |
+------------------+----------+
| DEFAULT_PLATFORM | ios      |
| PLATFORM_NAME    | ios      |
| LANE_NAME        | ios beta |
+------------------+----------+
[20:43:42]: No such file or directory @ rb_sysopen - /Applications/Xcode.app/Contents/Developer//Platforms/iPhoneOS.platform/Developer/usr/bin/PackageApplication

+------+------------------+-------------+
|           fastlane summary            |
+------+------------------+-------------+
| Step | Action           | Time (in s) |
+------+------------------+-------------+
| 1    | update_fastlane  | 7           |
| 2    | default_platform | 0           |
| 💥   | gym              | 48          |
+------+------------------+-------------+

[20:43:42]: fastlane finished with errors

Looking for related GitHub issues on fastlane/fastlane...

Found no similar issues. To create a new issue, please visit:
https://github.com/fastlane/fastlane/issues/new
Run `fastlane env` to append the fastlane environment to your issue
/Users/bunn/.fastlane/bin/bundle/lib/ruby/2.2.0/digest.rb:49:in `initialize': [!] No such file or directory @ rb_sysopen - /Applications/Xcode.app/Contents/Developer//Platforms/iPhoneOS.platform/Developer/usr/bin/PackageApplication (Errno::ENOENT)
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/2.2.0/digest.rb:49:in `open'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/2.2.0/digest.rb:49:in `file'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/2.2.0/digest.rb:34:in `file'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/gym/lib/gym/xcodebuild_fixes/package_application_fix.rb:23:in `block in patch_package_application'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/2.2.0/tmpdir.rb:88:in `mktmpdir'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/gym/lib/gym/xcodebuild_fixes/package_application_fix.rb:13:in `patch_package_application'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/gym/lib/gym/generators/package_command_generator_legacy.rb:16:in `generate'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/gym/lib/gym/generators/package_command_generator.rb:16:in `generate'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/gym/lib/gym/runner.rb:121:in `package_app'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/gym/lib/gym/runner.rb:19:in `run'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/gym/lib/gym/manager.rb:14:in `work'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/fastlane/lib/fastlane/actions/gym.rb:21:in `run'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/fastlane/lib/fastlane/runner.rb:252:in `block (2 levels) in execute_action'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/fastlane/lib/fastlane/actions/actions_helper.rb:50:in `execute_action'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/fastlane/lib/fastlane/runner.rb:230:in `block in execute_action'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/fastlane/lib/fastlane/runner.rb:226:in `chdir'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/fastlane/lib/fastlane/runner.rb:226:in `execute_action'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/fastlane/lib/fastlane/runner.rb:148:in `trigger_action_by_name'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/fastlane/lib/fastlane/fast_file.rb:146:in `method_missing'
	from Fastfile:16:in `block (2 levels) in parsing_binding'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/fastlane/lib/fastlane/lane.rb:33:in `call'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/fastlane/lib/fastlane/lane.rb:33:in `call'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/fastlane/lib/fastlane/runner.rb:49:in `block in execute'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/fastlane/lib/fastlane/runner.rb:45:in `chdir'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/fastlane/lib/fastlane/runner.rb:45:in `execute'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/fastlane/lib/fastlane/lane_manager.rb:52:in `cruise_lane'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/fastlane/lib/fastlane/command_line_handler.rb:30:in `handle'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/fastlane/lib/fastlane/commands_generator.rb:104:in `block (2 levels) in run'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/commander-fastlane-4.4.4/lib/commander/command.rb:178:in `call'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/commander-fastlane-4.4.4/lib/commander/command.rb:178:in `call'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/commander-fastlane-4.4.4/lib/commander/command.rb:153:in `run'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/commander-fastlane-4.4.4/lib/commander/runner.rb:476:in `run_active_command'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/fastlane_core/lib/fastlane_core/ui/fastlane_runner.rb:38:in `run!'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/commander-fastlane-4.4.4/lib/commander/delegates.rb:15:in `run!'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/fastlane/lib/fastlane/commands_generator.rb:303:in `run'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/fastlane/lib/fastlane/commands_generator.rb:42:in `start'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/fastlane/lib/fastlane/cli_tools_distributor.rb:65:in `take_off'
	from /Users/bunn/.fastlane/bin/bundle/lib/ruby/gems/2.2.0/gems/fastlane-2.23.0/bin/fastlane:20:in `<top (required)>'
	from /Users/bunn/.fastlane/bin/bundle/bin/fastlane:22:in `load'
	from /Users/bunn/.fastlane/bin/bundle/bin/fastlane:22:in `<main>'

[INSERT OUTPUT HERE]
Environment
| Key                         | Value                                       |
| --------------------------- | ------------------------------------------- |
| OS                          | 10.12.4                                     |
| Ruby                        | 2.2.4                                       |
| Bundler?                    | false                                       |
| Git                         | git version 2.11.0 (Apple Git-81)           |
| Installation Source         | ~/.fastlane/bin/bundle/bin/fastlane         |
| Host                        | Mac OS X 10.12.4 (16E195)                   |
| Ruby Lib Dir                | ~/.fastlane/bin/bundle/lib                  |
| OpenSSL Version             | OpenSSL 1.0.2g  1 Mar 2016                  |
| Is contained                | false                                       |
| Is homebrew                 | true                                        |
| Is installed via Fabric.app | false                                       |
| Xcode Path                  | /Applications/Xcode.app/Contents/Developer/ |
| Xcode Version               | 8.3                                         |

Hey @Bunn, thanks for submitting this issue. Do you have the legacy build API enabled? If so, can you try disabling it?

@KrauseFx That did the trick, thanks 👍
But it's interesting, I had the legacy build enabled for some reason, I can't remember but I think it was related to code signing. So I guess this is not an issue anymore.
Thanks again.

Yeah, it's unfortunate, but lots of people still use the legacy build API 😢

Hey @KrauseFx can you give a brief description of what did that parameter do? Because I ran into this same issue, and yes, removing it fixed it. But I had to had for some reason, right?
Thanks!!

In my case I'm not 100% sure but I think was something related to code signing an ad-hoc build.
I found more info here:
https://github.com/fastlane/fastlane/issues/6321 and here https://github.com/fastlane-old/gym/issues/104

Hi, @KrauseFx, I have the same error, and I disabled the legacy build API, and I have an other error.
Error Domain=IDECodesignResolverErrorDomain Code=5 "No matching provisioning profiles found" UserInfo={NSLocalizedRecoverySuggestion=No provisioning profiles with a valid signing identity (i.e. certificate and private key pair) matching the bundle identifier \U201xxxxxxx.xx.xx.x\U201d were found., NSLocalizedDescription=No matching provisioning profiles found, IDECodesignResolverError_ResolutionInputsKey=<IDECodesignResolutionInputs: 0x7fa774391160; portalTeamID='xxxx', usingTeamBasedSigning='NO', bundleIdentifier='xx.xx.xx.x.xt', targetName='(null)', provisioningProfilePurpose='1', requiresProvisioningProfile='YES', provisioningProfilePlatform='iOS', certificateKind='1.2.840.1136<IP_ADDRESS>.4', requiredEntitlements='{\n    "aps-environment" = development;\n    "beta-reports-active" = 1;\n    "keychain-access-groups" =     (\n        "9SP4AKXQTU.com.thanksys.fidcashx.user"\n    );\n}', requiredCodesignableDevices='(null)', requiredFeatures='(null)'>\n}";
};
and it works when I use Xcode 8.2, legacy API.

Hi @KrauseFx, Also getting errors, even after removing the legacy api
[18:57:04]: Exit status: 70
[18:57:04]: 📋  For a more detailed error log, check the full log at:
[18:57:04]: 📋  /Users/ronkliffer/Library/Logs/gym/DriverBox-DriverBox.log
+----------------------------+---------------------------------------------------------------------+
|                                           Lane Context                                           |
+----------------------------+---------------------------------------------------------------------+
| DEFAULT_PLATFORM           | ios                                                                 |
| PLATFORM_NAME              | ios                                                                 |
| LANE_NAME                  | ios release                                                         |
| SCAN_DERIVED_DATA_PATH     | /Users/ronkliffer/Library/Developer/Xcode/DerivedData/DriverBox-... |
| SCAN_GENERATED_PLIST_FILES | ["/Users/ronkliffer/Library/Developer/Xcode/DerivedData/DriverBo... |
| SCAN_GENERATED_PLIST_FILE  | /Users/ronkliffer/Library/Developer/Xcode/DerivedData/DriverBox-... |
| BUILD_NUMBER               | 20170329.185212                                                     |
| VERSION_NUMBER             | 8.9.10                                                              |
+----------------------------+---------------------------------------------------------------------+
[18:57:04]: Error packaging up the application
+------+---------------------------------------------+-------------+
|                         fastlane summary                         |
+------+---------------------------------------------+-------------+
| Step | Action                                      | Time (in s) |
+------+---------------------------------------------+-------------+
| 1    | default_platform                            | 0           |
| 2    | Switch to ios tests lane                    | 0           |
| 3    | scan                                        | 83          |
| 4    | xcov                                        | 8           |
| 5    | Switch to ios release_no_tests lane         | 0           |
| 6    | Switch to ios slack_notification_begin lane | 0           |
| 7    | slack                                       | 1           |
| 8    | increment_build_number                      | 1           |
| 9    | increment_version_number                    | 1           |
| 10   | Switch to ios build lane                    | 0           |
| 11   | get_version_number                          | 0           |
| 💥   | gym                                         | 288         |
+------+---------------------------------------------+-------------+
[18:57:04]: fastlane finished with errors

Same issue:
<details><summary>✅ fastlane environment ✅</summary>

### Stack

| Key                         | Value                                       |
| --------------------------- | ------------------------------------------- |
| OS                          | 10.12.4                                     |
| Ruby                        | 2.3.3                                       |
| Bundler?                    | false                                       |
| Git                         | git version 2.12.2                          |
| Installation Source         | ~/.rbenv/versions/2.3.3/bin/fastlane        |
| Host                        | Mac OS X 10.12.4 (16E195)                   |
| Ruby Lib Dir                | ~/.rbenv/versions/2.3.3/lib                 |
| OpenSSL Version             | OpenSSL 1.0.2j  26 Sep 2016                 |
| Is contained                | false                                       |
| Is homebrew                 | false                                       |
| Is installed via Fabric.app | false                                       |
| Xcode Path                  | /Applications/Xcode.app/Contents/Developer/ |
| Xcode Version               | 8.3                                         |

### System Locale

| Variable | Value       |   |
| -------- | ----------- | - |
| LANG     | en_GB.UTF-8 | ✅ |
| LC_ALL   |             |   |
| LANGUAGE |             |   |

### fastlane files:

<details><summary>`./fastlane/Fastfile`</summary>

```ruby
# Customise this file, documentation can be found here:
# https://github.com/fastlane/fastlane/tree/master/fastlane/docs
# All available actions: https://github.com/fastlane/fastlane/blob/master/fastlane/docs/Actions.md
# can also be listed using the `fastlane actions` command

# Change the syntax highlighting to Ruby
# All lines starting with a # are ignored when running `fastlane`

#select xcode version
#xcversion(version: "8.1") # Selects Xcode 8.1.0

xcode_select "/Applications/Xcode.app"

# If you want to automatically update fastlane if a new version is available:
#update_fastlane

# This is the minimum version number required.
# Update this, if you use features of a newer version
fastlane_version "1.111.0"

default_platform :ios

platform :ios do
  before_all do
    #cocoapods
  end

  desc "Runs all the tests"
  lane :test do
    scan
  end

  desc "Create a new Alpha Build"
  desc "This will also make sure the profile is up to date"
  lane :alpha do
    match(
      type: 'adhoc',
      team_id: CredentialsManager::AppfileConfig.try_fetch_value(:team_id),
      team_name: 'xxxx',
      app_identifier: CredentialsManager::AppfileConfig.try_fetch_value(:app_identifier),
      shallow_clone: false,
      force: true,
      force_for_new_devices: true
    )

    gym(
      scheme: 'xxxx',
      use_legacy_build_api:true,
      configuration: 'Alpha',
      output_directory: './build',
      output_name: 'xxxx-Alpha.ipa',
      include_symbols:true,
      include_bitcode:false,
      silent:true
    )

    # pilot
    hockey(
      api_token: 'xxxx',
      public_identifier: 'xxxx',
      ipa: 'build/xxxx-Alpha.ipa',
      notify: '1',
      status: '2',
      release_type: '2',
      teams: 'xxxx',
      tags: 'developer,tester',
      strategy: 'replace'
    )

    # sh "your_script.sh"
    # You can also use other beta testing services here (run `fastlane actions`)
  end

  desc "Create a new Beta Build"
  desc "This will also make sure the profile is up to date"
  lane :beta do
    match(
      type: 'adhoc',
      team_id: CredentialsManager::AppfileConfig.try_fetch_value(:team_id),
      team_name: 'xxxx',
      app_identifier: CredentialsManager::AppfileConfig.try_fetch_value(:app_identifier),
      shallow_clone: false,
      force: true,
      force_for_new_devices: true
    )

    gym(
      scheme: 'xxxx',
      use_legacy_build_api:true,
      configuration: 'Beta',
      output_directory: './build',
      output_name: 'xxxx-Beta.ipa',
      include_symbols:true,
      include_bitcode:false,
      silent:true
    )

    hockey(
      api_token: 'xxxx',
      public_identifier: 'xxxx',
      ipa: 'build/xxxx-Beta.ipa',
      notify: '1',
      status: '2',
      release_type: '0',
      teams: '37310',
      tags: 'developer,tester',
      strategy: 'replace'
    )

    # pilot
    # sh "your_script.sh"
    # You can also use other beta testing services here (run `fastlane actions`)

  end

  desc "Create a new Store Build"
  desc "This will also make sure the profile is up to date"
  lane :store do
    match(
      type: 'appstore',
      team_id: CredentialsManager::AppfileConfig.try_fetch_value(:team_id),
      team_name: 'xxxx',
      app_identifier: CredentialsManager::AppfileConfig.try_fetch_value(:app_identifier),
      shallow_clone: false,
      force: true,
      force_for_new_devices: true
    )

    # snapshot

    gym(
      scheme: 'xxxx',
      use_legacy_build_api:true,
      configuration: 'Store',
      output_directory: './build',
      output_name: 'xxxx-Store.ipa',
      include_symbols:false,
      include_bitcode:false,
      silent:true
    )

    deliver(
      skip_metadata: true,
      skip_screenshots: true,
      submit_for_review: false,
      force: true,
      username: 'xxxx'
    )
    # frameit

    # pilot
    # sh "your_script.sh"
    # You can also use other beta testing services here (run `fastlane actions`)

    hockey(
      api_token: 'xxxx',
      public_identifier:  'xxxx',
      ipa: 'build/xxxx-Store.ipa',
      notify: '1',
      status: '2',
      release_type: '1',
      teams: 'xxxx',
      tags: 'developer,tester',
      strategy: 'replace'
    )

  end

  # You can define as many lanes as you want

  after_all do |lane|
    #hockey

    puts "All done successfully..."

    # slack(
    #   message: "Successfully deployed new App Update."
    # )
  end

  error do |lane, exception|

    puts "Error: " + exception.message

    # slack(
    #   message: exception.message,
    #   success: false
    # )
  end
end

# More information about multiple platforms in fastlane: https://github.com/fastlane/fastlane/blob/master/fastlane/docs/Platforms.md
# All available actions: https://github.com/fastlane/fastlane/blob/master/fastlane/docs/Actions.md

# fastlane reports which actions are used
# No personal data is recorded. Learn more at https://github.com/fastlane/enhancer

`./fastlane/Appfile`
apple_id "xxxx" # Your Apple email address
team_id "V7J29L8Y68"  # Developer Portal Team ID

for_lane :test do
    app_identifier 'xxxx'
end

for_lane :alpha do
    app_identifier 'xxxx'
end

for_lane :beta do
    app_identifier 'xxxx'
end

for_lane :store do
    app_identifier 'xxxx'
end

# you can even provide different app identifiers, Apple IDs and team names per lane:
# More information: https://github.com/fastlane/fastlane/blob/master/fastlane/docs/Appfile.md

fastlane gems

Gem
Version
Update-Status

fastlane
2.23.0
✅ Up-To-Date

Loaded fastlane plugins:
No plugins Loaded
Loaded gems

Gem
Version

did_you_mean
1.0.0

slack-notifier
1.5.1

CFPropertyList
2.3.5

claide
1.0.1

colored
1.2

nanaimo
0.2.3

xcodeproj
1.4.2

rouge
1.11.1

xcpretty
0.2.4

terminal-notifier
1.7.1

terminal-table
1.4.5

public_suffix
2.0.5

addressable
2.5.0

multipart-post
2.0.0

word_wrap
1.0.0

tty-screen
0.5.0

babosa
1.0.2

highline
1.7.8

commander-fastlane
4.4.4

excon
0.55.0

faraday
0.11.0

http-cookie
1.0.3

faraday-cookie_jar
0.0.6

gh_inspector
1.0.3

uber
0.0.15

representable
2.3.0

retriable
2.1.0

hurley
0.2

jwt
1.5.6

memoist
0.15.0

multi_json
1.12.1

os
0.9.6

signet
0.7.3

googleauth
0.5.1

httpclient
2.8.3

google-api-client
0.9.26

mini_magick
4.5.1

multi_xml
0.6.0

security
0.1.3

xcpretty-travis-formatter
0.0.4

dotenv
2.2.0

faraday_middleware
<IP_ADDRESS>

json
1.8.3

activesupport
<IP_ADDRESS>

io-console
0.4.6

rubyzip
1.2.1

plist
3.2.0

unf_ext
<IP_ADDRESS>

unf
0.1.4

domain_name
0.5.20170223

fastimage
2.1.0

bundler
1.14.6

generated on: 2017-03-29

```

@WANGjieJacques I was seeing a similar error when building against an enterprise profile with Xcode 8.3 after removing the legacy build api setting. The solution was to add export_method: "enterprise" to the gym invocation.

@jgenden thanks, it works, for me, I add  export_method: "development"

Hello.
With use_legacy_build_api:true I get this error:
No such file or directory @ rb_sysopen - /Applications/Xcode.app/Contents/Developer//Platforms/iPhoneOS.platform/Developer/usr/bin/PackageApplication
Without use_legacy_build_api I get this:
"exportOptionsPlist error for key 'method': expected one of {}, but found ad-hoc" UserInfo={NSLocalizedDescription=exportOptionsPlist error for key 'method': expected one of {}, but found development}
Xcode 8.3
default config from https://fabric.io/features/distribution?utm_campaign=fastlane.tools
Have anyone fixed it?

I tryed to remove the legacy_mode thing, and add the export_method. But now i'm getting the error stated in:
https://github.com/fastlane/fastlane/issues/8704
Basically, on xcode 8.3, the PackageApplication which is inside xcode.app bundle is not present anymore.

I removed legacy mode but now I get "No matching provisioning profiles found". I think this was the reason I had to add legacy mode in the first place. It was working in Xcode 8.2.
I also tried with the setting posted by @WANGjieJacques export_method: "development" or export_method: "ad-hoc" but I still get the same error...

@i-schuetz maybe you can try other options, available options are app-store, ad-hoc, package, enterprise, development, developer-id.

@WANGjieJacques I need ad-hoc...

Hey everyone, sorry for jumping in so late, I had to do some investigations before replying 👋
Unfortunately the legacy build API was removed with Xcode 8.3. Please make sure to remove use_legacy_build_api from your ./fastlane/Fastfile and update the gym call to include the export method like this:
App Store Builds
gym(scheme: "MyScheme", export_method: "app-store")

Ad Hoc Builds
gym(scheme: "MyScheme", export_method: "ad-hoc")

Development Builds
gym(scheme: "MyScheme", export_method: "development")

In-House Enterprise Builds
gym(scheme: "MyScheme", export_method: "enterprise")

What to do if you run into code signing errors?
If you run into a code signing error, please check out our troubleshooting guide and common issue and how to solve them.
If that still doesn't help please submit a new issue instead of replying to this one. Make sure to share your fastlane env output when doing so, and mention that you followed all steps in the linked docs.
I hope that helps, please let me know if the documentation is unclear, or if there is anything else we can do better 🚀

So that others can find it also...
I had exactly the case as referred by @gleyba and in my case, the solution was that i had some subprojects included as dependencies which had on the building settings a wrong setting:
From:
Skip Install: No

Change to:
Skip Install: Yes

On all such subprojects.
You can test if your binary is fine by trying to "Archive" it in xcode.
If the binary ends up in the organizer inside the "Other items" and marked as a "Generic Xcode Archive", then there is something wrong with the libraries with that setting or there are public header-files being exposed (which can also trigger the same issue as the skip install setting).
If the binary ends up in the "iOS Apps" (no idea on the other kinds like macOS ones etc), then the binary should be fine and the new build process will not complain anymore of the "method" error.

@lferro9000 saved my struggle!! Thanks a lot :)
I was lost with the following issue,
binary ends up in the organizer inside the "Other items" and marked as a "Generic Xcode Archive"!
Sub project: Change to
Skip Install: Yes


lovis4u: command not found (conda installation issue)

Hello,
I installed lovis4u in a conda environment, with the command python3 -m pip install lovis4u as indicated. However, when I run the command lovis4u --linux then I have the error "command not found"
Thanks for your help
Corentin

Hi! Thank you for trying lovis4u and this report.
Yep, I tested on an empty conda env and got the same error.
Seems in general it can be fixed by the following way:
You first install (verify that installed) conda 'pip' library with : conda install pip and then install lovis4u directly with it wo specifying python version: pip install lovis4u which makes then "bin" file available within conda env. At least it worked in my case..
Let me know, please, if it doesn't solve your problem
As i replied in another issue comment, I have plans to add lovis4u to conda as well to avoid such problems in the next releases
Best,
Artyom

It's perfect, it works!
Thank you very much !
Best,
Corentin

I'm glad it helped, thank you too! :)
Best,
Artyom

@art-egorov Out of curiosity, is there a reason LoVis4u is not available as an actual Conda package? I was considering creating a recipe for it, but wanted to check with you first.

@apcamargo
Hi Antônio!
The only reason that I personally don't really conda envs :). It's still in the to do list to build a recipe and deploy there..
Seems there are some recipe building tools based on pypi packages for super fast deployment though. I'll have a look this week

No worries! I was just curious.
I was looking at the dependencies myself and noticed that there's a argparse dependency. Is there any reason for that, since it is included in the standard Python distribution?

Thank you for noticing! Nope, there is no good reason, just accidentally left there from old times. I'll publish a rare bug fixing update for features wo specified orientation today with this part removed.


add blogpost checker

Add blogpost-checker as merge gate.
See it in action: https://github.com/jo2/devblog/pull/1.
Once this is merged, we can add a branch protection on master which requires a successful run of pull-and-run-blogpost-checker-image.

Funktioniert! :) Bitte mergen.


Feature: loop through caveset

Press the left key to pass from the first selectable cave to the last selectable cave.
Press the right key to pass from the last selectable cave to the first selectable cave.

I do not fully understand your request. You mean selecting caves in the index by cursor keys?

When you load a caveset, you are proposed to play the first cave. If you want to play the 47th cave, you have to press Right 46 times, instead of to press Left 1 time and index on the last (selectable) cave, 47th, if the caveset has 47 caves, for instance.

Can not test at the moment but doesn’t FN or CTrL + left or right the trick?

Ctrl is used for Play, so Ctrl+Left or Ctrl+Right definitely don't work.
FN+Left, FN+Right don't work too.


Initial install gives 'WebpackError: ENOENT: no such file or directory, open '/apps/foundrs/public/.iconstats.json'

I've just added this to a project using the default config, but I initially get the following in logs:
success write out page data — 0.003 s
success write out redirect data — 0.001 s
success onPostBootstrap — 0.000 s

info bootstrap finished - 2.943 s

error There was an error compiling the html.js component for the development server.

See our docs page on debugging HTML builds for help https://goo.gl/yL9lND

  274 |  * Forces an update. This should only be invoked when it is known with
  275 |  * certainty that we are **not** in a DOM transaction.
> 276 |  *
      | ^
  277 |  * You may want to call this when you know that some deeper aspect of the
  278 |  * component's state has changed but `setState` was not called.
  279 |  *

  WebpackError: ENOENT: no such file or directory, open '/apps/foundrs/public/.iconstats.json'

@stefl So I installed the 2.1.1 version and the issue was resolved. I think the reason might have been that I had Gatsby 1.9.277 which was incompatible with the latest version of this plugin (which needs gatsby 2.0).

@jeriscc Thanks. Had a go, but the Gatsby 2 alpha blows up my SASS files.

Hello! I've just published an update to add a workaround for Gatsby v1, but it will still fail for the first build. See the mention added to the README: https://github.com/Creatiwity/gatsby-plugin-favicon#gatsby-v1-workaround
I'll close this issue for now, feel free to open another one if needed. :)

Change of plans. Badly, the last version of Gatsby doesn't let me export a v1 API in parallel with a v2 API. You will have to stick to the version 2.1.1 of this plugin with Gatsby v1. I added a mention about it in the README. Thanks again for the report.


Improve documentation for Quaternion.canonicalizedTransform

I'd like to make this more precise. If I wasn't concerned about jargon, I would say "the canonical representative of the equivalence class of quaternions representing the same 3d transformation."
Originally posted by @stephentyrone in https://github.com/apple/swift-numerics/pull/124

CC @markuswntr