@apply cannot be used with .mt-8 because .mt-8 either does not exist, or it's actual definition includes a pseudo-class like :hover, :active, etc.

Hi,
First of all - great work on the project.  Looking forward to using this, and replacing our old bootstrap based CSS.
I was trying to do the following:
h1, h2, h3, h4, h5, h6 {
    @apply .mt-8;
}

but when I compile (using Laravel Mix), I'm getting the following error:
`@apply` cannot be used with .mt-8 because .mt-8 either does not exist, or it's actual definition includes a pseudo-class like :hover, :active, etc.

How can I apply margins to my headings without having to add classes to all headings?
I agree that most items we can simply apply utility classes, but someting such as h tags would be something I would like to apply the classes to throughout the project :-)

Just wanted to add that I'm getting the same error, even when using the example from the docs:
.btn {
    @apply .font-bold .py-2 .px-4 .rounded;
}

results in the error
ERROR in ./resources/assets/tailwind/app.css
Module build failed: ModuleBuildError: Module build failed: Syntax Error

(2:5) `@apply` cannot be used with .font-bold because .font-bold either does not exist, or it's actual definition includes a pseudo-class like :hover, :active, etc.

  1 | .btn {
> 2 |     @apply .font-bold .py-2 .px-4 .rounded;
    |     ^
  3 | }

Can you share the whole file? It sounds like Tailwind’s utilities are not
being imported into your source file. Are you using a preprocessor like
Less or Sass or just plain CSS? If just plain CSS are you trying to use
imports but not using postcss-import?
On Sat, Nov 25, 2017 at 10:22 AM David Heremans<EMAIL_ADDRESS>wrote:

Just wanted to add that I'm getting the same error, even when using the
example from the docs:
.btn {
@apply .font-bold .py-2 .px-4 .rounded;
}
results in the error
ERROR in ./resources/assets/tailwind/app.css
Module build failed: ModuleBuildError: Module build failed: Syntax Error
(2:5) @apply cannot be used with .font-bold because .font-bold either does not exist, or it's actual definition includes a pseudo-class like :hover, :active, etc.
1 | .btn {

2 |     @apply .font-bold .py-2 .px-4 .rounded;
|     ^
3 | }

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/tailwindcss/discuss/issues/71#issuecomment-346946866,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEH3bGZ5pDyjl8VJH0_6BBQYPUVmhpRzks5s6DCigaJpZM4QqcfN
.

Awesome ok, so the problem is right now your compiled CSS is just using plain boring CSS imports which won't work with @apply; Tailwind needs to be able to find those classes in the same tree of styles.
So the solution for you is to add postcss-import to your build chain, which will take all of your @import statements and actually inline them.
Run this:
# Using npm
npm install postcss-import

# Or if using yarn
yarn add postcss-import

Then add postcss-import before Tailwind in your webpack.mix.js file:
  let mix = require('laravel-mix').mix;
  var tailwindcss = require('tailwindcss');
+ let atImport = require('postcss-import');

  mix.options({
          purifyCss: true, // Remove unused CSS selectors.
      })
      .js('resources/assets/js/app.js', 'public/build/js')
      .minify('public/build/js/app.js')
      .extract(['vue'])
      .postCss('resources/assets/tailwind/app.css', 'public/build/tailwind', [
+         atImport(),
          tailwindcss('./tailwind.js'),
      ])
      .minify('public/build/tailwind/app.css')
      .version([]);

Ok - you're brilliant.
I understand everything what you're saying.  The thing is I always wonder how one can know that this was the issue.
It did fix it, so I'm happy again, and will be able to continue migrating my layout to a new template - thanks so much for your assistance - MUCH appreciated!

Cool glad that helps!
Yeah you sort of need to have the right mental model of how @apply works to really understand why that would be the problem.
Basically, @apply can only apply classes from inside the same file, because it scans that file looking for a match.
By using postcss-import you are allowing yourself to keep things in separate files, but have everything get flattened into one file before Tailwind actually sees it, so by that time, @apply will work because all of your classes are now in the same file from Tailwind's perspective.
I'll see if I can think of a way to tweak the error messaging to help make that easier to understand, as well as update the documentation for that directive.

Maybe add some links to an article in your docs for certain errors.  VueJS does that too, and it does help you to keep your error concise, but at the same time direct your users to more detailed information :-)
That being said - thanks again for a great product!  So far, love it!

Closing to keep this great repo clean ;-)

you can keep this closed but for anyone else getting this error. my problem was that we added a tailwind.js file and instead of extending the default theme, we overwrote the existing one. which mean those classes would actually never get created.
the fix here is to extend the theme instead of overwriting it.

how this can be fixed in a regular, i.e react app?
I've tried:
//postcss.config.js
const tailwindcss = require('tailwindcss');
const atImport = require('postcss-import');

module.exports = {
 plugins: [
   atImport(),
   tailwindcss('./tailwind.js'),
   require('autoprefixer'),
 ],
};

But I'm still getting the related error:
CssSyntaxError: /Users/erik/Projects/webapp/src/styles/tailwind.css:78:3: `@apply` cannot be used with `.focus\:outline-none` because `.focus\:outline-none` either cannot be found, or its actual definition includes a pseudo-selector like :hover, :active, etc. If you're sure that `.focus\:outline-none` exists, make sure that any `@import` statements are being properly processed *before* Tailwind CSS sees your CSS, as `@apply` can only be used for classes in the same CSS tree.

Any help will be appreciated.

Hey! This is explained here in the docs:
https://tailwindcss.com/docs/functions-and-directives#apply

It's important to understand that @apply will not work for inlining pseudo-class or responsive variants of another utility. Instead, apply the plain version of that utility into the appropriate pseudo-selector or a new media query:

/* Won't work: */
.btn {
  @apply block bg-red-500;
  @apply hover:bg-blue-500;
  @apply md:inline-block;
}

/* Do this instead: */
.btn {
  @apply block bg-red-500;
}
.btn:hover {
  @apply bg-blue-500;
}
@screen md {
  .btn {
    @apply inline-block;
  }
}

Thank you @adamwathan


Feature request: Ability to set object visibility on headset and MRC seperately.

I'm trying to make mixed reality capture on Hololens 2 with MediaCapture API. What I want is to capture holograms on video but not show them on Hololens during capture.
Please add feature to set hologram hidden on headset, but record those hidden objects on mixed reality capture.

This is now available in v0.3.4, enjoy!


Simple OTA not working

I am trying to test simple OTA example using lightOnOff example using latest code from development branch.  Any help will be appreciated.
Environment
PlatformIO with following platformio.ini

[env:d1_mini]
platform = espressif8266
framework = arduino
board = d1_mini
upload_speed = 921600
lib_deps =
ArduinoJson @^5.0.0
AsyncMqttClient @^0.5.0
Bounce2 @^2.0.0

Steps taken

Download Latest lib
build lightOnOff
Upload (All good)
Add same bin to homie-ota
send firmware update from homie-ota (both binary and base64)
Update received, flashed, device reboots

Outcome:
device freezes/crashes and never comes online even after reset. Only way to bring device online is to manually re flash it.
Logs
Base64

🔌

Booting into normal mode 🔌
{} Stored configuration
• Hardware device ID: a020a601ac2
• Device ID: a020a601ac2
• Name: Pingy
• Wi-Fi:
◦ SSID: XXXXX
◦ Password not shown
• MQTT:
◦ Host: iot.eclipse.org
◦ Port: 1883
◦ Base topic: fic/devices/
◦ Auth? no

• OTA:
◦ Enabled? yes
↕ Attempting to connect to Wi-Fi...
✔ Wi-Fi connected, IP: <IP_ADDRESS>
Triggering WIFI_CONNECTED event...
↕ Attempting to connect to MQTT...
Sending initial information...
✔ MQTT ready
Triggering MQTT_CONNECTED event...
Calling setup function...
〽 Sending statistics...
• Wi-Fi signal quality: 84%
• Uptime: 1
✴ OTA available (checksum<PHONE_NUMBER>c65083b8dcf3bb4bd63bf2)
↕ OTA started
Triggering OTA_STARTED event...
Firmware is base64-encoded
Receiving OTA firmware (1052/6396)...
Receiving OTA firmware (2147/6396)...
Receiving OTA firmware (3242/6396)...
Receiving OTA firmware (4337/6396)...
Receiving OTA firmware (5432/6396)...
Receiving OTA firmware (6394/6394)...
✔ OTA succeeded
Triggering OTA_SUCCESSFUL event...
Device is idle
↻ Rebooting...
✖ MQTT disconnected
Triggering MQTT_DISCONNECTED event...
↕ Attempting to connect to MQTT...
✖ Wi-Fi disconnected
Triggering WIFI_DISCONNECTED event...
↕ Attempting to connect to Wi-Fi...
ets Jan  8 2013,rst cause:2, boot mode:(3,7)
load 0x4010f000, len 1384, room 16
tail 8
chksum 0x2d
csum 0x2d
v3ffe84a0
@cp:0
ld
r␘␂␀l��r�␀�#␂�n�␄��␌␘�␌␜��␜p�<���␀�8␂��ǒ��␌�8␌␘␌�nn�␂�;�nĒ��␌␛�␌b�#l␛$␛�␘��␀�<␒r�␛��;�␂p�n��܀␌␜���l␜����␌␜�␌b�␄n��n�$��␌␟�␌b�ľ~�n�␃����␀l␛��␒�#�n�␄�r␘␂␎␂nr���;␂��␌?�␌� ;␘␂␎r�ےn�␄��b�␘�␛Ğǌ�␜���␌�␃␂�;�␂��n�b
Binary

✴

OTA available (checksum<PHONE_NUMBER>c65083b8dcf3bb4bd63bf2)
↕ OTA started
Triggering OTA_STARTED event...
Firmware is binary
Receiving OTA firmware (1403/6394)...
Receiving OTA firmware (2863/6394)...
Receiving OTA firmware (4323/6394)...
Receiving OTA firmware (5783/6394)...
Receiving OTA firmware (6394/6394)...
✔ OTA succeeded
Triggering OTA_SUCCESSFUL event...
Device is idle
↻ Rebooting...
✖ MQTT disconnected
Triggering MQTT_DISCONNECTED event...
↕ Attempting to connect to MQTT...
✖ Wi-Fi disconnected
Triggering WIFI_DISCONNECTED event...
↕ Attempting to connect to Wi-Fi...
ets Jan  8 2013,rst cause:2, boot mode:(3,7)
load 0x4010f000, len 1384, room 16
tail 8
chksum 0x2d
csum 0x2d
v3ffe84a0
@cp:0
ld
r␘␂␀l��r�␀�#␂�n�␄��␌␘�␌␜��␜p�<���␀�8␂��ǒ��␌�8␌␘␌�nn�␂�;�nĒ��␌␛�␌b�#l␛$␛�␘��␀�<␒r�␛��;�␂p�n��܀␌␜���l␜����␌␜�␌b�␄n��n�$��␌␟�␌b�ľ~�n�␃����␀l␛��␒�#�n�␄�r␘␂␎␂nr���;␂��␌?�␌� {␘␂␎r�ےn�␄��b�␘�␛Ğǌ�␜���␌�␃␂�;�␂��n�b

Have you had any success with OTA on ESP8266? The "normal" way, using the esp8266/Arduino examples?

Very strange.
OTA isn't working here either. ANYMORE. It used to work flawlessly but since I've updated the library both @jpmens 's homie-ota nor @pezinek's script is working. Maybe it's related to #261.
I'm using a Wemos D1 mini. Firmware with latest commit, serial flash.
I'll try to flash while connected to serial later to get some debug info and confirmation that it's the same issue.  /ota/status gives a 500 internal error.

Strange for me Marvin was not able to reproduce it, I am able to reproduce it on multiple device.
Just some additional details I am using esp-wroom-02 board (which should not matter)
Also I have tried stand alone OTA over MQTT and it worked just fine. Device was able to take the update and boot up normally.
if there is more debugging I can turn on please let me know and I should be able to provide more info.

Base64 for Arduino ESP8266 has been improved in a January.
Can you try again with the latest version?
see https://github.com/esp8266/Arduino/pull/2883

@chaudhryfaisal have you tried with https://github.com/esp8266/Arduino/pull/2883 ?

Is this related to the b64 OTA issues?

I don't think so, but it does not harm to try. But I am afraid this is an esp8266/Arduino issue.

Your issue is that iot.eclipse.org truncates messages payload to ~6kB. A firmware is about 350kB.
I know there's probably something else that is wrong with the OTA, but closing for the other OTA issues.


First version of website

Issue to create first version of website.
I will base website on https://github.com/mmistakes/minimal-mistakes

Useful website: https://mmistakes.github.io/minimal-mistakes/docs/quick-start-guide/

Continues in #2


Card details not complete when confirming a setup intent, using PaymentSheet

Describe the bug
I'm using the PaymentSheet to collect card details.  We're using setup intents as we intend to process the payment later.  We are able to create the intent and collect the card details, but when I use confirmSetupIntent, I get the error from the title:
{
  "code": "Failed",
  "declineCode": null,
  "localizedMessage": "Card details not complete",
  "message": "Card details not complete",
  "stripeErrorCode": null,
  "type": null,
}

I'm getting confirmSetupIntent from the useStripe hook:
const { confirmSetupIntent } = useStripe();

Without confirming the setup intent, the card isn't saved and the payment won't be processed.
To Reproduce

Create setup intent.
Initialize payment sheet with setup intent.
Collect card details.
Confirm setup intent.

Expected behavior
The setup intent is confirmed without error.
Screenshots
n/a
Smartphone (please complete the following information):

Device: iPhoneX
OS: iOS 15.3.1
React Native 0.64.2
Stripe SDK 0.3.0

Additional context
I looked into the iOS/Android code and I think this isn't supported by the SDK.  The PaymentMethodFactory that is responsible for creating paymentMethodParams only seems to take a CardFormView or a CardFieldView.  It doesn't check for values in the PaymentSheet.
Is there another way to confirm the SetupIntent?  The PaymentSheet is important for our business needs.

confirmSetupIntent actually crashes Android, which could be a separate issue.

Device: Samsung Galaxy S10
OS: Android 11
RN 0.64.2
Stripe 0.3.0

Stack trace:
E/unknown:ReactNative: Exception in native call
    java.lang.RuntimeException: Could not invoke StripeSdk.confirmSetupIntent
        at com.facebook.react.bridge.JavaMethodWrapper.invoke(JavaMethodWrapper.java:383)
        at com.facebook.react.bridge.JavaModuleWrapper.invoke(JavaModuleWrapper.java:151)
        at com.facebook.react.bridge.queue.NativeRunnable.run(Native Method)
        at android.os.Handler.handleCallback(Handler.java:938)
        at android.os.Handler.dispatchMessage(Handler.java:99)
        at com.facebook.react.bridge.queue.MessageQueueThreadHandler.dispatchMessage(MessageQueueThreadHandler.java:27)
        at android.os.Looper.loop(Looper.java:246)
        at com.facebook.react.bridge.queue.MessageQueueThreadImpl$4.run(MessageQueueThreadImpl.java:226)
        at java.lang.Thread.run(Thread.java:923)
     Caused by: java.lang.reflect.InvocationTargetException
        at java.lang.reflect.Method.invoke(Native Method)
        at com.facebook.react.bridge.JavaMethodWrapper.invoke(JavaMethodWrapper.java:372)
        at com.facebook.react.bridge.JavaModuleWrapper.invoke(JavaModuleWrapper.java:151)
        at com.facebook.react.bridge.queue.NativeRunnable.run(Native Method)
        at android.os.Handler.handleCallback(Handler.java:938)
        at android.os.Handler.dispatchMessage(Handler.java:99)
        at com.facebook.react.bridge.queue.MessageQueueThreadHandler.dispatchMessage(MessageQueueThreadHandler.java:27)
        at android.os.Looper.loop(Looper.java:246)
        at com.facebook.react.bridge.queue.MessageQueueThreadImpl$4.run(MessageQueueThreadImpl.java:226)
        at java.lang.Thread.run(Thread.java:923)
     Caused by: java.lang.NoClassDefFoundError: Failed resolution of: Lcom/stripe/android/core/exception/InvalidRequestException;
        at com.reactnativestripesdk.ErrorsKt.createError(Errors.kt:75)
        at com.reactnativestripesdk.StripeSdkModule.confirmSetupIntent(StripeSdkModule.kt:602)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.facebook.react.bridge.JavaMethodWrapper.invoke(JavaMethodWrapper.java:372)
        at com.facebook.react.bridge.JavaModuleWrapper.invoke(JavaModuleWrapper.java:151)
        at com.facebook.react.bridge.queue.NativeRunnable.run(Native Method)
        at android.os.Handler.handleCallback(Handler.java:938)
        at android.os.Handler.dispatchMessage(Handler.java:99)
        at com.facebook.react.bridge.queue.MessageQueueThreadHandler.dispatchMessage(MessageQueueThreadHandler.java:27)
        at android.os.Looper.loop(Looper.java:246)
        at com.facebook.react.bridge.queue.MessageQueueThreadImpl$4.run(MessageQueueThreadImpl.java:226)
        at java.lang.Thread.run(Thread.java:923)
     Caused by: java.lang.ClassNotFoundException: Didn't find class "com.stripe.android.core.exception.InvalidRequestException" on path: DexPathList[[zip file "/data/app/~~RbEGsoKdRrtuNQmsF3SPjQ==/org.treecard.app.development-sLpmCI14MsLml3Frj6RR7Q==/base.apk"],nativeLibraryDirectories=[/data/app/~~RbEGsoKdRrtuNQmsF3SPjQ==/org.treecard.app.development-sLpmCI14MsLml3Frj6RR7Q==/lib/arm64, /data/app/~~RbEGsoKdRrtuNQmsF3SPjQ==/org.treecard.app.development-sLpmCI14MsLml3Frj6RR7Q==/base.apk!/lib/arm64-v8a, /system/lib64, /system/system_ext/lib64]]
        at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:207)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:379)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:312)
        at com.reactnativestripesdk.ErrorsKt.createError(Errors.kt:75)
        at com.reactnativestripesdk.StripeSdkModule.confirmSetupIntent(StripeSdkModule.kt:602)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.facebook.react.bridge.JavaMethodWrapper.invoke(JavaMethodWrapper.java:372)
        at com.facebook.react.bridge.JavaModuleWrapper.invoke(JavaModuleWrapper.java:151)
        at com.facebook.react.bridge.queue.NativeRunnable.run(Native Method)
        at android.os.Handler.handleCallback(Handler.java:938)
        at android.os.Handler.dispatchMessage(Handler.java:99)
        at com.facebook.react.bridge.queue.MessageQueueThreadHandler.dispatchMessage(MessageQueueThreadHandler.java:27)
        at android.os.Looper.loop(Looper.java:246)
        at com.facebook.react.bridge.queue.MessageQueueThreadImpl$4.run(MessageQueueThreadImpl.java:226)
        at java.lang.Thread.run(Thread.java:923)

Hello, we have a similar problem with confirmPayment.
We want allow users retry after fail 3DS. After store client_secret ant token we want to show up confirmPayment in a new screen.
With react-native-stripe 0.1.4 this is not possible because the confirmPayment method is attached to the cardField element. Until cardfiled is not complete confirmPayment throws this error, and we are sending a token...
¿Anyone know how to fix it?

Can anyone from Stripe comment on whether we can use Setup Intents with the PaymentSheet?  We would like to save cards for future transactions using the Payment Sheet, and are wondering if this is possible.

Hi! Sorry, lost track of this issue :(
You can indeed use setupintents with the paymentsheet. I think the problem here is your are attempting to use confirmSetupIntent, where you should be using confirmPaymentSheetPayment (or you can confirm outside of the client app, see the docs for more details on that and the rest of the flow)

Oh perfect, thanks @charliecruzan-stripe!  That makes sense, just didn't think of it.

No worries, im wondering if there's a way we could get rid of the need for that and instead just save whether or not you had a payment sheet flow open before calling confirmPayment or confirmSetupIntent, but would have to handle the off chance that someone does that flow but then confirms a separate payment/setup intent

Yeah, it would be more intuitive for them to work regardless of component, but I'm sure it's not easy on your end. It wasn't immediately obvious to me how wiring up confirmPayment and confirmSetupIntent to the payment sheet would work.

Gotchya, I have a todo item to update the doc strings for all the components/methods, so you should have to rely less on the actual integration docs and get a better idea of exactly what to do right in your editor. This will be at the top of that list!


WHOIS command: handle unknown nick

…Instead of crashing.
                        const userID = ircDetails[socket.discordid].members[whoisUser];
                                                                           ^
TypeError: Cannot read property '<NICK>' of undefined
    at /opt/discordIRCd/server.js:1392:76
    at Array.forEach (native)
    at Socket.<anonymous> (/opt/discordIRCd/server.js:1149:19)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at readableAddChunk (_stream_readable.js:176:18)
    at Socket.Readable.push (_stream_readable.js:134:10)
    at TCP.onread (net.js:547:20)

Dictionary key is unchecked at: https://github.com/creesch/discordIRCd/blob/04e14d3/server.js#L1392

This seems to happen when using an ircServer.username different from the Discord username.


Create uitest flavor

It is not clear why all of OdkDatabaseServiceImplTest.java needs to be in uiTests. There appears to be many tests that do not require files from app-designer. Those portion of tests should not be in the normal androidTest directory instead of UiTest

It turns out all of the OdkDatabaseServiceImplTest could be moved to regular androidTest. I've made the change


MODOAIPMH-11 OAI-PMH: Support for MARC21 XML

Implemented MarcJson to MarcXML mapper
Create MapperFactory
Add metadataPrefix enum
Add marc4j dependency

SonarQube analysis reported 1 issue

 1 info

Watch the comments in this conversation to review them.


Custom c definitions

I'm trying to provide a custom definition for SOKOL_LOG but the only simple way I could come up with was editing sokol_defines.h directly. That's not ideal since that file is overwritten with every update of the library.
What I currently do is edit sokol_defines.h like so:
// sokol_defines.h
#define SOKOL_ZIG_BINDINGS
#define SOKOL_NO_ENTRY
#define SOKOL_LOG(msg) sokol_log_callback(msg)
#if defined(_WIN32)
    #define SOKOL_WIN32_FORCE_MAIN
    // #define SOKOL_LOG(msg) OutputDebugStringA(msg)
#endif
// FIXME: macOS Zig HACK without this, some C stdlib headers throw errors
#if defined(__APPLE__)
#include <TargetConditionals.h>
#endif

And then in my zig code that statically links sokol, I do
export fn sokol_log_callback(msg: [*c]const u8) void {
    std.log.scoped(.sokol).err("{s}", .{msg});
}

It would be nice if there was a way to customize sokol from build.zig. One way would be to provide an option for a path to your own sokol_defines.h and fall back to the default one if not specified.
// Sidenote: Maybe consider replacing some of these parameters with a struct, which could then also have default-initialized options like the backend.
pub fn buildSokol(b: *Builder, [...], comptime defines_file: ?[]const u8) *LibExeObjStep {
    // ...
    lib.addCSourceFile(sokol_path ++ csrc, &[_][]const u8{"-DIMPL", backend_option, "-DSOKOL_DEFINES=" ++ (defines_file orelse (sokol_path ++ "sokol_defines.h"))});
    // ...
}

And then in all the sokol_*.c files, use it like this:
// sokol_gfx.c
#if defined(IMPL)
#define SOKOL_GFX_IMPL
#endif
#include SOKOL_DEFINES
#include "sokol_gfx.h"

But, to be honest, I feel like you would want to provide this customizability per sokol library. So maybe let the user provide their own versions of sokol_*.c? Or even just a single custom sokol.c that would include all sokol headers as needed.
Another way I thought about was "fixing" up the LibExeObjStep instance produced by buildSokol, iterating over all link_objects, pull out the c files, and replace them with my own. But that felt pretty hacky so I didn't even try this route.
Thoughts?

I think the best solution would be to provide a runtime callback in the C APIs, same as I did somewhat recently with the allocator functions, which were previously also preprocessor defines, but now work like this:
https://github.com/floooh/sokol/blob/01eeb35658fee9b6fe2e31145925fbd5e22faf47/sokol_gfx.h#L663-L690

I agree, moving stuff out of the build or preprocessor into the API itself sounds like the ideal solution. It looks like this doesn't only affect logging, though. Skimming through sokol_gfx.h there's SOKOL_ASSERT, for example, which would have the same problem. But it sounds doable without changing too much around.
On the other hand, there's also the option to define SOKOL_DEBUG to force-enable debug stuff in release mode. This one sounds very useful but looks like it has to be specified at build-time. And there's probably more like that (SOKOL_TRACE_HOOKS?). Any plans or thoughts for that?
By the way, thanks for the fast reply and your excellent work on these headers and language bindings. ❤

Ok, PR has been merged, closing this ticket :)


document not defined in web workers

When you try to use habitat in a web worker, it has issues because of document

Hopefully covered by the more general issue: #45


teamlogo

Posted a file with the logo. Let me know what you think.

au pretty. First thing I see: fire. And it reminds me of volume for some reason. Or copy number level. Same thing. This gets my seal of approval haha

Looks awesome! Thanks for doing that!!
Sent from my iPhone

On Mar 22, 2017, at 2:06 PM, Nick Giangreco<EMAIL_ADDRESS>wrote:
au pretty. First thing I see: fire. And it reminds me of volume for some reason. Or copy number level. Same thing. This gets my seal of approval haha
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub, or mute the thread.


menu hopping

One of my greatest irritations in latest firmwares is the removal of the "home" button from the reading screen.  Sometimes I have to hit return two or three times to get back to home.
Would it be possible to add items from the home screen menu (like home, my books, my articles, settings) so they could be added to the reading overflow menu?

I'll look into that. My only concern with doing this is how it will interact with Nickel's reading session management, so I'll need to check how that works and if MainWindowController handles everything as needed.

OK, this will be doable (EventEngine::endReadingSession is called by ReadingController::viewWillDisappear, which is an implementation of AbstractController::viewWillDisappear).

Ironically, you added everything except what I asked for -- a link to the home screen.
Not to be ungrateful, I think this change is actually wonderful, and although I'd still like to have HOME still, with all these other options, I'm not sure I'd ever use it!!   Thanks!

Oh, I guess I misinterpreted the question. I'll add a home option for the next version, although it'll be part of nickel_misc rather than nickel_open for technical reasons.


Expose response status and metadata

Currently for unary and request stream calls there is no easy way to get metadata and status if we want those
Server:
function sayHello(call, callback) {
  const md = new grpc.Metadata()
  md.set('foo', 'bar')
  call.sendMetadata(md)

  const smd = new grpc.Metadata()
  smd.set('biz', 'baz')
  callback(null, { message: 'Hello ' + call.request.name }, smd);
}

With the core client:
const call = client.sayHello({ name: user }, function (err, response) {
  console.log('response')
  console.log(response)
});

call.on('metadata', m => {
  console.log('metadata')
  console.dir(m, {depth: 4, colors: true})
})

call.on('status', m => {
  console.log('status')
  console.dir(m, {depth: 4, colors: true})
})

metadata
Metadata { _internal_repr: { foo: [ 'bar' ] } }
response
{ message: 'Hello world' }
status
{ code: 0,
  details: 'OK',
  metadata: Metadata { _internal_repr: { biz: [ 'baz' ] } } }

A proposal:
From an api perspective... we could keep the present api for easy of use, simplicity, and compatibility. In addition we could introduce a new "lower-level" Request and Response API, something similar to:
Fluent Request API:
const caller = require('grpc-caller')
const PROTO_PATH = path.resolve(__dirname, './protos/helloworld.proto')
const client = caller('<IP_ADDRESS>:50051', PROTO_PATH, 'Greeter')

const req = new client.Request('sayHello', { name: 'Bob'}) // the method we wish to invoke with the params
  .setOptions({}) // any normal "call" options
  .setMetadata({'request-id': '1234'}) // request call metadata
  .responseMetadata(true) // we care about getting the response metadata, if false we won't wait for and catch the 'metadata' event
  .responseStatus(true) // we care about getting the response status, if false we won't wait for and catch the 'status' event

const res = await req.exec() // promise based, optionally take callback for callback-based API

console.log(res.response)
// the response payload / "body"
// { message: 'Hello Bob!' }

console.log(res.metadata)
// the response "header" metadata converted into a map for us
// { foo: 'bar' }

console.log(res.status)
/*
the response status / trailer metadata:
{ code: 0,
  details: 'OK',
  metadata: { biz: 'baz' } }
*/

console.log(res.call) // just the call instance itself

We could wrap the above in an simpler functional API:
const res = await client.request('sayHello', { name: 'Bob'})
  .setOptions({})
  .setMetadata({'request-id': '1234'})
  .responseMetadata(true)
  .responseStatus(true)
  .exec()

or callback based:
client.request('sayHello', { name: 'Bob'})
  .setOptions({})
  .setMetadata({'request-id': '1234'})
  .responseMetadata(true)
  .responseStatus(true)
  .exec((err, res) => {
    console.log(res.response)
})

Underneath the present "simple" API would be implemented using the more verbose API.

Hey @bojand. First, thanks for all your gRPC work with Node. I appreciate it massively!
Second, how's progress coming along with this? I've seen you've got your req_res branch in progress and it's looking good so far!
I need access to the response metadata, hence why I'm here :)

Hello, Yea I think I got it mostly working but wasn't really happy with the resulting API or something for some reason and I think I kind of abandoned it... Don't remember now heh. I'll have to return to this soon and review it with a clearer mind now and see what's up.

@yoitsro I believe #11 adds the functionality. Please provide feedback on the new API. Thanks!

@bojand Thank you so much for this! I've not had chance to check it out yet, but will do in a few days time. And yeah, it makes sense having grpc required separately. It's similar to what I do with https://github.com/yoitsro/joigoose and mongoose :)

#11 is merged and published as 0.5.0.


Connection to local language model

I am looking to connect this to a smaller local model for offline use. Is this supported, or does it only work with cloud based language models?

Thanks, see also: https://github.com/hwchase17/langchain/issues/737#issuecomment-1421356996

Thanks - very helpful.


[HUDI-1493] Fixing args for schema compatability check

What is the purpose of the pull request
Fixing schema compatibility check. Reader and writer schema were wrongly sent. INT to LONG schema evolution for the same field fails if not for the patch.
Check https://github.com/apache/hudi/issues/2063 for issue raised by customer.
Reran the customer script and ensured the patch fixed the issue.
Brief change log

Fixed correct args for schema compatibility check.

Verify this pull request
This change added tests and can be verified as follows:

Added tests to TestTableSchemaEvolution to verify the change.

Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@n3nash @bvaradar : Can you confirm that this is a bug in our schema compatibility check. Also, please review when you can.

Codecov Report

Merging #2381 (cd4ab14) into master (286055c) will decrease coverage by 42.16%.
The diff coverage is n/a.

@@              Coverage Diff              @@
##             master    #2381       +/-   ##
=============================================
- Coverage     52.20%   10.04%   -42.17%
+ Complexity     2659       48     -2611
=============================================
  Files           335       52      -283
  Lines         14981     1852    -13129
  Branches       1505      223     -1282
=============================================
- Hits           7821      186     -7635
+ Misses         6535     1653     -4882
+ Partials        625       13      -612

Flag
Coverage Δ
Complexity Δ

hudicli
?
?

hudiclient
?
?

hudicommon
?
?

hudihadoopmr
?
?

huditimelineservice
?
?

hudiutilities
10.04% <ø> (-59.62%)
0.00 <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.


sync to flutter 3.0

to the new flutter, update the depencies

but I have problem update the sentry and http..
when update sentry , it will error during init


Going from 3.8.5 to 1.2.32 threw an error and killed NPM env var

Had to switch back to an older version. We have a repo at work that needs Node 0.10.12, so I figured I'd just set NPM to match the version that was released at the same time.
I got the following error.
C:\>npm-windows-upgrade --version:1.2.32
npm-windows-upgrade 0.3.2
? This tool will upgrade npm. Do you want to continue? Yes
Upgrading.. /Error:
C:\Users\jwilcurt\AppData\Roaming\npm\node_modules\npm\lib\npm.js:18
  var fs = gfs.gracefulify(require('fs'))
               ^
TypeError: Object #<Object> has no method 'gracefulify'
    at C:\Users\jwilcurt\AppData\Roaming\npm\node_modules\npm\lib\npm.js:18:16
    at Object.<anonymous> (C:\Users\jwilcurt\AppData\Roaming\npm\node_modules\np
m\lib\npm.js:394:3)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at C:\Users\jwilcurt\AppData\Roaming\npm\node_modules\npm\bin\npm-cli.js:26:
13
    at Object.<anonymous> (C:\Users\jwilcurt\AppData\Roaming\npm\node_modules\np
m\bin\npm-cli.js:76:3)

Upgrading.. -

Could not determine npm version.
You wanted to install npm 1.2.32, but the installed version isnull.
Please consider reporting your trouble to http://aka.ms/npm-issues.
Here is the output from the upgrader script:
[ 'All done!\r\n' ] [ '\r\nC:\\Users\\jwilcurt\\AppData\\Roaming\\npm\\node_modu
les\\npm\\lib\\npm.js:18\r\n  var fs = gfs.gracefulify(require(\'fs\'))\r\n
          ^\r\nTypeError: Object #<Object> has no method \'gracefulify\'\r\n
at C:\\Users\\jwilcurt\\AppData\\Roaming\\npm\\node_modules\\npm\\lib\\npm.js:18
:16\r\n    at Object.<anonymous> (C:\\Users\\jwilcurt\\AppData\\Roaming\\npm\\no
de_modules\\npm\\lib\\npm.js:394:3)\r\n    at Module._compile (module.js:456:26)
\r\n    at Object.Module._extensions..js (module.js:474:10)\r\n    at Module.loa
d (module.js:356:32)\r\n    at Function.Module._load (module.js:312:12)\r\n    a
t Module.require (module.js:364:17)\r\n    at require (module.js:380:17)\r\n
at C:\\Users\\jwilcurt\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.j
s:26:13\r\n    at Object.<anonymous> (C:\\Users\\jwilcurt\\AppData\\Roaming\\npm
\\node_modules\\npm\\bin\\npm-cli.js:76:3)\r\n' ]

Now NPM won't run properly:
C:\>npm --version
C:\Users\jwilcurt\AppData\Roaming\npm\node_modules\npm\lib\npm.js:18
  var fs = gfs.gracefulify(require('fs'))
               ^
TypeError: Object #<Object> has no method 'gracefulify'
    at C:\Users\jwilcurt\AppData\Roaming\npm\node_modules\npm\lib\npm.js:18:16
    at Object.<anonymous> (C:\Users\jwilcurt\AppData\Roaming\npm\node_modules\np
m\lib\npm.js:394:3)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at C:\Users\jwilcurt\AppData\Roaming\npm\node_modules\npm\bin\npm-cli.js:26:
13
    at Object.<anonymous> (C:\Users\jwilcurt\AppData\Roaming\npm\node_modules\np
m\bin\npm-cli.js:76:3)
C:\>

Also won't let me go back
C:\>npm-windows-upgrade
npm-windows-upgrade 0.3.2
? This tool will upgrade npm. Do you want to continue? Yes
We could not show latest available versions. Try running this script again with
the version you want to install (npm-windows-upgrade --version:3.0.0

C:\>npm-windows-upgrade --version 3.0.0
npm-windows-upgrade 0.3.2
? This tool will upgrade npm. Do you want to continue? Yes
We could not show latest available versions. Try running this script again with
the version you want to install (npm-windows-upgrade --version:3.0.0

To fix this, I would uninstall nodejs, and reinstall the version of node you want.
I would follow my nodejs clean uninstall instructions, found here https://github.com/felixrieseberg/npm-windows-upgrade#fix-an-attempted-upgrade
I don't think NPM 3.x is compatible with < 4.x of node.
You are using an old version of this package, once you reinstalled node, npm i -g npm-windows-upgrade
then update npm.

Yeah, @robertbaker is totally right here - npm 1 in particular is shockingly old and you'll very likely not have a good time. What seemed to have happened here is that you ended up with a Node/npm version mismatch, but there's sadly little we can do against that (given that we need npm to upgrade npm itself - once it's broken, we're lost...)


Improve root volume setup and add some EBS volume support

There are probably a few things that need to be done here which need to be fleshed out, including:

[ ] Tuning the size or type of the root volume (magnetic vs. SSD; ext4 setup; etc.)
[ ] Configuring additional EBS volumes (persistent storage)
[ ] Configuring additional instance store volumes (ephemeral storage); good as scratch space
[ ] Picking good defaults for this stuff
[ ] Deciding on how much of it should be configurable

I have been needing a larger root drive on my instances (10GB is not nearly enough!!), and have added the capability to choose the EBS size and type for the root drive.  As a nice fringe benefit, EBS volumes backed by SSD drives (type "gp2") are much more performant than the magnetic drive-backed EBS volumes.  With an SSD root drive, instances boot and install software much quicker.  These SSD-backed EBS volumes only cost twice as much as magnetic-backed EBS, which is well worth the cost in my opinion.  I think a 20 GB gp2 drive would be a sensible default for the root partition.
Here is the code snippet that creates a root drive (of a certain size and type).  On my Amazon Linux AMI, the root drive was automatically resized to take advantage of all of the available space allotted to it on the underlying block store.  I suspect this happens on CentOS AMIs as well.
           # Configure instance root drive. Note that if you choose a size
           # larger than 10GB, you may need to resize the filesystem once your
           # instance is running, using resize2fs, although some AMIs do this automatically.
           from boto.ec2.blockdevicemapping import BlockDeviceMapping, BlockDeviceType, EBSBlockDeviceType
           block_map = BlockDeviceMapping()
           device = EBSBlockDeviceType()
           device.size = int(ebs_root_vol_size)   # size in GB
           device.volume_type = ebs_root_vol_type # [ standard | gp2 | io1 ]
           device.delete_on_termination = True
           block_map["/dev/xvda"] = device

           reservation = connection.run_instances(
               image_id=ami,
               min_count=(num_slaves + 1),
               max_count=(num_slaves + 1),
               key_name=key_name,
               instance_type=instance_type,
               placement=availability_zone,
               security_group_ids=[sg.id for sg in security_groups],
               subnet_id=subnet_id,
               placement_group=placement_group,
               tenancy=tenancy,
               block_device_map=block_map,
               ebs_optimized=ebs_optimized,
               instance_initiated_shutdown_behavior=instance_initiated_shutdown_behavior)

I'm running up a slightly-different version of this problem -- I really want spark to be able to use the ephemeral storage for hdfs, but it seems for r3.8xlarge instances you have to manually specify the ephemeral block device mapping on start-up. Akin to the ec2-ami-list.yaml, could we add in information about the default ephemeral stores available for each instance type, and attach them through the block device mapper?
It seems like everything except for ebs root and ephemeral storage can be configured once the instances are up.

Following up on my previous comment, this appears to be a known problem. From spark_ec2.py,
    # AWS ignores the AMI-specified block device mapping for M3 (see SPARK-3342).
    if opts.instance_type.startswith('m3.'):
        for i in range(get_num_disks(opts.instance_type)):
            dev = BlockDeviceType()
            dev.ephemeral_name = 'ephemeral%d' % i
            # The first ephemeral drive is /dev/sdb.
            name = '/dev/sd' + string.letters[i + 1]
            block_map[name] = dev

SPARK-3342

Hey @ericmjonas, you're talking about adding additional instance store volumes and not changing the root volume, right?
Just for the record, to switch between instance store and EBS for the root volume, you need to switch AMIs. Additionally, Amazon recommends EBS root volumes because they launch faster.
If we're talking about adding additional, instance store-backed volumes, that seems reasonable to me. Could you elaborate on why you prefer them over having the option to add additional EBS-backed volumes?
I know spark-ec2 launches clusters with an ephemeral-hdfs and persistent-hdfs. Is this related to that idea? An area that Spark can use for SPARK_LOCAL_DIRS, large shuffles and the like?

@nchammas I had been talking about using the physical on-the-instance "ephemeral storage" -- it turns out the M3 type we use most often doesn't by-default make these available to the host OS. Many (most?) of the instance types provide this local storage and it works really well for ephemeral-hdfs and other temporary storage, and (unlike the EBS volumes) doesn't require us to hit the network.
This seems like a bug on AWS's side but they ... at least document it? ""For M3 instances, you must specify instance store volumes in the block device mapping for the instance. When you launch an M3 instance, we ignore any instance store volumes specified in the block device mapping for the AMI.""

Yup, instance store is AWS lingo for their ephemeral storage.
OK, so we do want this for use as scratch space. I'll update the issue description accordingly.

@nchammas I've been thinking about trying to make a bit of progress on this front, and wanted feedback before I really dive in. The official behavior here is complex, esp w.r.t supporting arbitrary AMIs:

it seems like any AMI can specify a default block device mapping. We can override this when we launch an instance, but for some AMIs this might break things.
Many AWS-optimized amis, such as the official AMI from amazon, don't have any block device mapping besides the root volume. So If you want to be able to touch the ephemeral disks, you need to manually map them at start.
This requires that you understand which of ephemeral0-ephemeral20 is present at lauch for a particular instance type. Thus we'll need to keep some additional per-instance-type metadata around.
I haven't confirmed but @shivaram in the amplab told me "SSDs don’t get formatted but the hard disks get formatted automatically". So that's a fun additional wrinkle.

One possible solution is:

query the AMI to see what its block device situation is
for the ephemeral drives that it's not using, mount them at some canonical points, format them, and use them for ephemeral hdfs
Make it so this can be customized if people really want to.

query the AMI to see what its block device situation is

Query the AMI, or did you mean query the instance type?
Anyway, I think there's a lot to think about here and it's easy to get sucked in to all details. I'm worried about adding too much complexity without getting enough return in the long run.
Let me ask this, for starters: As a user, what's the absolute minimum level of disk setup that would make Flintrock useful to you? What kind of setup are you really going to use, 90% of the time? I think that would be a good starting point for a discussion.
spark-ec2 does maintain a mapping of local volumes available by instance type, copied from Amazon's docs. I wonder if there's an elegant way around having to do that ourselves.

Sorry, I meant query the ami:
import boto.ec2

conn = boto.ec2.connect_to_region('us-west-2')
#test_ami ='ami-9ff7e8af'
test_ami = 'ami-bb46558b'

def get_ami_ephemeral_mounts(conn, ami):
    img = conn.get_image(test_ami)

    mp = {}
    for mount_point, device in img.block_device_mapping.items():
        mp[device.ephemeral_name] = mount_point
    return mp

print( get_ami_ephemeral_mounts(conn, test_ami))

For the case with ami-9ff7e8af
(py3k)dhcp-47-226:keystone jonas$ python testami.py
{None: '/dev/xvda'}

and with ami-bb46558b
(py3k)dhcp-47-226:keystone jonas$ python testami.py
{'ephemeral14': '/dev/sdp', 'ephemeral3': '/dev/sde', 'ephemeral5': '/dev/sdg', 'ephemeral4': '/dev/sdf', 'ephemeral15': '/dev/sdq', 'ephemeral10': '/dev/sdl', 'ephemeral1': '/dev/sdc', 'ephemeral13': '/dev/sdo', 'ephemeral8': '/dev/sdj', 'ephemeral12': '/dev/sdn', None: '/dev/sda1', 'ephemeral2': '/dev/sdd', 'ephemeral11': '/dev/sdm', 'ephemeral0': '/dev/sdb', 'ephemeral9': '/dev/sdk', 'ephemeral7': '/dev/sdi', 'ephemeral6': '/dev/sdh'}

I think "what's the absolute minimum level of disk setup that would make Flintrock useful to you? What kind of setup are you really going to use, 90% of the time?" is 100% the right attitude! The minimum I need is for flintrock to put the instance storage in the block-device-mapping, otherwise I can't do anything with the ephemeral storage after the instance is up.
Attaching additional EBS block devices, etc. are all things I can do after. I could even install and deploy HDFS later, although I think that also comes in the "what I'm going to use 90% of the time" column.

The minimum I need is for flintrock to put the instance storage in the block-device-mapping, otherwise I can't do anything with the ephemeral storage after the instance is up.

So to lay it out again like you did earlier, the minimum we want to start with is:

Query the AMI for mount points.
If any are not assigned names, that means they need to be manually formatted on launch. Do that.

Did I get that right?

Attaching additional EBS block devices, etc. are all things I can do after. I could even install and deploy HDFS later, although I think that also comes in the "what I'm going to use 90% of the time" column.

Yup, I favor leaving this stuff for later if possible. You might be able to do the HDFS setup for now using run-command, but if not we'll look at adding it in. Is Spark usable for your use cases with local storage but without HDFS?
By the way, about something you mentioned earlier:

This seems like a bug on AWS's side but they ... at least document it? ""For M3 instances, you must specify instance store volumes in the block device mapping for the instance. When you launch an M3 instance, we ignore any instance store volumes specified in the block device mapping for the AMI.""

I took a look at this, and it looks like Amazon updated their docs to remove special mention of M3 instances. So perhaps they fixed this irregularity.

To add another data point:
My "absolute minimum" list would be:
1. Making sure that ephemeral instance store disks are in the block device map, and making sure they are formatted and mounted when the instance boots up.  I use these frequently as scratch space.
2. Being able to (optionally) resize the root device to be bigger than the standard 10GB of most AMIs.
And although it is not strictly necessary, I think it would be very nice to be able to specify the storage type of the root device so that I could leverage faster SSD-backed root drives when desired, as these speed up the responsiveness of the cluster considerably.  This is basically a freebie that goes along with (2) above, since it is just a one-line tweak to the block device map.
If Flintrock does attempt to support 3rd party AMIs, then the issue of mapping AMI block device maps does become complicated, as they may differ from the instance block device map.  This is simplified quite a bit if Flintrock only supports a standard CentOS image, in which case the instance block mapping is the only one potentially in need of configuration.
Just out of curiosity, @ericmjonas , do you and your colleagues at the amplab rely heavily on your own customized AMIs?  Or do you get by ok with a "vanilla" AMI like the Amazon Linux that spark-ec2 uses?

Being able to (optionally) resize the root device to be bigger than the standard 10GB of most AMIs.

This is a good one. I would favor just picking a larger size (I believe you mentioned 20GB earlier) and hardcoding it in.

And although it is not strictly necessary, I think it would be very nice to be able to specify the storage type of the root device so that I could leverage faster SSD-backed root drives when desired, as these speed up the responsiveness of the cluster considerably. This is basically a freebie that goes along with (2) above, since it is just a one-line tweak to the block device map.

Same here. If SSDs are generally better, I would favor just baking that configuration in.

If Flintrock does attempt to support 3rd party AMIs, then the issue of mapping AMI block device maps does become complicated, as they may differ from the instance block device map. This is simplified quite a bit if Flintrock only supports a standard CentOS image, in which case the instance block mapping is the only one potentially in need of configuration.

I'd rather have Flintrock focus on 1 type of AMI, like the base CentOS AMI. If people want to try others, they're on their own. We should try not to make that intentionally difficult, but I wouldn't go out of my way to accommodate people using their own stuff either.
There's a small chance that, down the line, we can revive the image-build scripts and maintain a set of Flintrock AMIs. But that's way in the future.

@nchammas  you're to pick an AMI and keep it up to date / peg to a common one (like the AWS-supplied one) that would simplify a LOT of work. For some reason I thought that AMI-neutrality was a design goal.
@broxtronix As I understand it spark_ec2 points to a variety of lightly-customized but often-outdated AMIs (because who has time to build new ones and deploy to all availability regions?) This leads to things being out of date and a real problem when we're trying to use non-java software (I use spark-python with an anaconda-based python that often then calls out to custom C++ -- stale AMIs make my life hell)
I'd like to +1 @nchammas suggestion to just support SSDs with 20GB initial root partitions at the moment. Other than cost, are there any downsides?

I'm really excited by the flintrock model of "by default we use the latest of everything, although via config file you can peg to an older version if you need to"

For some reason I thought that AMI-neutrality was a design goal.

@ericmjonas Can you elaborate on what you mean?
For starters I'm testing Flintrock against the official, base CentOS AMI. To a lesser extent, I also test against the official, base Amazon Linux AMI, since CentOS and Amazon Linux are so similar.
I think it makes sense to do this because CentOS is available on other providers as well (e.g. GCE, which Flintrock may support in the future), and we don't run the burden of having to maintain our own AMIs like spark-ec2. Supporting other distributions like Ubuntu means we have to change our setup scripts accordingly and increases our testing surface area, which I'm against for now.
There is a related discussion in #11 you might be interested in.

As I understand it spark_ec2 points to a variety of lightly-customized but often-outdated AMIs (because who has time to build new ones and deploy to all availability regions?) This leads to things being out of date

Just a side-comment on this: I did a whole bunch of work to fix this for spark-ec2 in SPARK-3821 by automating the process of updating its AMIs. I've imported that work here under image-build.
It never made it into spark-ec2 because I couldn't get a committer to shepherd it through, but we may opt to revive it for use with Flintrock. I would rather avoid that, honestly, but it's an option.

I'd like to +1 @nchammas suggestion to just support SSDs with 20GB initial root partitions at the moment. Other than cost, are there any downsides?

Don't think so. And is the cost difference large enough to matter to users? Does it matter to either of y'all?

I'm really excited by the flintrock model of "by default we use the latest of everything, although via config file you can peg to an older version if you need to"

This stems from my general fear of being bogged down supporting older versions of stuff over the long term. Latest is what we support. You can reach into the past, but the further back you reach the more on your own you are. :)

@ericmjonas Can you elaborate on what you mean?

I guess I assumed that, because there wasn't an official flintrock AMI, the goal was to support all possible AMIs. Now that I know that's not the case, it makes life so much easier!

Don't think so. And is the cost difference large enough to matter to users? Does it matter to either of y'all?

Not to me or anyone at AMPLab, and we're at a pretty big scale (although we don't keep large clusters up for a long time -- so maybe for those users it might matter more. Not sure).

The cost difference is pretty minimal.  Magnetic storage is $0.05 / GB-month, and SSD storage is $0.10 / GB-month.  These costs are tiny in comparison to the cost of the instances themselves.  I don't think users of smaller clusters would even notice the difference too much.
Regarding the block device maps, it sounds based on this discussion like there will not be a need to support arbitrary AMI block device maps (thank goodness, as that sounds like a Sisyphean task!).
However, as a user I would still hope to be able to use any ephemeral instance store disks that are bundled with the instance.  My understanding is that these must be configured as part of the block store at instance launch time, so they cannot be attached by the user at a later point as part of their own custom configuration.  Since these instance store disks often form the ephemeral storage substrate for network filesystems like Hadoop, I suspect it'll be a commonly requested feature.  I use it in my clusters for both Hadoop and for temporary working directories for my pyspark processes, and would consider it to be one of the necessary features to really be able to use flintrock day-to-day.
Unfortunately it does not look to me like there is any way to query for a "default" block store that would include these local instance store disks... although perhaps one of you may find something in boto that I overlooked.  An automated way to set up the appropriate "default" block store for a given instance type would be ideal, but it sounds like people generally configure these manually.  I think a lookup table, as was done in spark-ec2, may be the only way to implement this feature, but I'm curious if either of you have another take on how this might be implemented in a way that minimizes maintenance down the road.

I guess I assumed that, because there wasn't an official flintrock AMI, the goal was to support all possible AMIs.

I'll probably add a note to the README explaining that we target and support CentOS, though users are free to try other distributions.

An automated way to set up the appropriate "default" block store for a given instance type would be ideal, but it sounds like people generally configure these manually. I think a lookup table, as was done in spark-ec2, may be the only way to implement this feature, but I'm curious if either of you have another take on how this might be implemented in a way that minimizes maintenance down the road.

I'm looking at how spark-ec2 uses this table and I'm a little confused. :confused:
What is the relationship between the instance type-specific information in this table and the AMI-linked mount point information Eric mentioned earlier?

What is the relationship between the instance type-specific information in this table and the AMI-linked mount point information Eric mentioned earlier?

I have been thinking of it as the following:

Each physical instance type has some number of ephemeral devices that physically sit in the box.
the block device mapping has the hypervisor "virtually connect" those devices to the guest OS / ami. If the hypervisor doesn't connect (attach) them, the AMI will never see them
an AMI can contain metadata telling the hypervisor to attach these devices. before the hypervisor launches an ami as a guest, it can attach things as the AMI requests. AMIs don't always have the metadata to make this request, because (say) a particular AMI wants to run on as many instance types as possible.
Somewhere the metadata "c3.8xlarge has two ephemeral SSD disks" has to be tracked so that a script (like spark EC2) knows which ephemeral devices to request the hypervisor attach.

My two cents:

While creating the AMI you can add as many mappings as you want. So while creating the Spark EC2 AMIs I think we just adding 4 mappings
From what I know, if the instance only has one disk and finds 4 mappings EC2 just satisfies the first few mappings.
So from the point of view of spark-ec2 the mappings already exist when the instance is launched and the things it does after that are:

Formatting the drives if they are SSDs
Creating HDFS data dirs, Spark local dirs and setting them in the config (This is what the mapping is actually used for)

From flintrock's perspective I guess you need to figure out if you assume the AMI has the right mappings or if you want to add it during launch

What is the relationship between the instance type-specific information in this table and the AMI-linked mount point information Eric mentioned earlier?

I am not certain, but I believe that the AMI block map includes any EBS volumes that happen to be attached to an instance when it is frozen and an AMI is created.  This can include mappings to local "ephemeral" instance store disks as well as other EBS-backed volumes that may have been mapped and attached to an instance.
I think we are aiming here to just support the ephemeral instance storage disks that are listed in this link.  These are actual disks that are attached to the host on which the instance is running, and are intended as local scratch space.
I really like the suggestion made by @shivaram, where we would simply create 4 mappings and leave it to Amazon to fulfill as many of those mappings as it can, subject to the number of disks available on an instance.  I tried that out this morning, using the following block of code:
            # Configure instance root drive. Note that if you choose a size
            # larger than 10GB, you may need to resize the filesystem once your
            # instance is running, using resize2fs, although some AMIs do this automatically.
            from boto.ec2.blockdevicemapping import BlockDeviceMapping, BlockDeviceType, EBSBlockDeviceType
            block_map = BlockDeviceMapping()
            device = EBSBlockDeviceType()
            device.size = int(ebs_root_vol_size)   # size in GB
            device.volume_type = ebs_root_vol_type # [ standard | gp2 | io1 ]
            device.delete_on_termination = True
            block_map["/dev/xvda"] = device

            # The first ephemeral drive is /dev/xvdb
            emphemeral_names = [ '/dev/xvdb', '/dev/xvdc', '/dev/xvdd', '/dev/xvde' ]
            for i in range(4):
                device = BlockDeviceType()
                device.ephemeral_name = 'ephemeral%d' % i
                block_map[ emphemeral_names[i] ] = device

This worked, inasmuch as it made two ephemeral instance store SSD disks available on my g2.8xlarge instance.  Oddly, one of these was formatted and mounted automatically, while the other was formatted but not mounted.  I'm using a version of Amazon Linux that supports GPUs, so maybe this is a bug in this particular AMI.  We should try it with the CentOS image that Flintrock is intending to standardize on to see how this works.
For future reference, the lsblk command is useful for debugging which block devices are available on your instance once it boots up.  Here is the output of lsblk on my instance after running the above code:
[ec2-user@ip-10-33-165-245 ~]$ lsblk
NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
xvda    202:0    0  100G  0 disk
└─xvda1 202:1    0  100G  0 part /
xvdb    202:16   0  130G  0 disk /media/ephemeral0
xvdc    202:32   0  130G  0 disk

You can see from this output that xvdb was formatted and mounted, and xvdc was not mounted.  (Turns out it was formatted... I was able to mount it with sudo mount /dev/xvdc /media/ephemeral1)
I believe that something very much like the code above will mostly address the needs being discussed here.  We may need to mount an additional stray disk or two manually once the cluster is up and running, but other than that this seems to work.

+1 @broxtronix 's solution -- it seems easy and a good intermediate point. My understanding is one of the major benefits of the amazon-linux amis is they handle a lot of the formatting for you, but I don't know if we should depend on that.

@broxtronix Are you ok with me starting to hack on this as a PR? I'm basically going to copy your approach but I wanted to make sure I wasn't stepping on any toes. I'm blocked on both this and HDFS deployment.

Of course, @ericmjonas !!   I will gladly follow along and help review the PR!   Have at it. :)

@broxtronix the Amazon centos-based AMIs resize the root partition automatically, but it appears the "Centos 7 with updates" AMIs do not. Do you know of an easy way to do this? resize2fs seems to need me to manually muck with the partition table first and i'm curious if there's an easier way. I've googled around to little avail, with a lot of advice from 2009 that seems less than applicable today.

Hmmm, it's too bad that the Centos 7 AMI doesn't doesn't grow the root partition automatically.  To do this manually, I think there are basically two steps involved:

Rewriting the partition table so that the partition uses all available space on the block device.
Resizing the filesystem to use all available space on the partition.

This post delves into more details about how this is done with the Centos AMI in particular.  They use the growroot.sh script to automatically resize partitions and resize the filesystem in one go.  They are pretty simple to use... they recommend trying:
yum update -y
yum install -y epel-release
yum install -y perl cloud-init dracut-modules-growroot
rpm -qa kernel | perl -pe 's/^kernel-//'  | xargs -I {} dracut -f /boot/initramfs-{}.img {}
<reboot instance>

I used this script that performs a very similar set of steps to resize the root partition on GCE instances.  I would try running these commands by hand first, and if it works we could figure out how to incorporate it into the launch process for new instances.
Let me know if this helps!

Is anyone actively working (or intending to work) on this issue? If not, I can take it on now that I finished work on adding HDFS support to Flintrock.

I started but then got totally slammed by a friday conference deadline :(
All I had really done is stolen Michael's code and put it in the right spot
-- if we're focusing on the Amazon AMI it's even easier because the root
partition resizing is handled automatically.
On Mon, Nov 9, 2015 at 5:51 PM, Nicholas Chammas<EMAIL_ADDRESS>wrote:

Is anyone actively working (or intending to work) on this issue? If not, I
can take it on now that I finished work on adding HDFS support
https://github.com/nchammas/flintrock/issues/41 to Flintrock.
—
Reply to this email directly or view it on GitHub
https://github.com/nchammas/flintrock/issues/28#issuecomment-155224223.

I am not actively working on this either, but would be glad to give advice & PR feedback if it is helpful!

@ericmjonas:

if we're focusing on the Amazon AMI it's even easier because the root
partition resizing is handled automatically.

Yeah, I was thinking about the notes earlier on CentOS not resizing volumes automatically. Part of the work we'll need to do here is provider-specific (e.g. creating and attaching EBS volumes), but part of it is tied to the distribution we use.
If we support GCE as an alternate provider down the line, will we end up having to implement this manual resizing stuff anyway? If so, it might be good to get this out of the way now.
For what it's worth, I'm open to revisiting what distribution to support (e.g. maybe Ubuntu can offer us the convenience of Amazon Linux without tying us to an EC2-specific approach), but I think a better time to do that would be when we are ready to add a new provider like GCE.
Anyway, I can work on something else in the meantime. Just let us know if you get too busy with other stuff and someone else can step in.
@broxtronix:

I am not actively working on this either, but would be glad to give advice & PR feedback if it is helpful!

Advice and PR feedback is absolutely helpful. Please feel free to chime in regardless of who is working on an issue. :+1:

Oh I'm sorry Nick, i was trying to suggest that I am not actively working
on it and haven't made any reasonable progress anyway, having merely copied
(and tested) the block of code that @broxtronix had added. Go ahead and
work on it! :)
On Mon, Nov 9, 2015 at 6:18 PM, Nicholas Chammas<EMAIL_ADDRESS>wrote:

@ericmjonas https://github.com/ericmjonas:
if we're focusing on the Amazon AMI it's even easier because the root
partition resizing is handled automatically.
Yeah, I was thinking about the notes earlier on CentOS not resizing
volumes automatically. Part of the work we'll need to do here is
provider-specific (e.g. creating and attaching EBS volumes), but part of it
is tied to the distribution we use.
If we support GCE as an alternate provider down the line, will we end up
having to implement this manual resizing stuff anyway? If so, it might be
good to get this out of the way now.
For what it's worth, I'm open to revisiting what distribution to support
https://github.com/nchammas/flintrock/issues/11#issuecomment-154886283
(e.g. maybe Ubuntu can offer us the convenience of Amazon Linux without
tying us to an EC2-specific approach), but I think a better time to do that
would be when we are ready to add a new provider like GCE.
Anyway, I can work on something else in the meantime. Just let us know if
you get too busy with other stuff and someone else can step in.
@broxtronix https://github.com/broxtronix:
I am not actively working on this either, but would be glad to give advice
& PR feedback if it is helpful!
Advice and PR feedback is absolutely helpful. Please feel free to chime in
regardless of who is working on an issue. [image: :+1:]
—
Reply to this email directly or view it on GitHub
https://github.com/nchammas/flintrock/issues/28#issuecomment-155230222.

Oh, word. OK, I'll go ahead and start on this then!
Like with adding HDFS support, I'll open a PR early on so y'all can provide feedback as I work on this feature.


policy: No-op Identity Allocator

Ref: https://github.com/cilium/cilium/issues/33360
Description
As part of the network policy modularization and decoupling of the network policy enforcement system from the endpoint, the no-op allocator aims to provide a possibility of disabling the identity allocation system when network policies are disabled.
Motivation
The scalability of network policy feature is limited when the scalability dimensions, # of nodes and pod churn rate, are stretched. Cilium has many networking features that scale better, or are unaffected by these scalability dimensions. Currently, disabling network policies to reach higher scale for other features is the only feasible solution.
Note: The plan is to improve scalability of network policies, so that it can reach higher scales that are required.
Follow-up
Subsequently, the plan is to agree on a configuration set that would determine if the identity allocation system should be disabled, and put the no-op allocator to use.
Proposed config:

EnablePolicy=never
DisableCiliumEndpointCRD=true
EnableK8sNetworkPolicy=false
EnableCiliumNetworkPolicy=false
EnableCiliumClusterwideNetworkPolicy=false

Signed-off-by: Dorde Lapcevic<EMAIL_ADDRESS>

/test

Clustermesh failures seems like bug introduced in this PR:
panic: runtime error: invalid memory address or nil pointer dereference
github.com/cilium/cilium/pkg/clustermesh.(*remoteCluster).Status(0xc00026a5a0)
    /go/src/github.com/cilium/cilium/pkg/clustermesh/remote_cluster.go:159 +0xd4

corresponds to:
status.NumIdentities = int64(rc.remoteIdentityCache.NumEntries())

Marcel Zięba Do you have any ideas why it would affect its behavior?

Yes, you can call method on nil pointer, but not on nil interface. On nil interface it causes panic.

/test

Hi @tklauser, could you please take a look for sig-policy?
cc @squeed

/test

/test

/test


StateError Bad state: No element

Describe the bug
We are seeing this from Sentry.io.
typewriter_value.dart in TypeWriterValue.text at line 87 within typewritertext
typewriter.dart in _X.build.. at line 229 within typewritertext
https://github.com/Nialixus/typewritertext/blob/2f081beace2772cf4627ec0e7739fe418d25bfef/lib/v3/typewriter_value.dart#L87
To Reproduce
Steps to reproduce the behavior:

Setup:

Flutter version: Flutter 3.24.2 (stable)
Dart version: Dart 3.5.2 (stable)
Operating System: iOS 17.5.1

Steps:

not able to reproduce; looking to log the String we are sending to TypeWriter

Context
While we cannot reproduce reliably, we are seeing it in production on occasion.  In inspecting the code, we note that it's possible that if one of the index added to indexes is empty then this would explain the error.
In particular, we note in [typewriter_value.dart around line 76](https://github.com/Nialixus/typewritertext/blob/2f081beace2772cf4627ec0e7739fe418d25bfef/lib/v3/typewriter_value.dart#L76) there is no check for item.runes.length.  If this is 0 then you will create an empty index.  And this will trip an error then at line 82
Current Code
    var current = 0;
    var indexes = <List<int>>[];
    for (String item in data) {
      var index = <int>[];
      for (int i = 0; i < item.runes.length; i++) { # this could be item.rune.length == 0
        index.add(current + i);
      }
      indexes.add(index); # if 0 then this an empty index
      current += item.runes.length;
    }
    return indexes;
  }

A proposed fix would just ensure the that item.runes.length is non-zero within the loop and skipping that item if so.
    var current = 0;
    var indexes = <List<int>>[];
    for (String item in data) {
      if (item.runes.length == 0){
          continue;  # defensively block in case runes.length is 0
      }
      var index = <int>[];
      for (int i = 0; i < item.runes.length; i++) { # this could be item.rune.length == 0
        index.add(current + i);
      }
      indexes.add(index); # if 0 then this an empty index
      current += item.runes.length;
    }
    return indexes;
  }

Hey man, thanks for the heads-up.
I’ve added a try-catch block to handle returning the text. Let me know if the issue persists.
TypewriterText v3.0.9

Thanks!  Will update and see if we see elimination of error.


Added TypeScript deceleration file for public facing API

Fixes #217

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@sayrer @fat Is this something you would consider adding to hogan.js or would it make more sense being added to DefinitelyTyped?

Fixed in https://github.com/DefinitelyTyped/DefinitelyTyped/pull/34469


Error messages in log

I'm getting lots of error messages in my log like this:

A post created activity record with id '9169bd78-8409-4363-bfb2-a88800e7d7a7' has a post id 'c3e082ba-cdf6-4633-95b2-a88800e7d765' that is not found in the post table.

Generated by:

MvcForum.Core.dll | ConvertToSpecificActivities | MvcForum.Core.Services.ActivityService

I get similar ones to do with topics, member joined, badges...  any idea what's causing them and what I can do to track down the issue please?

It just means a post has been deleted that is referenced in the activity table. I think I'll keep the log item, but delete the activity record to stop it showing.

Ah. That makes sense. Thanks for the update.  I was just getting worried about the log filling up rapidly.
Presumably it's the same for the topic, user & badge ones I'm getting then?

Yeah, anything like that, I should log then delete the activity record really.


sbt exclusion rules don't handle Scala version specific dependencies

steps
"net.liftweb" %% "lift-mapper" % "2.6-M4" % "compile" excludeAll(ExclusionRule("net.liftweb", "lift-json"))

problem
"net.liftweb" %% "lift-json" is actually not excluded (because you need cross versioned "lift-json_2.10").
expectation
ExclusionRule could smart up some how, or accept GroupArtifactID. The LHS of excludeAll is ModuleID so it should already know the crossVersion mode that its in.
original report
This came up as an issue recently, referenced in this Stack Overflow post from Channing Walton. Basically, the issue is that since exclusion rules are a pretty direct mapping (as I recall, anyway) onto Ivy, they don't include relevant information that sbt knows, such as the Scala version being referenced. So, while you can include a dependency that's Scala specific using the %% syntax, there's no similar mapping available for exclusions. I'm not sure what form a solution to this might take, but at least extending it to include something along the lines of the ModuleID support for %% might make sense, as a start.

+1, I'm also interested in this.

I did add project-level exclusion that uses ModuleID syntax in #1748

+1 for a version of exclude that takes into consideration the scala binary version.

+1 (+100) - really would be very useful indeed...

1

+1

This actually just bit me again. Might take a look in the near future at adding a fix for this, if time permits.

:+1:

+1

👍

+1

I'm using this workaround:
"com.holdenkarau" %% "spark-testing-base" % s"1.6.3_0.4.7" exclude("org.scalatest", s"scalatest_${scalaBinaryVersion.value}")

If anyone has better/cleaner ideas I'd love to hear them!

hmm.. any updates? was affected by this just today.. >2 years since ticket was opened..

Annoying me, too

+1

+1

Hello everyone and thank you for your patience!
The Scala Center has contributed a fix for this here: https://github.com/sbt/librarymanagement/issues/88. It will be merged in 1.0. That PR depends on a big pending PR reformatting all the sources, so don't get scared when you see the diff. The actual fix is this commit and these are the tests that prove the issue is fixed.

@jvican that's great news! Very much looking forward to seeing this commonly encountered, but tricky to recognize scenario getting resolved. Makes me all the more excited to see 1.0 come to life!

I am still having the issue related to the number of version in the exclusion
This is not excluding :
  val scalaKafkaClient = "net.cakesolutions" %% "scala-kafka-client-akka" % scalaKafkaClientVersion exclude ("com.typesafe.akka", "akka-actor")

But this works
  val scalaKafkaClient = "net.cakesolutions" %% "scala-kafka-client-akka" % scalaKafkaClientVersion exclude ("com.typesafe.akka", "akka-actor_12")

I thought from sbt 1.0 I would not need to explicitely set the scala version ?

@samudurand
val scalaKafkaClient = "net.cakesolutions" %% "scala-kafka-client-akka" % scalaKafkaClientVersion exclude ("com.typesafe.akka" %% "akka-actor")

@jvican looks good ! I will try that tonight

@jvican this does work but not with exclude , gotta use excludeAll()

Temp fix:
      libraryDependencies += "com.outworkers" %% "phantom-dsl" % Version.phantom excludeAll (
        ExclusionRule(organization = "org.typelevel"),
      ),
    ),

Does this really work with sbt 1.9.7? For me it seems not...
I have a dependency:
val atomRenderer = ("com.gu" %% "atom-renderer" % "1.2.0")

Now I want to exclude a dependency and it only works when appending _2.13 to it, everything else fails:
.exclude("com.typesafe.play", "twirl-api_2.13") // works

.exclude("com.typesafe.play", "twirl-api") // does NOT work

.excludeAll("com.typesafe.play" %% "twirl-api") // does NOT work

Same here, excludeAll("..." %% "...") does not work for me (sbt version 1.8.2)


Enable Artifact Streaming on mariner+fips

What type of PR is this?
/kind cleanup

What this PR does / why we need it:
Enable Artifact Streaming on Mariner + fips env
Which issue(s) this PR fixes:

Fixes #
Requirements:

[ ] uses conventional commit messages

[ ] includes documentation
[ ] adds unit tests
[ ] tested upgrade from previous version

Special notes for your reviewer:
Release note:
none

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 79.39%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
2292

Relevant Lines:
2887

💛  - Coveralls


Cyr rewrite

Codecov Report

Merging #111 (56eb0fc) into main (b7abc86) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #111   +/-   ##
=======================================
  Coverage   37.12%   37.12%
=======================================
  Files          12       12
  Lines         827      827
=======================================
  Hits          307      307
  Misses        475      475
  Partials       45       45

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b7abc86...56eb0fc. Read the comment docs.


🍱 adds emqx logo with new size of 512*512

Added the png icon with 512x of height

Please add an SVG (https://raw.githubusercontent.com/docker-library/docs/68aa4264fa058f323993fdaceacd63a8acbbeb48/emqx/logo.svg?sanitize=true) and make a new PNG from that.

@walkxcode: The SVG is already available in the main branch: https://github.com/walkxcode/dashboard-icons/blob/main/svg/emqx.svg

Ah! Okay 😅


(FACT-3452) Make Xen resolver more strict

Prior to this commit, Facter could misidentify non-Xen systems as being Xen-based.
In some circumstances, a non-Xen system will have /dev/xvd* files present as symlinks to other devices. This was observed on an Amazon Linux 2023 machine, which was running on a Nitro (KVM-based) hypervisor but symlinked /dev/xvda1 to an nvme device.
This commit updates the Xen resolver to check if /dev/xvda1 is a symlink to avoid misidentifying if a system is Xen-based.

On an Amazon Linux 2023 machine running on a t4g.medium instance on Nitro, this is the current behavior:
$ git describe
4.2.5-320-g421f9731c
$ bundle exec facter cloud

$ bundle exec facter virtual
xenu

Note that Facter doesn't resolve the cloud fact and misidentifies the machine as Xen.
With this PR:
$ git describe
4.2.5-375-ga11f4f8a4
$ bundle exec facter cloud
{
  provider => "aws"
}
$ bundle exec facter virtual
kvm

This should resolve https://github.com/puppetlabs/facter/issues/2648


fix(files): Remove use of localstorage.

As discussed this removes the local tab in favor of the
all and pinned tabs. Also makes the drop area for files
visible at all times.
Closes #32
@jbenet @whyrusleeping please take a look if this is okay for listing files for now or if there are better methods.

I think thats good for now. It would be cool to list things inside ipfs files in the future

@whyrusleeping why don't you open an issue for that :P

How does it list all? ipfs refs local ?

:+1: for removing local storage

@jbenet this is how it does it currently:

pinned: https://github.com/ipfs/webui/blob/fix/remove-local-storage/app/scripts/pages/files.js#L35
all: https://github.com/ipfs/webui/blob/fix/remove-local-storage/app/scripts/pages/files.js#L42

(that was like this before) would refs local be better? What's the exact difference?

this is fine i think

a files tab may want to list the ipfs files root
i havent looked extensively at the code-- @diasdavid maybe?

LGTM from a UX perspective

Other than the one comment, LGTM :) nice!


make laid power cables more visible

About the PR

I offset cable sprites
Why / Balance

Adds a lot of visibility. Neither cables nor pipes are now obscured by the other or by catwalks when laid down together.
Media

old: either cables or pipes are obscured, depending on which became first. new: both are clearly visible.

new

Requirements

[X] I have read and am following the Pull Request and Changelog Guidelines.
[X] I have added media to this PR or it does not require an ingame showcase.

Breaking changes

n
Changelog

:cl:
tweak: Power cables on the ground are now offset and do not obscure each other or pipes beneath.

Looks great. What happens if you rotate the camera? Does the offset rotate with it?

thank you my child

Looks great. What happens if you rotate the camera? Does the offset rotate with it?

Tested and the offset is anchored to the sprite, it does not float around unexpectedly when rotating camera.

Blessed PR

This sucks ingame
cables display over tiles nearby now, and if they're under a window they stick out to the side. They also drift around when you're moving.

Could you make an issue with in-game screenshots of the problems?

oh i see the issue... the offset is literally done by offsetting the cable sprites in yaml, this has an unintended effect of making cables no longer align with grid, hence them sticking out onto nearby tiles, the proper way to do it would be to edit the actual sprite pngs.

this should be reverted and re-done by editing the sprites. I didn't even realize that you could do it this way.

I'm in the middle of fixing it right now.


Issues with sonarr

For some reason recent updates has broke sonarr. It loads too a blank page most of the time, some times it'll let me refresh in. I've asked for support from them and it's gotten no where. I'm hoping you can chime in at this point. They wanted me too add this set_proxy_header Host $host; too my conf witch I did and it gave me a http 400 error when trying to load.. any help would be great 😊

Here's the error I'm getting
[error] 440#440: *1 recv () failed (104: Connection reset by peer)
while reading response header from upstream, client: <IP_ADDRESS>, server:
request: "GET /api/v3
/queue?apikey
=&include
Episode=true HTTP/2.0",
upstream: "http
://<IP_ADDRESS>:8989/api/v3/queue?apikey=&include


New Project - Predicting whether bank will give loan or not to a customer

Description!

No description provided.

Here's the description :
Banks before giving loan to any applicant usually analyze a certain number of factors like applicant's age, education, salary before giving them loan, my project aims to predict that whether an applicant will be given loan or not.

Kindly assign this issue to me as part of Devscript winter of code

I am assigning this issue to you but I am gonna need the info like from where did you get the dataset and what does the model exactly do and all. @manvi0308

Sure

@manvi0308 also star the project

Sir I want to work on this issue.

Sir I want to work on this issue.

I want to work on this issue

I want to work on this issue

Is this issue open??

Is this issue open??

no

no


Feature Request [Quality of life]: Remove memory addresses/ids of objects from Op versioning

Background
Right now, memory addresses in the repr of instantiated objects cause spurious diffs when versioning Ops in Weave. This makes it difficult to track meaningful changes between runs.

Workaround
I use the following utility method to remove  at <memory_address> from reprs to eliminate those spurious diffs:
import re
def _strip_memory_address_from_repr(obj: object) -> None:
    original_repr = obj.__class__.__repr__
    obj.__class__.__repr__ = lambda self: re.sub(r" at 0x[0-9a-fA-F]+", "", original_repr(self))  # type: ignore[method-assign]

Question
Is there something more foundational that can be done to cover this issue globally?
Code?
Digging around the codebase, are these the appropriate places?
https://github.com/wandb/weave/blob/a76414b5d78785c674c5aa26d6c47b3b5d200c8c/weave/trace/serialize.py#L98-L111
https://github.com/wandb/weave/blob/a76414b5d78785c674c5aa26d6c47b3b5d200c8c/weave/trace/serialize.py#L214-L226
But I guess that wouldn't cover the case of a bad repr in which case an object's id could create a spurious diff as well, ex.

@TeoZosa Thanks for the writeup, this is a great feature request/bug. We had already noted this internally as low-hanging fruit to improve the comparison page. As you have noted, these memory addresses are generated pretty deep in our code capture handling, which is rather delicate.
For now, looks like you have a decent workaround, but we will be working on this in the near future.

@gtarpenning Got it, thanks for getting back to this so fast. Will keep my workaround for the time being. The team has been putting out some pretty major features recently and this one is definitely not as urgent.


Additional Protocol Support

Today's OSS tracing systems are focused on http, the typical enterprise has a lot of other protocols which are critical to the application. Some examples include:
1. JMS
Data can be embedded in message properties without harm, and this allows correlation and identification of the presence of a messaging system. The data can be tagged going in and measured for message put. Similarly, the receiving end can measure the time in the queue and the time it takes to process said message.
2. Native MQ
This is more difficult to handle since native MQ protocols do not have specific areas where message data can be inserted without the fear of breaking the downstream legacy code. There are several tricks here which can help avoid the problem, and there are published recommendations from IBM, Tibco, and others. The problem is that the downstream applications typically ignore the recommendations.
3. AMQP
Similar to JMS, supporting AMQP is possible due to having extensible message formats. This still must be done by the library or programmer to inject the right data into the message.
4. DB Calls (JDBC / ADO.NET / other DB access types)
Although this is not something I've seen tracing tools do today, it could be of use to have similar data going to the database. This is something that Oracle has done internally, for example, the database can automatically record a transaction id with the record for trackability (https://docs.oracle.com/cd/B19306_01/java.102/b14355/endtoend.htm#BEICGHJH). It would depend on the driver and database but could be of use.
5. RMI or other RPC
In Java, these calls would typically be via CORBA or via raw calls to a downstream system. For things like CORBA, there are fields which are available to do this type of thing. Similarly, on Hadoop RPC (https://wiki.apache.org/hadoop/HadoopRpc) this would be possible as well. Some of these would require adding data to a message on one end, and possibly removing it on the remote end to avoid breakage.
6. Protobuf and other binary protocols
These are potentially interesting technologies we could embed into, but once again it depends if we can remove the data before it's processed downstream, or find a way to embed data into the payload without causing issues. This is currently a challenge for APM tools.

We can discuss them when we have settled the HTTP.

Some finding after writing specs for binary, AMQP and MQTT protocols. Two big ones:
Charset of tracestate value
HTTP-imposed limitation on character set of the opaque value of tracestate value doesn't may not be an issue for other protocols, which generally support UTF-8 charset. This potentially may lead to various issues like one may assume UTF-8 charset for messaging protocol and will assume something like urlencoding will be applied to non-ASCII characters. However specification operates with opaque values and does not assume any encoding. So value may be simply dropped by implementation.
One of the way to solve it is in fact suggest some encoding mechanism specifically for HTTP protocol. This will, however, complicate implementations and is hard to test in existing test harness (it only supports binary now).
Parsing of tracestate
Binary protocol and AMQP benefits from having a list of tracestate values, not simply treating it as a string. However, this format adds a requirement for every implementation to have a parsing logic for the tracestate implemented. Even if implementation simply wants to forward it from HTTP to GRPC or AMQP.


k8s namer does not handle ungraceful watch termination

The Kubernetes namer establishes a watch on endpoints within a namespace so it has up-to-date topology information. Watch events are delivered in a streaming HTTP response, and if the connection between linkerd and the k8s API is severed (ie. not terminated gracefully), linkerd may have stale discovery information and never recovers without a restart. 🚑
This test case exhibits the problem. The Go program in there acts as both a mock k8s API and a downstream emojum service.

Fire up linkerd using the config from the Gist and the k8s API mock (go run k8s-fake.go)
Make a (successful) request via linkerd to the emojum service:

$ curl -v "http://localhost:8080/" -H "Host: emojum"

GET / HTTP/1.1
Host: emojum
User-Agent: curl/7.43.0
Accept: /

< HTTP/1.1 200 OK
< Date: Thu, 01 Sep 2016 08:04:26 GMT
< Content-Length: 4
< Content-Type: text/plain; charset=utf-8
🐹

   Observe from the mock's logs that linkerd has made the expected call to list endpoints and establish a watch:

2016/09/01 09:07:06 [<IP_ADDRESS>:8001] /api/v1/namespaces/srv/endpoints
2016/09/01 09:07:08 [<IP_ADDRESS>:8001] /api/v1/namespaces/srv/endpoints?watch=true&resourceVersion=1
2016/09/01 09:07:08 [emojum] /
3. Instruct the mock to tear down the watch connection:
   ```bash
$ curl -v "http://localhost:8001/control?action=terminate"
> GET /control?action=terminate HTTP/1.1
> Host: localhost:8001
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Thu, 01 Sep 2016 08:08:34 GMT
< Content-Length: 0
< Content-Type: text/plain; charset=utf-8

Observe in the mock's logs that the watch was terminated, and linkerd successfully re-established the watch:
2016/09/01 09:10:02 [localhost:8001] /control?action=terminate
2016/09/01 09:10:02 Watch ended: terminated
2016/09/01 09:10:02 [<IP_ADDRESS>:8001] /api/v1/namespaces/srv/endpoints
2016/09/01 09:10:02 [<IP_ADDRESS>:8001] /api/v1/namespaces/srv/endpoints?watch=true&resourceVersion=1

Note: This step isn't really necessary, it's just to illustrate that linkerd is correctly handling more graceful termination
4. Now, forcefully stop (Ctrl+C/SIGINT) the mock process and restart it. Notice that this time, linkerd does not re-establish the watch: 😱
signal: interrupt
$ go run k8s-fake.go
2016/09/01 09:12:18 Listening on :8001…

From what I've seen, a ChannelClosedException is returned by the request reader, but the code to re-establish the watch is never run. I mentioned what I thought was the cause of this issue on Slack yesterday, but I think I was quite off-base and I don't really understand how the exceptions propagate internally. I'll be fascinated to see how to solve this; I've really enjoyed poking at the internals although I think I need some help from the experts. 👷

Thank you for a very detailed repro!

Thanks for the repro, @obeattie!  I think I now see what you meant the other day about AsyncStream and exceptions.  An exception in an AsyncStream will effectively halt traversal over the stream.  I think we need to make sure that exceptions don't make it into the stream, even if we encounter them while reading from the Reader.  I'll put together a unit test and fix, and make sure to validate it against the repro steps you posted.


Preview pane keeps jumping back to editor cursor position when someone else is editing

Description
When I am scrolling in the preview pane while someone else is editing, the preview pane keeps taking me back to the place where my cursor is in the editor.
This makes it impossible to peruse the document while someone else is editing.
Reproduction URL
No response
Browsers
Firefox
OS
macOS

This is a bug we need to fix. Meanwhile, note that you can disable the jumping through "View -> Scroll on Type".


parse new xml in travis log

fix #6
You might wanna change your .yaml first, since it does not use our new style.

Can you rebase to master? (for SmalltalkCI)

@chistopher this wasn't a rebase

@Nef10 You're right, that's not a rebase, but I think it is fine anyway. Thanks fro your work! :)