UI: Fixes UI bug

Description
This PR for fixes some UI Bugs

[x] The project selected color is not displayed after switching project
[x] Project view, the cards are not separated but stick together
[x] Dashboard: The color of the selected zone is not displayed
[x] Dark mode: Hover through row tables that are discolored

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[x] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale

[ ] Major
[x] Minor

Bug Severity

[ ] BLOCKER
[ ] Critical
[ ] Major
[ ] Minor
[ ] Trivial

Screenshots (if appropriate):

How Has This Been Tested?

@blueorangutan ui

@utchoang a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

AFTER Fixed:

UI build: :heavy_check_mark:
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6261 (SL-JID-1376)


chapter 06 - summarization - processing the entire dataset

Information
The question or comment is about chapter:

[ ] Introduction
[ ] Text Classification
[ ] Transformer Anatomy
[ ] Multilingual Named Entity Recognition
[ ] Text Generation
[X] Summarization
[ ] Question Answering
[ ] Making Transformers Efficient in Production
[ ] Dealing with Few to No Labels
[ ] Training Transformers from Scratch
[ ] Future Directions

Question or comment
Great  book.
My question is very simple :
How can I extend the summarizing process for the entire dataset.
I.e. , from the first row:
sample_text = dataset["train"][1]["article"][:2000]
To all rows.
Apologies if sounds very silly.

sample_texts = [article[:2000] for article in dataset["train"]["article"]]

def shorten_article(example):
    example["article"] = example["article"][:2000]
    return example

dataset["train"] = dataset["train"].map(shorten_article)

not sure if these work. try em out


[build] Support Tiny C compiler

This improvement has a double pourpose, first just offer an alternative to the already supported C compilers (GCC, clang, MSVC), second, offer a really tiny lightweight environment to play with raylib.
raylib is provided to students as an installer including the library, a preconfigured version of Notepad++ and MinGW package. This installer is pretty big and I'd like to reduce its size. Current MinGW package is around 200 MB; in comparison, TCC with basic libraries is just around 3 MB.

I also managed to build raylib with TCC on Windows 7. I added some headers from winapi-full-for-0.9.27.zip , some are from TDM-GCC I had installed on Windows, and some are from mingw git repo. I made some changes in couple of header files and added some missing defines. The zip file with everything included is available here https://gist.github.com/gen2brain/ea182882eaa1de337cc6aebfbc9a84fb , you can just unzip it to c:\TCC and add that dir to PATH.
All system .def files are regenerated, and opengl32.def is added, it seems that needs to be done so missing math functions and functions from shell32, user32 etc. is corrected with this.
In src Makefile I changed CC to tcc -DSTBI_NO_SIMD and AR to tcc -ar.
In examples Makefile also CC, and I had to add -lkernel32 -lshell32 -luser32 . I also regenerated raylib.rc with windres -O coff raylib.rc -o resources.o (note the .o for object, also changed accordingly in Makefile). And I removed LDLIBS += -static -lpthread (needed for physac, you must remove at least -static from there).
Note that TCC will compile and link examples with raylib.dll that is compiled with gcc, but that is not supported and will crash. To be sure, remove all libraries from release directory and then compile one with TCC.
Now, the only problem left is this error, it is reported for all functions from raymath, e.g.
../release/libs/win32/mingw32/libraylib.a: error: 'Clamp' defined twice
../release/libs/win32/mingw32/libraylib.a: error: 'Vector2Zero' defined twice
../release/libs/win32/mingw32/libraylib.a: error: 'Vector2One' defined twice
../release/libs/win32/mingw32/libraylib.a: error: 'Vector2Add' defined twice

Apparently, support for inline functions is lacking:

A flag won't help, I'm pretty sure this is an inherent limit of TCC.
The lack of stored parse information about functions means that the
information that TCC would need to perform "proper" inlining just
isn't available to the compiler when the inlining must be done,
because it isn't kept anywhere.

Ideally, there would be a tcc --ignore-duplicate-symbols flag. Lacking that, we will have to either change the extern inlines to static inline or remove inline altogether. I will draft a fix.

WOW! UNBELIEVABLE! It sounds SUPER-AMAZING to me!
As soon as we get it working I could prepare a slim-installer-package for students!

What is also super cool is that when tcc is built with --enable-cross you get this:
/usr/bin/arm-tcc
/usr/bin/arm-wince-tcc
/usr/bin/arm64-tcc
/usr/bin/c67-tcc
/usr/bin/i386-tcc
/usr/bin/i386-win32-tcc
/usr/bin/tcc
/usr/bin/x86_64-osx-tcc
/usr/bin/x86_64-tcc
/usr/bin/x86_64-win32-tcc

I built tcc on Windows with build-tcc.bat but should be possible also on Windows. So you can for example have profile for cross compile for RPi, or something like that :)

Could you take a look at the resulting binary with Dependency Walker? It might show some missing libraries/symbols.

@gen2brain GLFW issue is related to VM, seen it before, OpenGL 3.3 is not supported on VM environment. It should work ok on Windows. Didn't know about cross-compiler build option, AMAZING!
About mini_al issue when compiling with TCC, probably @mackron should be aware of that.

Fix for audio examples works, now they compile.
I tried dependency walker but whole vmware crashes if I try snipping tool or just printscr button to take a screenshot. Here are some of the errors/warnings:
Error: At least one module has an unresolved import due to a missing export function in an implicitly dependent module.
Error: Modules with different CPU types were found.
Warning: At least one delay-load dependency module was not found.
Warning: At least one module has an unresolved import due to a missing export function in a delay-load dependent module.

API-MS-WIN-APPMODEL-RUNTIME-L1-1-0.DLL
API-MS-WIN-CORE-WINRT-ERROR-L1-1-0.DLL
API-MS-WIN-CORE-WINRT-L1-1-0.DLL
API-MS-WIN-CORE-WINRT-ROBUFFER-L1-1-0.DLL
API-MS-WIN-CORE-WINRT-STRING-L1-1-0.DLL
API-MS-WIN-SHCORE-SCALING-L1-1-1.DLL
DCOMP.DLL
GPSVC.DLL
IESHIMS.DLL

For all these it reports Error opening file. The system cannot find the file specified (2)

And it works OK in VM if I compile with GRAPHICS_API_OPENGL_11 ;)

Oh, apparently Dependency Walker hasn't kept up with newer Windows versions and these are just false positives...

And it works OK in VM if I compile with GRAPHICS_API_OPENGL_11 ;)
Great!

And it works OK in VM if I compile with GRAPHICS_API_OPENGL_11 ;)

That's great, so, it should work ok on non-VM Windows.
What about audio? Does it work?

Tried to run it on Windows 10 64bit, it seems TCC needs to be recompiled for my OS:

This version of C:\raylib\tcc\tcc.exe is not compatible with the version of Windows you're running. Check your computer's system information and then contact the software publisher.

@raysan5 audio_music_stream and audio_sound_loading both works, raw_stream I think I disabled because of some issues, and that is with #define MAL_NO_WASAPI, I will see if I can also include audioclient.h and deps.
Compile tcc for your system, but copy includes from my zip file. And you should probably just cd to tcc\lib and do the tcc -impdef opengl32.dll and also overwrite all .def files like this.

My results:

[x] Recompiled tcc (latest version from tccrepo)
[x] Regenerated .def (opengl32, gdi32, kernel32, user32, msvcrt)
[x] Copied your library files
[x] Defined -DMAL_NO_WASAPI for audio module compilation
[x] Compiled raylib
[ ] Compile example --> FAIL >

tcc -o core_basic_window.exe core_basic_window.c -Iexternal -I../../src -L../../src -lraylib -lopengl32 -lgdi32 -lkernel32 -lshell32 -luser32 -std=c99
tcc: error: undefined symbol 'sqrtf'
tcc: error: undefined symbol 'asinf'
tcc: error: undefined symbol 'cosf'
tcc: error: undefined symbol 'sinf'
tcc: error: undefined symbol 'atan2f'
tcc: error: undefined symbol 'powf'
tcc: error: undefined symbol 'logf'
tcc: error: undefined symbol 'ceilf'

Tried also with -static to link default libc:
tcc -o core_basic_window.exe core_basic_window.c -Iexternal -I../../src -L../../src -static -lraylib -lopengl32 -lgdi32 -lkernel32 -lshell32 -luser32 -std=c99
tcc: error: library 'opengl32' not found

Those math functions are included in msvcrt.def in Windows 7, but I had the same error, it was gone after I did tcc -impdef msvcrt.dll in lib dir.
You cannot compile with -static, and all system libs must be used via def files. But you can use raylib.a static lib, just the binary cannot be fully static. At least that is how I understood.

Tried the same process in my Surface with Windows 10 64bit and I get similar errors, also tried proposed solution but I keep getting errors:
tcc -o core_basic_window.exe core_basic_window.c -I../../src -Iexternal -L../../src -lmsvcrt -lraylib -lopengl32 -lgdi32 -luser32 -lkernel32 -lshell32 -std=c99
tcc: error: undefined symbol 'sqrtf'
tcc: error: undefined symbol 'asinf'
tcc: error: undefined symbol 'cosf'
tcc: error: undefined symbol 'sinf'
tcc: error: undefined symbol 'atan2f'
tcc: error: undefined symbol 'powf'
tcc: error: undefined symbol 'GetClassLongPtrW'
tcc: error: undefined symbol 'logf'
tcc: error: undefined symbol 'ceilf'

On this computer I also got an additional undefined symbol! That's weird...
As per my understanding, generated .def do not include those symbols, so I opened them in Notepad++ (they are plain text files) and added the symbols manually; compiled again and everything worked!
So, it seems generated .def do not use right version of those .dll (or use the first version they get access to), then the generated executable uses the right .dll of the system...

@a3f I've integrated your fix upstream (https://github.com/dr-soft/mini_al/commit/02b7ef1c2c3fbc68d06c4c6e84cf202d9e01d903).
@gen2brain

I added in mini_al.c #define MAL_NO_WASAPI , because audioclient.h etc. headers are not included, there were some issues with those.

The next version of mini_al does not depend on audioclient.h, etc. so #define MAL_NO_WASAPI should become unnecessary.
Any other mini_al specific issues here that I've missed?

Any other mini_al specific issues here that I've missed?

@mackron, nothing else for the moment! :)

Tried on Windows 10 32bit and, again, I got the missing symbols issue:
tcc: error: undefined symbol 'sqrtf'
tcc: error: undefined symbol 'asinf'
tcc: error: undefined symbol 'cosf'
tcc: error: undefined symbol 'sinf'
tcc: error: undefined symbol 'atan2f'
tcc: error: undefined symbol 'powf'
tcc: error: undefined symbol 'logf'
tcc: error: undefined symbol 'ceilf'

But this time, adding them to msvcrt.def is not enough, program compiles but when trying to run it, it complains about not finding sqrtf. That's very weird... any idea?

My guess: MSVCRT defines the float math functions as inline functions that expand to the double version, e.g. inline float sqrtf(float f) { return (float)sqrt((double)f); }, but MinGW headers define them as external symbols and that mismatch causes the missing symbols.

Great guess @a3f! I sounds reasonable to me... but then, why on Windows 10 64bit it works? Maybe MSVCRT.dll 64bit exposes those symbols but not MSVCRT.dll 32bit? Do you know how could I check that? Any proposed solution for that issue? Maybe I can just define my own float functions the same way?

Possible fix: amend tcc's <math.h> with
static inline float sqrtf  (float f)            { return (float)sqrt(f); }
static inline float asinf  (float f)            { return (float)asin(f); }
static inline float cosf   (float f)            { return (float)cos(f); }
static inline float sinf   (float f)            { return (float)sin(f); }
static inline float atan2f (float f1, float f2) { return (float)atan2(f1, f2); }
static inline float powf   (float f1, float f2) { return (float)pow(f1, f2); }
static inline float logf   (float f)            { return (float)log(f); }
static inline float ceilf  (float f)            { return (float)ceil(f); }

why on Windows 10 64bit it works

No idea. That's why it's a guess!

you know how could I check that?

DLL walker noted further up is what I used so far for checking exported symbols on Windows.

Any proposed solution for that issue?

Check out what MSVC's <math.h> actually defines/declares.

Maybe I can just define my own float functions the same way?

I'd edit <math.h> and try adding
static inline float sqrtf  (float f)            { return (float)sqrt(f); }
static inline float asinf  (float f)            { return (float)asin(f); }
static inline float cosf   (float f)            { return (float)cos(f); }
static inline float sinf   (float f)            { return (float)sin(f); }
static inline float atan2f (float f1, float f2) { return (float)atan2(f1, f2); }
static inline float powf   (float f1, float f2) { return (float)pow(f1, f2); }
static inline float logf   (float f)            { return (float)log(f); }
static inline float ceilf  (float f)            { return (float)ceil(f); }

MSVC's math.h defines:
#if defined _M_X64 || defined _M_ARM || defined _M_ARM64
    ...
    _Check_return_ _ACRTIMP float __cdecl sqrtf(_In_ float _X);
    ...
#else
    ...
    _Check_return_ __inline float __CRTDECL sqrtf(_In_ float _X)
    {
        return (float)sqrt(_X);
    }
    ...
#endif

So, as _M_X64 is defined for x64 processors, in Windows 10 64bit, those functions are compiled with symbols but on Windows 10 32bit, those functions are inlined, so no symbols on MSVCRT.dll 32bit.
But TCC math.h that comes from MinGW defines it correctly for #ifndef __x86_64 to inline functions... or that's apparently what should happen with:
__CRT_INLINE float sqrtf(float _X) { return ((float)sqrt((double)_X)); }

Let's take a closer look... _mingw.h defines:
#define __CRT_INLINE extern __inline__

But __inline__ is a non-standard extension; we can't say with certainty if its supported by the compiler.
Moreover, digging a bit further, I found this statement:

TCC doesn't support function inlining, because the 'inline' keyword is ignored, so if a function needs to be inlined, you must use defines instead.

I love to discover and understand how things work! :D
Now, let's implement a solution.

The inline in static inline is just a hint anyway, so it's ok that tinycc ignores it. It shouldn't cause an error or miscompilation (<raymath.h> uses `static inline definitions as well for tinycc).
Use of #defines here would be wrong. The standard requires externally visible sqrtf, acosf ... etc. definitions for hosted implementations. Otherwise, you couldn't take pointers to these functions.

Just tried using static inline funtion declarations on math.h, it works.

TCC compiler support is complete, tested on win32 and win64. CLosing this issue.

Just unzip the right binary release from http://download.savannah.gnu.org/releases/tinycc/ and you're done...


implement (de)serialize_with for variants

Work in progress for #1013. So far I've implemented the serialize_with half. Any early feedback would be appreciated.

@dtolnay That should wrap up the implementation (no longer WIP).

The clippy thing is unfortunate. @dtolnay do you think it would make sense to allow derefs in serialize_with and deserialize_with? We'd have to generate an intermediate function/closure at the expanded call site, but that should be doable.

We already allow derefs in serialize_with and deserialize_with. That is why https://github.com/serde-rs/serde/issues/631#issuecomment-279271870 works, for example. The tests added in this PR can be changed as clippy says: use &str instead of &String.
The generated code is doing the equivalent of this, which works:
struct __SerializeWith<'__a> {
    values: (&'__a String, &'__a u8),
}

fn main() {
    let sw = __SerializeWith { values: (&"".to_owned(), &0) };
    serialize_some_other_variant(sw.values.0, sw.values.1);
}

fn serialize_some_other_variant(_: &str, _: &u8) {}

Oh, so you can. I didn't realize that. I've pushed a fix.


Tomee 7.0.x

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?


IGV omit setting defaults not correctly set on project creation

Yak shaving returns. While looking into #1924 I noticed that the site-wide IGV omit settings are not correctly set when creating a project.
The defaults get used when building the study cache for cache and germline projects, even if the values appear empty in the project update form. This is obviously wrong.

Fixed.


deckgl style functionality

It would be good to be able to render 2d and 3d elements within the 3d maps.
I dont know how is it is to inject 3d objects into the rendering framework though. Anyone know ?

@gedw99 vector tiles have "pseudo" 3d capabilities (often called 2.5d) which is achieved by encoding attributes into features like height that the client side renderer can use for extrusions. I know Mapbox can render 3d object but I believe they're introduced as external files and are not part of the vector tiles themselves. Here's some documentation
3D terrain is usually done with DEM raster tiles, so again not specific to tegola.


Only snapshot frame visits that have actions

Frames that use data-turbo-action to promote their navigations to page visits require cache snapshots to be taken prior to them loading, in order for those snapshots to be accurate.
However snapshots can be expensive to create for complex pages. In cases where a frame is not being promoted, we can avoid taking that snapshot.
This helps with some performance issues observed when using frames on very large pages.
/cc @afcapel

To pass the test suite in CI, you might want to review and merge https://github.com/hotwired/turbo/pull/822, then rebase.


Feedback!

Thanks for providing this library!
I couldn't see how to feedback on my experience, but I just wanted to convey that using the library has been very pleasant. Here's a code snippet of how I set up my cache to read secrets from Hashicorp's Vault:
let cache: TtlCache = LoadingCache::with_meta_loader(
    TtlCacheBacking::with_backing(
        *unauthorized_timeout,
        LruCacheBacking::new(max_secrets_cached),
    ),
    move |secret_path| {
        let task_state = Arc::clone(&state);
        async move {
            let result = task_state
                .client
                .get(
                    task_state
                        .server
                        .join(&format!(
                            "{}v1/secret/data/{}",
                            task_state.server, secret_path
                        ))
                        .unwrap(),
                )
                .bearer_auth(&task_state.client_token)
                .send()
                .await;
            if let Ok(response) = result {
                let secret_reply = if response.status().is_success() {
                    response.json::<GetSecretReply>().await.ok()
                } else {
                    None
                };
                let lease_duration = secret_reply.as_ref().map(|sr| {
                    let mut lease_duration = None;
                    if let Some(ttl_field) = task_state.ttl_field.as_ref() {
                        if let Some(ttl) = sr.data.data.get(ttl_field) {
                            if let Ok(ttl_duration) = ttl.parse::<humantime::Duration>() {
                                lease_duration = Some(ttl_duration.into());
                            }
                        }
                    }
                    lease_duration.unwrap_or_else(|| Duration::from_secs(sr.lease_duration))
                });
                Ok(secret_reply).with_meta(lease_duration.map(TtlMeta::from))
            } else {
                Err(Error::Unavailable)
            }
        }
    },
);

Feel free to use this as a more (incomplete) complex example if that helps. Thanks once again!

Hey! Sorry -  Somehow this notification got lost in my inbox, - just some PR popped in and I saw this. Thank you for providing this example, I will see if I can link it somewhere!


PP-13030: Use enums for the role name

When creating a new target service (which will only have a Worldpay test account) and is branched off from an original source service, we want to ensure the admin users on the source service are also admins on the new target service. To this end we want to get admin users for the existing service. We can get all users for a service using GET /v1/api/services/{serviceExternalId}/users, but we'd like to introduce a query parameter of role=admin for example. Doing this has the advantage of transferring less data across the wire, as the User object is quite large. To this end this PR makes the role an enumerated type which will provide validation on the role query parameter.
This PR:

Introduces a RoleName enum
Deletes the RoleDbFixture. Roles are initialised into the database, so where there are integration tests, there is never a need to create custom roles. We should use the predefined roles.
Deletes the RoleEntityFixture for the same reason as above.

This PR does not:

Add a role query parameter to GET /v1/api/services/{serviceExternalId}/users
Enum-ify the role description.

I’m in favour of this change. Just a few minor things.

Yay 🥳


changes to get dream.py working on M1

use MPS specific fork of k_diffusion
move all device init logic to T2I.init
handle m1 specific edge case with autocast device type
check torch.cuda.is_available before using cuda

~Besides these changes, there is still an open question about what to do will the requirement to use --full_precision for M1. See discussion here for details: https://github.com/lstein/stable-diffusion/issues/237~ This is now addressed in the second commit.

Thanks! Confirmed it works for me. (Clean install, M1 Max MBP, miniforge environment)

@Birch-san's k-diffusion prevents conda env create from finishing on my computer. I've got miniconda. I know others have gotten it to work so I'll have to debug it.
dream.py runs w/o crashing but produces black images, probably because of k-difusion.

@magnusviri

@Birch-san's k-diffusion prevents conda env create from finishing on my computer. I've got miniconda. I know others have gotten it to work so I'll have to debug it.
dream.py runs w/o crashing but produces black images, probably because of k-difusion.

Try this:
https://github.com/lstein/stable-diffusion/issues/63#issuecomment-1227849008

I confirm this works out of the box, for some reason I could make it work by just running conda env update -f environment-mac.yaml - server then just hanged on image generation with no error.
But after a real clean conda env it works flawlessly on my m1 max.

Crashes on CUDA systems during image generation with "expected scalar type Half but found Float". The problem is this line in choose_autocast_device():
    if device_type != 'cuda' or device_type != 'cpu':
        return 'cpu'

Since the device can't be both cuda and cpu, this will always return "cpu". The fix is trivial, and I'll take care of it, but the take-home message is to always run the thing to make sure it is working! (I don't always follow that advice myself, so I know how it is).

@lstein I only have a 2GB CUDA capable GPU and it's currently not installed in anything. It was easier for me to use my Mac :)
However, I've actually been thinking of setting up GitHub actions. I don't think they provide GPU's though.

Crashes on CUDA systems during image generation with "expected scalar type Half but found Float". The problem is this line in choose_autocast_device():
    if device_type != 'cuda' or device_type != 'cpu':
        return 'cpu'

Since the device can't be both cuda and cpu, this will always return "cpu". The fix is trivial, and I'll take care of it, but the take-home message is to always run the thing to make sure it is working! (I don't always follow that advice myself, so I know how it is).

doh! should have been and


Negative Response Caching with Proxy.config.http.negative_caching_lifetime vs TTL-in-Cache

cache.config
url_regex=.*    ttl-in-cache=20s

records.yaml
records:
  http:
    negative_caching_enabled: 1
    negative_caching_lifetime: 5

ttl-in-cache in cache.config sets a default TTL, applying this duration to cached responses regardless of the response code.
proxy.config.http.negative_caching_lifetime in records.yaml is intended to set the TTL for negative responses (e.g., 403, 404). However, if ttl-in-cache is defined in cache.config, it takes precedence and applies a TTL of 20 seconds to all responses, including 403.
Is there any way to use the value set by proxy.config.http.negative_caching_lifetime instead of the ttl set by ttl-in-cache for negative responses?

The following configuration works as I wanted it to.
It would be better if there was a secondary specification called 'status' in the cache.config file.
(e.g dest_domain=example.com status=403 ttl-in-cache=60s)
If the default TTL is set to 1 day and negative response is set to 60 seconds
Cache-Control and Cache.config are not configured.
records.yaml
proxy.config.http.cache.heuristic_min_lifetime : 86400
proxy.config.http.cache.heuristic_max_lifetime : 86400

proxy.config.http.negative_caching_enabled : 1
proxy.config.http.negative_caching_lifetime : 60

Unfortunately, it's unlikely anyone from the current development team have interest in adding features to caching.config. Rather, what people tend to do these days is using the header_rewrite plugin, with configurations such as
cond %{READ_RESPONSE_HDR_HOOK} [AND]
cond %{STATUS} >199 [AND]
cond %{STATUS} <299
    set-header Cache-Control "max-age=20"

with an optional rule as well (removing CC to clients)
cond %{SEND_RESPONSE_HDR_HOOK} [AND]
cond %{STATUS} >199 [AND]
cond %{STATUS} <299
    rm-header Cache-Control

(or even just always remove the CC header regardless). Using header_rewrite here, rather than the cache.config, has many benefits:

It's much faster than evaluating one or many regular expressions. Although you can of course also do regexes here if you want, to only apply caching to some specific URLs.
With header_rewrite (or Lua or Cripts) you have a lot more flexibility of making complex rules.
Safer. You limit the rules to only ever apply to the remap line that it's configured for.

Probably not the answer you wanted, and likely we ought to eliminate cache.config in some future version.


csv button giving error on console

Hey, Thanks for this awesome library! I'm instantiating a datatable using the buttons extension and getting the following error on the console when that button is clicked:
Error: a[e].replace is not a function
TypeError: a[e].replace is not a function
    at i (buttons.html5.min.js?ba315e4f178f1329ec9e90aedf7d3d1156fb9bb5:7)
    at B (buttons.html5.min.js?ba315e4f178f1329ec9e90aedf7d3d1156fb9bb5:7)
    at Object.n.ext.buttons.csvHtml5.action (buttons.html5.min.js?ba315e4f178f1329ec9e90aedf7d3d1156fb9bb5:13)
    at HTMLAnchorElement.<anonymous> (dataTables.buttons.min.js?7b8b315ccd79137061a5c682776a47eac9033fec:13)
    at HTMLAnchorElement.jQuery.event.dispatch (jquery.js:4665)
    at HTMLAnchorElement.jQuery.event.add.elemData.handle (jquery.js:4333)

any help appreciated, best

Can you try the version in this repo please, if you haven't already. If you have and it is still causing issues, please link to a test case showing the issue.

I believe this 'error' situation is being triggered when DataTables is configured to hide certain columns, then when pressing the CSV button, it will iterate through all the columns requested through the data property and it will iterate over a column that is not visible.
So the quick answer would be to ensure that your configuration is aware of these hidden columns and adjust the initialization part of your buttons:
                exportOptions: {
                    columns: ':visible'
                }

You need to add this to the extend part of the CSV button like so:
                    {
                        extend: 'csv',
                        exportOptions: {
                            columns: ':visible'
                        }
                    }

Hope this helps,

I just experienced the same problem (with DT 1.10.9) -- tracked it down to a member of my returned ajax json being an array instead of a single value.  The table itself still worked just fine with that value, but CSV export did not.  Ensuring it was a single value cleared it all up for me.

I have used the last version of Buttons (1.0.3) and DataTables(1.10.9) and i have the same issue.
My solution is to cast the data to String for the Export.
Something like:
                {
                    "data": "id",
                    "render": function ( data, type, full, meta ) {
                        if (type !== 'display') {
                            return String(data);
                        }
                        return customFoo(full, meta);
                    }
                },
                ...

Note: the button use 'filter' Orthogonal data type

Have you tried using the nightly version. It should already be fixed there.

Just tried and it fix the conversion problems but introduce 2 bug.

csvhtml5 --> first title have an initial strange char

ï»¿"first_title"

excelhtml5 --> convert boolean to 0 for both true and false

i have tried on ubuntu 14.04 with Firefox 41.0.1 and Chrome 45.0.2454 with the same results

That is the UTF8 BOM. Most spreadsheets and text editors should handle it correctly. If you don't want the BOM at all - then you can change the character set - code.

It does convert? Can you please link to a test case.

I have created a demo with the less possible noise here:
http://live.datatables.net/polagopi/1
I can't reproduce the UTF8 BOM problems and that are probably some problems with mine server(Django) but the conversion to 0 is present for the column 'Global'.
With this example the excel export give a directory instead of a file with Firefox.

Committed fix for the boolean issue.
Updated example: http://live.datatables.net/mocazaku/1/edit

excelhtml5 --> null and objects are convert to 0
csvhtml5 --> object are displayed as [object Object]

Objects shoudl be [object Object] as they effectively are turned into strings. null shouldn't be displayed in a database - however the output will now be "null" - i.e. null+''.


update nix flake and ci

#329  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @fracek and the rest of your teammates on  Graphite


Visualization problems

Hello, I meet a problem. I can not display the algorithm as yours. How can I visualize it? Do I need to install something else? Looking forward to your reply! Thank you!

自问自答了，安装ompl库，并且把ompl的目录放进sst里面即可。


Add support for did:tz:kt1

Discussed on today's call -- last commit in this PR diverges too far from current spec, reverted to honor current spec.  Tracking issue opened for potential spec changes that would make a more efficient implementation possible along these lines.


Merge refs/heads/dependabot/go_modules/update-dependencies/github.com/stretchr/testify-1.8.4 into alpha

Merge refs/heads/dependabot/go_modules/update-dependencies/github.com/stretchr/testify-1.8.4 into alpha

This PR has not been merged into alpha. Please fix the issues and re-open the PR.

This PR has been merged into alpha. Thanks for your contribution!


pipenv on Windows10

Hi @dfurtado,
Is it a must to use pipenv for this chapter? I am having hard time create virtualenv using pipenv. I am having nasty errors. Can I use "python3.6 -m venv virtualenv" and go from there to complete the course? Shall I face issue during deployment to AWS if I don't use pipenv?
Regards

Hi @armaanreza ,
Not, not at all. Pipenv is just a new tool that tries to simplify the development workflow so developers create the virtual environment and install packages using the same tool. You can still use python3.6 -m venv virtualenv and then use pip to install the project's dependencies in your virtual environment.
There won't be any problem to deploy the project to AWS.
Maybe it would be a good idea to report the issue you are having with pipenv in the pipenv's GitHub, it can be helpful to the team working on it.
Best regards


Capacitor banks misbehaving with tesserack after relog

Hi,
I have found a really odd issue. Capacitor banks behaves oddly after relogging into map. I have done some tests and it seem do only if TE Tesserack is connected into it. Even via conduit.
Tested with:
EnderIO-1.7.10-<IP_ADDRESS>7_beta
EnderIO-1.7.10-<IP_ADDRESS>8_beta
EnderCore-1.7.10-<IP_ADDRESS>_beta
Other mods used to test with:
CoFHCore-[1.7.10]3.0.3-303
ThermalDynamics-[1.7.10]1.1.0-161
ThermalExpansion-[1.7.10]4.0.3B1-218
ThermalFoundation-[1.7.10]1.2.0-102
Setup / steps to reproduction:
http://imgur.com/wNjoqyQ
From left to right:

creative energy cell (set to max output 2000 rf/t)
"imput" tesseracks (both set to same frequency 1, left one set to send, right one to receive energy)
capacitor bank (1x3x3, left side set to pull, right one to push, make sure you have the energy "screens" like in the picture)
"output" tesseracks (both set to same frequency 2, left one set to send, right one to receive energy)
resonant energy cell (set to max input 1000 rf/t)
In the picture we have two sides. On the left side of the capacitor bank is input, this side charges it. Right side is output, it sends energy from it into resonant energy cell.

The issue:
Normally, it will show that it is receiving 2000 rf and sending 1000 rf (on the vertical screen which say both in / out; or 1000 rf on the small one). Exit the map and then go back into it. Screen on the battery show 1000 rf in and 2000 rf out. Input has decreased by 50% and output increased 2X. However, battery is still charging and so does the resonant energy cell with no change (is charging with 1000 rf/t, not with 2000 as CapBank screen say) Again, Relog. Battery show different values, again. Again, input decreased by 50% and output increased by 2X. Waila and CapBank gui will also show the same exact incorrect values.
After relogging, CapBank shows that it is outputting 2000 rf/t even if the resonant energy cell is set to only accept 1000 rf/t. Set the input to 0. CapBank is still outputting 1000 rf/t. Disable (remove / provide redstone signal..) Tesserack that is on the right side of the CapBank. It will stop outputting energy.
Other notes:

this happens even if Tesserack is connected to CapBank via energy cable / conduit
it seem to happen with TE Tesseracks only
Transdimensional Receiver doesn't seem to be affected
it can be fixed by restarting the game

Can you do us a favor and try your test setup with this debug version? http://loenwind.info/EnderIO-1.7.10-<IP_ADDRESS>8pre2_beta.jar
It will output information about what the cap bank sends into the logfile. Please check what it says. Thank you.
(This version is compiled with Oracle's JDK, which has a bug. So it needs Immibis Microparts to be installed if you want to place conduits.)

I can't upload the whole log because it's too big. But there are few excerpts from it:
Notes:

same exact setup
I'm referring to Tesserack that is connected on the right side of the battery that is set to Push, On/Off refers to its settings, whether it's sending on low redstone signal (on) or high redstone signal (off)
also I'm referring to Resonant Energy Cell and its input settings

Normal, Tesserack off
[21:45:09] [Server thread/INFO] [STDOUT]: [crazypants.enderio.machine.capbank.network.CapBankNetwork:sendPowerTo:300]: Tick 3252/id=0: Offered 225000 RF to cofh.thermalexpansion.block.ender.TileTesseract@46fe0a73 at x=816 y=56 z=-576. It took 0 RF.
Normal, Tesserack on, Cell max input 1000 RF
[21:45:56] [Server thread/INFO] [STDOUT]: [crazypants.enderio.machine.capbank.network.CapBankNetwork:sendPowerTo:300]: Tick 3439/id=0: Offered 225000 RF to cofh.thermalexpansion.block.ender.TileTesseract@46fe0a73 at x=816 y=56 z=-576. It took 1000 RF.
1st relog, Tesserack on, Cell max input 1000 RF
[21:46:59] [Server thread/INFO] [STDOUT]: [crazypants.enderio.machine.capbank.network.CapBankNetwork:sendPowerTo:300]: Tick 4181/id=1: Offered 225000 RF to cofh.thermalexpansion.block.ender.TileTesseract@29098bc7 at x=816 y=56 z=-576. It took 2000 RF.
1st relog, Tesserack on, Cell max input 0 RF
[21:47:28] [Server thread/INFO] [STDOUT]: [crazypants.enderio.machine.capbank.network.CapBankNetwork:sendPowerTo:300]: Tick 4288/id=1: Offered 225000 RF to cofh.thermalexpansion.block.ender.TileTesseract@29098bc7 at x=816 y=56 z=-576. It took 1000 RF.
1st relog, Tesserack off
[21:47:49] [Server thread/INFO] [STDOUT]: [crazypants.enderio.machine.capbank.network.CapBankNetwork:sendPowerTo:300]: Tick 4390/id=1: Offered 225000 RF to cofh.thermalexpansion.block.ender.TileTesseract@29098bc7 at x=816 y=56 z=-576. It took 0 RF.
2nd relog did absolutely nothing different
3rd relog, Tesserack on, Cell max input 1000 RF
[21:49:35] [Server thread/INFO] [STDOUT]: [crazypants.enderio.machine.capbank.network.CapBankNetwork:sendPowerTo:300]: Tick 5744/id=3: Offered 225000 RF to cofh.thermalexpansion.block.ender.TileTesseract@474ba46b at x=816 y=56 z=-576. It took 3000 RF.
3rd relog, Tesserack on, Cell max input 0 RF
[21:50:11] [Server thread/INFO] [STDOUT]: [crazypants.enderio.machine.capbank.network.CapBankNetwork:sendPowerTo:300]: Tick 5832/id=3: Offered 225000 RF to cofh.thermalexpansion.block.ender.TileTesseract@474ba46b at x=816 y=56 z=-576. It took 2000 RF.
3rd relog, Tesserack off
[21:51:01] [Server thread/INFO] [STDOUT]: [crazypants.enderio.machine.capbank.network.CapBankNetwork:sendPowerTo:300]: Tick 5949/id=3: Offered 225000 RF to cofh.thermalexpansion.block.ender.TileTesseract@474ba46b at x=816 y=56 z=-576. It took 0 RF.

I'm not sure whether it'll help but I did some tests with Power Monitors.
Same exact setup except there are conduits inbetween and Creative Energy Cell on back of the power monitors to avoid tampering with readings. Sides of the bank that are in touch with them are set to disabled.
Setup: http://imgur.com/egZO0L8
I will refer the Power Monitors (from left to right) as  P1, P2, P3, P4.
Results:
Normal
P1: 2000 RF/t Out, 2000 RF/t In
P2: 0 RF/t Out, 2000 RF/t In
P3: 1000 RF/t Out, 1000 RF/t In
P4: 1000 RF/t Out, 1000 RF/t In
1st relog
P1: 2000 RF/t Out, 2000 RF/t In
P2: 0 RF/t Out, 1000 RF/t In
P3: 2000 RF/t Out, 2000 RF/t In
P4: 1000 RF/t Out, 1000 RF/t In
2nd relog
P1: 2000 RF/t Out, 2000 RF/t In
P2: 0 RF/t Out, ~660 RF/t In
P3: 3000 RF/t Out, 3000 RF/t In
P4: 1000 RF/t Out, 1000 RF/t In
1st and 4th Power monitor didn't show any issue. Only 2nd and 3rd did show some inconsistency.
However, the ones that are connected to Tesserack and battery were giving me the same exact numbers of in/outcomming energy as battery.

Ok, that is helpful. You see the number after "took"? That is what the tesseract actually took. There's absolutely no Ender IO code touching that number. So the tesseract is behaving funny here, the cap bank is doing fine.
(I don't have the code open right now, but the call is something like took=otherTileEntity.receiveEnergy(side, offeredAmound))

Well, I guess you're right. Tesserack is the issue, not the CapBank.
I did some tests with other batteries. Same exact setup as in picture (with conduits and power monitors) I've posted except the in the middle was:

Mekanism, multiblock Induction Cell
Mek, Energy Cube
TE Resonant Energy Cell

After relogging I get same exact result (issue) as above. I should have done these tests before reporting it. My bad.
However, this doesn't happen when the setup is simplified. Remove the middle battery and Tesseracks connected to it (creating Creative Cell > Tesserack > Tesserack > Resonant Cell) and it'll work even after relog.
I'll report or refer it to team COFH and until then, its either Tesseracks and not relogging or Transdimensional Receiver.


Fix image tag markup

replacing alt="image" with alt="" the reason that the images are just for decoration.
replacing srcSet with src. srcset for changing resolutions and not compatible with all browsers.

thank you @Muhnad !!!


Contact form is throwing Exception

After submitting contact form, it is returning
Unhandled Exception: Exception

Message

Could not find template match for "staffNotice.email.tpl", checked paths:

html-templates/contact/staffNotice.email.tpl
html-templates/staffNotice.email.tpl
Code

0
URI

/contact

Request Method

POST

Request Body

Name=Miki&Email=some_test%40email.com&Phone=0911770017&Message=This+is+just+testing.+Ignore+pls.

Referrer

http://staging.codeforcroatia.org/contact

User Agent

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5

This should be fixed now, we needed to pull an email template down from upstream

Thanks!
I'm not getting contact message submitted via /contact page. I've defined my email in ContactRequestHandler.config.php. Anything else needs to be setup?


Create interactive guide that explains how to use CSX

Users would have an easier time understanding CSX if they would be guided through the various parts of the UI at the start of their first session.

Will be implemented in #116


Allow users to disable anonymous tracking from the UI

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/1055
Originally created by @dmor on 2019-09-26 17:14:45

In our documentation we provide instructions for opting out of anonymous tracking from the command line, and we need to also make this available to users in the UI (for users who will never go to the CLI)
https://meltano.com/docs/quickstart.html

View 6 previous comments from the original issue on GitLab


Added new module that publishes the Arquillian extension

This should fix #318
I added a new module asciidoctor-arquillian-extension that creates the jar from dedicated test classes from the asciidoctorj-core project (i.e. all Arquillian classes)
The dependencies on Arquillian are defined there again, so this is maybe not so nice.
I also changed the asciidoctorj-test-support plugin to use the nebula-test-jar plugin.
But maybe it would be better to publish the test-support module not under the tests classifier but without a classifier.
Because now all dependencies in the generated pom are also test dependencies.
And the dependency from asciidoctorj-arquillian-extension to asciidoctorj-test-support does not get the tests classifier so a consumer of the arquillian extension still has to define the dependencies like this:
dependencies {
    compile 'org.asciidoctor:asciidoctorj-arquillian-extension:1.5.3-SNAPSHOT'
    compile 'org.asciidoctor:asciidoctorj-test-support:1.5.3-SNAPSHOT:tests'
}

Apparently nebula does not like Java 6.
Will have to find a different way.
Maybe even simply remove asciidoctorj-test-support and move the classes to the arquillian-extension?

Maybe even simply remove asciidoctorj-test-support and move the classes to the arquillian-extension?

I'm fine with that.

Apparently nebula does not like Java 6.

I think it's okay if our build requires Java 7. We just need to maintain Java 6 support for running AsciidoctorJ for now (I don't think the world is ready for us to drop the runtime choice yet).

I am fine with that.
I was just surprised that the plugin does not run with Java 6 as I did not test with it locally.

Am 05.06.2015 um 23:32 schrieb Dan Allen<EMAIL_ADDRESS>Maybe even simply remove asciidoctorj-test-support and move the classes to the arquillian-extension?
I'm fine with that.
Apparently nebula does not like Java 6.
I think it's okay if our build requires Java 7. We just need to maintain Java 6 support for running AsciidoctorJ for now (I don't think the world is ready for us to drop the runtime choice yet).
—
Reply to this email directly or view it on GitHub.

Hi,
as you use the helper classes in the test-support package in the context of arquillian, it makes sense to move them to that package.
The test-support package was introduced to remove the duplication of the helper classes.
But I don't understand why you have to add the test-support dependency as well, if you are using the arquillian-extension dependency.
The test-support dependency should be transient in your current setup.
You can check that with gradlew :asciidoctorj-pdf:dependencies for example.

@mojavelinux If you have the time, could you please take another look at this PR? It got some great additions from @ancho!
The sources of the arquillian extension are now in the asciidoctorj-arquillian-extension module without a split into API and Impl.

@ancho Thanks!!!!
Rebased and merged locally and pushed to branch asciidoctorj-1.6.0

Thank you too @robertpanzer. You are doing a great job.

Agreed!


Improved the unchecked transform.

Improved the unchecked transform.

The ci build failed, please fix it

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.wudeyong seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Ability to name the Managed Identity in AAD

I've been creating initiative assignments in my environment that includes polices that use deployIfNotExists. So therefore a managed identity is created (Shows up in Azure AD as a Enterprise Application)
My issue is that I can't seem to control the name. This is a problem as when you review access control permissions across Azure (IAM) you see this GUID type named App that has "Contributor" rights. It should have a meaningful name otherwise it just triggers alarm bells as operators assume there is a rouge app that has been given permissions to our subscriptions. I've found I can't make any changes to the managed identity....

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

  Since this isn't an issue with docs, but a feature request, please make the request on UserVoice. The team evaluates the ideas there to include for future planning.  Thanks!
@MicrosoftDocs/azure-cxp-triage Please label 'product-question' and #please-close.


Initialization: Do not overwrite /etc/resolv.conf

Hi! Is there a particular reason to overwrite /etc/resolv.conf on image launch?
If run in a kubernetes environment, overwriting the default DNS configuration effectively prohibits name-based communications with other pods within the cluster, restricting (in our case) virtual schemas.
I suggest dropping this altogether, or at least preserving an existing configuration and appending any necessary values.

Hi nastye,
completely agree! This is a show stopper for docker-db in kubernetes.
We are trying to implement a DataOps automation with exasol on a kubernetes cluster and exasol needs to access different pods on the cluster to retrieve data to import (Kafka in this case). Without name resolution this will fail, because Kafka is dynamically providing TCP/IP names as brokers to access by the UDF within exasol.
These lines:
nameserver <IP_ADDRESS>
search dev.svc.cluster.local svc.cluster.local cluster.local
options ndots:5

have to be preserved for the kubernetes dns to work.
Please, could you change this bevahiour?

Hi! With 7.0 you can set up in the Global section of EXAConf the parameter ResolvFilename to something different, then /etc/resolv.conf. Then the autogenerated resolv.conf will be written to that file and the /etc/resolv.conf will not be touched.
Can you verify, whether that works for you?

Hi! I just came back to this and had some time to work with this.
My current approach is to both replace /etc/resolv.conf with the un-edited version from the pod im running my initialization service from and then also include the ResolvFilename setting in EXAConf.
While this seems possible, it doesn't feel like a very clean approach and I still do not understand why forcing Google DNS is necessary in the first place.
Could you elaborate on this @ooke


Update dependabot config

Track only submariner-operator as it includes admiral and submariner
Remove config for 2.5 as we're no longer releasing from it

/cc @sridhargaddam

/approve

/lgtm

/retest

/retest


[FLOC-3112] Support large state changes

Fixes https://clusterhq.atlassian.net/browse/FLOC-3112

This looks good. Merge after

the base branch is merged
the review comments are addressed
buildbot is green.

Reviewed 2 of 2 files at r4.
Review status: all files reviewed at latest revision, 2 unresolved discussions, some commit checks failed.

flocker/control/test/test_protocol.py, line 138 [r4] (raw file):
current will be out of date immediately. Annotate it with a date or a link or something.

flocker/control/test/test_protocol.py, line 621 [r4] (raw file):
:question: Should we assert that the value actually round-trips here and below?

Comments from the review on Reviewable.io

Numerous spurious failures due to problems unrelated to this change.  Merging.


[SPARK-15470] [SQL] Unify the Configuration Interface in SQLContext

What changes were proposed in this pull request?
We introduced RuntimeConfig in SQLContext in the PR https://github.com/apache/spark/pull/12669. Now, SQLContext has both conf and runtimeConf. SQLContext is being replaced by SparkSession. Like SparkSession, we should not have two configuration interfaces. That means, we should not expose conf to external users.
This PR contains three major parts:

removed conf from SQLContext.
added the missing functions into RuntimeConfig, including two set functions and one clear function.
fixed the test cases in SparkSessionBuilderSuite.scala. Without this fix, we are unable to individually run the test cases. All the test cases require initialSession.

@rxin @andrewor14 @yhuai @cloud-fan  Do you think this PR is valid? Thanks!
How was this patch tested?
Existing test cases cover it.

Test build #59090 has started for PR 13247 at commit f443064.

Test build #59090 has finished for PR 13247 at commit f443064.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59090/
Test FAILed.

Merged build finished. Test FAILed.

What's the difference between runtime conf and normal conf?

@cloud-fan Based on my understanding, runtime conf (class RuntimeConfig) is designed as the public/external interface for users to access the internal conf. If users want to make a change on Config at runtime, they must use RuntimeConfig. In the future, we can further enhance it to block external users to change the internal conf?  Also easier to manage Hadoop configuration in RuntimeConfig?
SQLConf will be just an internal implementation of configuration. We do not expect external users to directly access it.
You know, this is just my understanding. : )

I don't get it. ConfigEntry is internal to Spark. Why are we exposing that?

You are right. Let me add protected[sql] before def set[T](entry: ConfigEntry[T], value: T): Unit
https://github.com/gatorsmile/spark/blob/d972e8d8832c5a04d8ff4cc1ee10293cdbfac977/sql/core/src/main/scala/org/apache/spark/sql/RuntimeConfig.scala#L67-L69
That will be like what RuntimeConfig already has for get, which is protected[sql] def get[T](entry: ConfigEntry[T], default: T): T
https://github.com/gatorsmile/spark/blob/d972e8d8832c5a04d8ff4cc1ee10293cdbfac977/sql/core/src/main/scala/org/apache/spark/sql/RuntimeConfig.scala#L115-L117

I'd like to avoid contaminating RuntimeConfig with private[sql] methods if possible. Those visibility modifiers have no effect in Java.

In this case I guess it is ok though.

I see. Thanks!

Test build #59091 has started for PR 13247 at commit d972e8d.

Test build #59093 has started for PR 13247 at commit a8414f4.

Test build #59091 has finished for PR 13247 at commit d972e8d.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59091/
Test FAILed.

Merged build finished. Test FAILed.

Test build #59093 has finished for PR 13247 at commit a8414f4.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59093/
Test FAILed.

Test build #59108 has started for PR 13247 at commit 8a34781.

Test build #59108 has finished for PR 13247 at commit 8a34781.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59108/
Test PASSed.

Merged build finished. Test PASSed.

@rxin I am wondering if what this PR does is OK?

This looks ok, but can you remind me again what this pull request is actually solving? I feel it's just changing code for the sake of changing code here ...

Previously, @yhuai mentions we need to issue exceptions when users change the static configuration at runtime. See https://github.com/apache/spark/pull/13128#issuecomment-220411852
When trying to do it in a cleaner way, I plan to add these logics into RuntimeConfig. However, I found SQLContext has two entrances for configuration, one is conf and another is runtimeConf. Thus, I think we have to remove the duplicate before working on it.
Do you think my concern is OK?

Is the idea to have a list of configs that are marked as mutable and explicitly throw exceptions when users modify them?
I still don't see how that relates to the changes here... Why not just have that list in SQLConf?

Initially, I did try to do it in that way. The questions bothering me are how to know which changes are made at runtime? which changes are from external users?
After reading the code base, based on my understanding, we will not externalize SQLConf after introducing RumtimeConfig. The set APIs of SQLConf will be for internal usage only? Thus, we can do whatever we want, if necessary. For example, to verify the internal behaviors, our test suites are still allowed to change the configuration at runtime. For example, in multiple test cases, like  ddlsuite, we change spark.sql.warehouse.dir at runtime.
Then, I am wondering if we just need to block the changes on the static configuration properties through RuntimeConfig? That is the reason why this PR is to remove the conf from SQLContext.
Feel free to let me know which ways are preferred. Thanks!

Can we just add a flag to SQLConf to indicate whether SparkSession has been properly initialized?

OK, I can do it by adding a flag. Do you want me to close this PR?

Please close this. I just feel it adds virtually no benefit but requires non-trivial changes.

One thing that would be a lot more useful is to get rid of SQLContext in our test cases, and use only SparkSession. Then most of the problem in your pr just goes away.

Sure, let me close it. Thanks!


Where's WC v2 source code?

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
I can't find the code for v2 WC relay.
Requested Behavior
Have the source code for a v2 relay. Else the whole ecosystem would rely on a private proprietary piece of software. Providing his own relay would be difficult, hence concentrating all traffic into a private company service.
But maybe I just looked at the wrong place.
Anything else?
No response

Agreed with the points you state
https://github.com/WalletConnect/ts-relay is an implementation of the spec
Open sourcing a more recent implementation will follow shortly. You can check out my EDCON talk about the decentralization roadmap. We will also share more at ETHCC.


Update README

Removed
Requirements (nodejs)
coming soon
because it was listed as an issue and is not available.

Thank you @DeGarmoTech . Simple fix for sure but helps clarify any questions about what we support currently as opposed to eventually :)


Add jest

Add jest for unit testing the CLI
Add one simple spec file

Can I work on this?

sure! @kuldeepkeshwar

I will give pr in sometime


Bump express in /microservices/frontwebservice/angularUI/ilender-ui2

Bumps express from 4.17.1 to 4.18.2.

Release notes
Changelog
Commits

updated-dependencies:

dependency-name: express dependency-type: indirect ...

bla bla

blahhh

blallaaaa

vdfdsfsfad


⚠ ClamAV has degraded performance

In b3d87bc, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 194 ms

Resolved: ClamAV performance has improved in 8ef15c4.


Adding FCLuaScriptItem and FCLuaScriptItems to RGP Lua and JetStream Controller

I've been working on an idea that @Nick-Mazuk requested to be able to execute scripts from a different U.I. designed in Lua itself. To that we added the capability to add scripts on the fly. Today I suddenly realized this is going to address a JetStream Controller requirement as well.
Right now the program pops up an annoying dialog and enters a number which then executes a script. Well, with this new feature (coming in 0.64), your script will be able to fish its own item out of the list, modify the prefix on the fly, then re-execute itself with a different prefix. The details are a little more complicated than that, but I think it will work.
Sound interesting? If so, I'd be interested to know a little more detail on the design of the JetStream Controller now. The sequence is

KM or Applescript executes the dialog box and fills in the code specifying which item to run
Is that then a require statement within the script? (I suppose I can look at the source code now. Lol.)

@CJGarciaMusic @jwink75

This issue will probably answer your question of how the JetStream currently works in case you don't want to dig through all the source code: https://github.com/CJGarciaMusic/PowerUserWorkflow/issues/11

I have uploaded tentative documentation to the master branch of the PDK Framework Docs Repo. To review it:

Clone the repo to your local drive by downloading the zip file (easiest).
Open the first html file with your browser. (I believe it is called annotated.html.)
Navigate to FCLuaScriptItem.

Feedback is much appreciated before I get too much further into it.

On further reflection, I realize this does not address the fundamental challenge for JetStream Controller: that of passing a trigger value from the controller to the Lua script. But maybe this will be useful in a future refactoring.

Ok, just to make sure I understand the docs, if I were to want to run a script I could do something like this?
local script_to_execute = "My script's name"
local script_items = finenv.CreateLuaScriptItems()
for item in each(script_items) do
    if item.GetMenuItemText() == script_to_execute then
        finenv.ExecuteLuaScriptItem(item.GetExecutableIndex())
        break
    end
end

This seems like a good API to me. Couple questions/thoughts, though:

It looks like you can add a script using finenv.AddLuaScriptItem? I assume this would not add scripts to the menu bar and any added scripts would not persist between Finale restarts, but just want to double-check.
Likewise, I assume methods like SetMenuItemText would not update the menu bar and the changes would not persist to the next session. I also assume it wouldn't update the source file.
GetPrefix: what does this do? I can't think of what a script prefix is.
AddLuaScriptItem seems like a security issue. Usually a user needs to download the script, and then they've implicitly consented to running that script. However, I could see someone creating a malicious script that adds a new script then immediately executes it. This other Lua script doesn't even necessarily need to be a script the user downloads. Now, would I think this ever would happen? No. Would it be easier to just write the malicious Lua code directly in your script? Yes. Would it be even easier to just execute an arbitrary terminal command (which is already possible)? Yep. But still, I can't think of a reason why someone would want to add a new script that's compelling enough to add another attack vector. With that said, I'm not against this feature since in reality it doesn't let a malicious user do anything new, but I think it's something to think about.
SetFilePath this seems like a security issue. Let's say we have three scripts, A: "a.lua", B: "b.lua", and C: "c.lua". "c.lua" as some malicious code and may not even be a Finale Lua script. A uses SetFilePath to change the file of B to "c.lua". User tries to run "b.lua" by clicking on "B" and accidentally runs "c.lua". Whoops! They've now run malicious code. There are the same caveats as above, though.

You can't change the menu bar, so no.
See 1. SetMenuItemText doesn't have much use to Lua scripts.
GetPrefix allows you to supply a Lua snippet that runs before the script does.
AddLuaScriptItem is how you are going to execute a script on the fly. I don't see this as fundamentally more dangerous than a script that (as you say) does the malicious code without it.
SetFilePath cannot change already-configured script items. Its primary use is for creating a new item to execute on the fly. CreateLuaScriptItems makes a copy of the already-existing items and it does not allow you to copy them back. Only add new ones.

I see. I think that eases my concerns. LGTM.

It used to be a UserValueInput box, but now it’s just a small CustomLuaWindow with an entry box.
There is now code in there that parses multiple commands, and also code that accepts multiple different input values beyond the 4-digit codes.
This lets you put in, say “p < -f” and it will execute functions to add a starting p dynamic, add in a hairpin, and add a f dynamic at the end. I’m actually going to modify this further so that it will accept “p < f” and turn the second dynamic into an “end” dynamic.
I actually LOVE the little dialog box and don’t want to lose it, but if there is a way to pass the plug-in arguments without actually opening it that really would be slick.

On Jul 30, 2022, at 3:35 PM, Nick Mazuk @.***> wrote:
Ok, just to make sure I understand the docs, if I were to want to run a script I could do something like this?
local script_to_execute = "My script's name"
local script_items = finenv.CreateLuaScriptItems()
for item in each(script_items) do
if item.GetMenuItemText() == script_to_execute then
finenv.ExecuteLuaScriptItem(item.GetExecutableIndex())
break
end
end
This seems like a good API to me. Couple questions/thoughts, though:
It looks like you can add a script using finenv.AddLuaScriptItem? I assume this would not add scripts to the menu bar and any added scripts would not persist between Finale restarts, but just want to double-check.
Likewise, I assume methods like SetMenuItemText would not update the menu bar and the changes would not persist to the next session. I also assume it wouldn't update the source file.
GetPrefix: what does this do? I can't think of what a script prefix is.
AddLuaScriptItem seems like a security issue. Usually a user needs to download the script, and then they've implicitly consented to running that script. However, I could see someone creating a malicious script that adds a new script then immediately executes it. This other Lua script doesn't even necessarily need to be a script the user downloads. Now, would I think this ever would happen? No. Would it be easier to just write the malicious Lua code directly in your script? Yes. Would it be even easier to just execute an arbitrary terminal command (which is already possible)? Yep. But still, I can't think of a reason why someone would want to add a new script that's compelling enough to add another attack vector. With that said, I'm not against this feature since in reality it doesn't let a malicious user do anything new, but I think it's something to think about.
SetFilePath this seems like a security issue. Let's say we have three scripts, A: "a.lua", B: "b.lua", and C: "c.lua". "c.lua" as some malicious code and may not even be a Finale Lua script. A uses SetFilePath to change the file of B to "c.lua". User tries to run "b.lua" by clicking on "B" and accidentally runs "c.lua". Whoops! They've now run malicious code. There are the same caveats as above, though.
—
Reply to this email directly, view it on GitHub https://github.com/finale-lua/lua-scripts/issues/287#issuecomment-1200305584, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMCBUQ22JJZ6FCRPR4DPV5TVWWUZTANCNFSM55DZFJMQ.
You are receiving this because you were mentioned.

Actually, based on your feedback (@Nick-Mazuk ), I realized the Add and Delete APIs are probably unnecessary. All we really need is Create (collection) and Execute. The Execute method can execute one you create from scratch, which allows for on-the-fly additions.
FWIW: I can't see any increased security risk from Execute over what is already there with require. The Execute API just allows for the script to get its own Lua state.

With excellent feedback from this thread, I think I have zeroed in on a workable set of APIs that preserves a modicum of security. There will basically be two flavors of FCLuaScriptItem depending on how you construct them. (You will have to construct them with one of the two finenv functions. I've removed the Lua-accessible constructors.)

Items from Finale's plugin menu.
Adhoc items.

Items from Finale's plugin menu will be launch-and-forget and behave exactly as if you launched them from the menu. As a security precaution, your launcher script cannot see the file paths for them. (It occurred to me that a malicious script could rewrite third-party scripts if it knew where they were. This allows you to launch them without knowing anything about them other than their menu and other descriptive text.) Furthermore, the only thing that matters for these scripts is the (read-only) "execution index". No changes you make to the script item have any effect on the launching of these scripts. (Specifically, changing the prefix has no effect.)
Adhoc items will allow you to launch any file of your choosing. If it isn't proper Lua, of course, you'll just get an error. The big difference with these is your script has to stay alive as long as the launched script stays alive. For scripts with modeless dialogs and retained lua states, this is longer than just a single script invocation. Also, the properties on FCLuaScriptMenu do matter for these. Especially, the prefix, undo text, and booleans.
I've updated the docs at the doc repo.
If you would like to try out a prerelease version of it, you can download it here. No guarantees on how long the link stays up. But a few days at least. The Apple version is not notarized, so the usual rigamarole will be required to bless it.

P.S. I've also hooked up FCCtrlComboBox in the same version in the link above.

P.P.S. I removed the constructor on FCLuaScriptItems as well, but I now realize that was a mistake. It'll be restored in the next version.

Awesome, thank you Robert!

I've been thinking that a script-launching script might like to have access to some of the other finaleplugin fields. The tradeoff is between security (and not revealing too much about a script to 3rd parties.) And usability of the launcher. The ones that occur to me are
Notes
Categories
ScriptGroupName
ScriptGroupDescription
required min max versions
Any others?

It would also be nice to also get the script's text contents, or at least the hash (SHA256?) of the contents. And the filename and path of the script (which I know I already argued against earlier).
I've been trying to figure out a good way of making it easy for users to update their scripts. After thinking about it, I put together a proposal in #322, but it would need these things. I'm open to another way of creating an updater. I just haven't figured out another way, yet.

I am fairly concerned about revealing a file path on the user's computer to a third party. A malicious third party could replace its contents, and if it were done carefully deflect the blame back to the  script it modified.
Is there some way we could do this online? One of the finaleplugin attributes that Jari planned to enforce was a unique Id, unique to his repository. (Search for finaleplugin.Idon this page.) If we implemented that for every script, then you could find it at our site that way without knowing anything about the user's local disc. FCLuaScriptItem could provide a hash of the contents for comparison.

I've done some preliminary investigation into adding hashing to RGP Lua, and it is not completely straightforward or necessarily lightweight. Here is my proposal:

Add an Id requirement for insertion in this repo. A human readable Id with a reverse url is fine with me. Alternatively we could use the filename itself, but an Id seems like the more reliable method. (And it is already a documented property for JW Lua scripts.)
When the deployment program creates the dist version of the script, it inserts a hash into the plugindef() function. I would propose finaleplugin.Hash = "<string-value>".
RGP Lua can report this hash value back in the FCLuaScriptItem class.

The advantages of this approach are:

Minimal technical information supplied to 3rd party scripts.
The hash is only computed once.
The hash computation is guaranteed to be the accurate, because it only happens once.

Disadvantages are:

A version mismatch will not be noticed if the user edits the downloaded script.
(Others I haven't thought of, without doubt.)

I like the idea of inserting the hash into the plugindef at bundle time. Definitely less error prone.

I am fairly concerned about revealing a file path on the user's computer to a third party. A malicious third party could replace its contents, and if it were done carefully deflect the blame back to the script it modified.

+1 to this.
My follow-up question would be then how to actually update the script. My proposal in #322 would require the script to get the location of the outdated script in order to automatically update it.
A stop-gap would be to just direct users to the website to download it themselves. While this would work, in my opinion updating scripts should be as painless and simple as possible for end users. I want to try and keep as many people as possible on the latest versions of everything.
An alternative is we could add a new method to RGP Lua that allows other scripts to update a script file. Script A will call this API to update script B and will tell RGP Lua B's new contents. RGP Lua will then ask the user whether script A should be allowed to update script B. If the user accepts, RGP Lua will overwrite script B's code with the new contents and return true to script A. If the user declines, RGP Lua will not update script B and will instead return false to script A.
Lastly, since a user will likely run the updater many times, and each time many scripts will be updated, it would be nice for allow the user to "trust" the updater script. And in the future, RGP Lua will not prompt for the user's consent in the future.
All this would have to be done inside the RGP Lua plugin. Thoughts?

On another note, @ThistleSifter suggested in #145 of letting a script report a success/error case. I think that's a decent proposal. Perhaps the finenv.ExecuteLuaScriptItem function can return whatever the script itself returns?
Also clarification, does finenv.CreateLuaScriptItems() return individual scripts or menu items? I think the latter is preferred (and it shouldn't impact the updater at all).

Answers in no particular order:

finenv.CreateLuaScriptItems returns a collection of menu items, and the latest version of finenv.CreateLuaScriptItemsFromFilePath returns a collection of all the menu items in the script. (The version I uploaded does not do this yet.)
Regarding the return value from ExecuteScriptItem: returning multiple values from C++ is challenging. The easiest approach would be to return a string if there is an error and null if it executed successfully. Right now, RGP Lua is displaying that string the same as if the script had been executed from the menu. Should it continue to so do? Should I try to figure out a way to return the Lua status as well? (Understanding that sometimes there is no Lua status, such as when you try to execute a script that has version restrictions.)
One of the challenges that both RGP Lua and a script will have in updating scripts is that they may not be in a directory that a program can write to without admin privilege. I'm thinking the update should be external to both. The most reliable solution would be an external installer that downloads and installs the update. There is no automatic way for RGP Lua to run under admin privilege, because it is running as part of Finale.
Adding some kind of privileged status to scripts to allow them to take actions that otherwise would cause user notifications is easy enough. But it is not very secure. A malicious script-author could figure out how to defeat it with a little digging. So I'd prefer not to rely on that.
Jan Angermüller has done considerable work with trying to download files from URLs in Lua, and he says it is fraught with difficulty due to OS restrictions around shelling out to do it. I'm probably going to add a function to get a string from a URL directly. I already added this to Patterson Plugins, and it's not much code. You get back whatever the http response is, so if the file doesn't exist you'll likely get the 404 message. I would be interested in brainstorming any security considerations around this.

finenv.CreateLuaScriptItems returns a collection of menu items, and the latest version of finenv.CreateLuaScriptItemsFromFilePath returns a collection of all the menu items in the script. (The version of RGP Lua I uploaded does not do this yet.)

I see, thanks for the clarification!

Regarding the return value from ExecuteScriptItem: returning multiple values from C++ is challenging. The easiest approach would be to return a string if there is an error and null if it executed successfully. Right now, RGP Lua is displaying that string the same as if the script had been executed from the menu. Should it continue to so do? Should I try to figure out a way to return the Lua status as well? (Understanding that sometimes there is no Lua status, such as when you try to execute a script that has version restrictions.)

I think simply returning nil on success and a string on error is sufficient. Though I'm curious on @ThistleSifter's thoughts.
However, the next question is what that string should be. For instance, if a script puts return "hello" at the end of the file unconditionally, would that always be counted as an error?
One alternative is status codes and a message. One popular implementation of this pattern is absl::Status (https://abseil.io/docs/cpp/guides/status). So potentially the return value can always be a string that represents the status code. Even if the script doesn't run because of version restrictions, it can still return the "Unavailable" status code.
Of course, we may still wish to have a custom message, so perhaps a tuple of strings?
-- structure
return "status code", "message"

-- default if the return statement isn't included
return "ok", ""

Then from anywhere in the script you can return "ok" or an error.
We can easily abstract the details of this into a library function to ensure consistency across this repo.
-- status creation
status.ok() -- "ok", ""
status.ok("my message") -- "ok", "my message"

-- status parsing option 1
local status_code, message = invoke_script()
local is_ok = status.is_ok(status_code)

-- status parsing option 2 (which may be more forward compatible should we choose to change the format in the future)
local result = invoke_script()
local is_ok = status.is_ok(result)
local message = status.get_message(result)

One of the challenges that both RGP Lua and a script will have in updating scripts is that they may not be in a directory that a program can write to without admin privilege. I'm thinking the update should be external to both. The most reliable solution would be an external installer that downloads and installs the update. There is no automatic way for RGP Lua to run under admin privilege, because it is running as part of Finale.

Hmm… isn't this already an issue with the configuration library? How does the configuration library currently work if the script is in a protected directory?
While I'm not against for an external installer, I'd prefer keeping it in Lua if possible. That would just be much simpler to maintain since we already have a pipeline for that. A fallback is that if RGP Lua or the script cannot write the file, we can just prompt the user to download the script from the website manually.

Adding some kind of privileged status to scripts to allow them to take actions that otherwise would cause user notifications is easy enough. But it is not very secure. A malicious script-author could figure out how to defeat it with a little digging. So I'd prefer not to rely on that.

Ah true. Since the best place to store this is likely in the XML file, which is in a consistent place for every RGP Lua user.

Jan Angermüller has done considerable work with trying to download files from URLs in Lua, and he says it is fraught with difficulty due to OS restrictions around shelling out to do it. I'm probably going to add a function to get a string from a URL directly. I already added this to Patterson Plugins, and it's not much code. You get back whatever the http response is, so if the file doesn't exist you'll likely get the 404 message. I would be interested in brainstorming any security considerations around this.

That would be a useful function.
The main security issue I can think of is that now the host of the external URL is an attack vector. If we point to URLs we control (for instance, the website), it's likely not a huge deal. However, if someone were to ever take control of the external URL, they can potentially get arbitrary code to run on a user's computer. The two main ways I can think of are the following:

loadstring to a Lua string as code: https://stackoverflow.com/questions/44386060/execute-lua-string-as-lua-code
Saving the contents of the string in a Lua script.

However, I'm not particularly worried about these.
loadstring itself is already a fairly large security risk (because if you own the string, why not just write the original Lua?). We can easily mitigate that by not using the loadstring function in our Lua scripts. And should anyone else use that function, I don't see an increased security risk since they are likely using that function with a string they don't control already.
For saving the contents of the string to the Lua script, that's the entire point of the operator. And the user needs to consent for this to happen. If we're worried about our website becoming compromised, the source code is open-source. So should anyone add malicious code, it'll be easy for us to replace it (or to move the site to a new domain). Also, all my accounts are secured with 2FA and the DNS has DNSSEC turned on, so it should be fairly difficult for an attacker to spoof the site. If someone does do something malicious with the site, I think that means we likely have larger problems since right now, there's practically no incentive to.
If the script points to a third-party site, I think one could make the case that the user assumed that risk at the time of downloading the script.
I think the largest consideration is that any HTTP call should be secured with HTTPS (this also goes for the images in #327). HTTP is very, very insecure.
If we were really concerned about the URL host being compromised, we could also have this be another popup generated by RGP Lua, saying "the script wants to access this URL", and potentially letting the script give some sort of reasoning like ("so we can check to see which scripts are out of date"). However, one could make the case that we'd need to do this for images as well, which could get very unwieldy very fast.

I don't see any way in the APIs to restrict downloads to HTTPS. I could force all URL strings to start with https however.
Jan Angermüller updates his code with an external script. This strikes me as a much more reliable update process than building it into RGP Lua. Writing the configs is not a permission problem because they are specifically being written to a location designated for writing 3rd party configuration parameters. But a lot of people are putting their Lua scripts in the Finale plugin directory, and this is definitely a protected location on both Mac and Win.
I'm thinking Execute will return the status code, and I'll think of a way to return the message as well, if any. I'm more interested in whether RGP Lua should display the message as normal.

Just following up, I figured out how to get Windows to refuse http links, and macOS was already doing it. (It may be possible to enable them somewhere in user settings, but disabled is the default.)
I have also figured out how to return multiple values to Lua from C++, so ExecuteLuaScriptItem will return a status and the message, if there is one.
Because the read-from-url function is a synchronous call, it is not going to be appropriate to download more than a few 10s of kilobytes with it.

I'm beginning work on finishing this up. The finenv.ExecuteLuaScriptItem will return a boolean success as its first return value and a string or nil as the second. One question remaining is, should ExecuteLuaScriptItem display the error string as RGP Lua does when executing directly from the menu? Right now it does, but it could be suppressed. Then it would incumbent on the calling script to display the message to the user.


Update gem command

To avoid gemsplit breaking on the deprecation warning:
"WARN: Please note that rvm gem ... is only an alias to rvm do gem ...,
it might work different as in earlier versions of RVM and will be shortly removed!"

Thanks!


How many feature number do you used?

As you said, 'I use half num of features (128) to make training faster for those results below.' I am a little confuse. How many of your feature number of result image shown at the page. 64, 128 or 96. Thanks.

Sorry, I wrote that explanation almost a year ago. So now it's little bit changed.
Usually I use 96 features. You can use 32, 64, 128, 256 or even more, however, in my experiments I can't getter better results with using more than 128 features. It depends on other hyper parameters. You can try by yourself. :)

I mean, I recommend you to use 64 or 96 for the first. So the default is 96 in my code.
However, the table of PSNR results in the readme was calculated by 128 features.

Your speed of reply is as fast as light. Thanks.


[WIP]Restrict: 'google-api-client', '~> 0.11'.

https://github.com/google/google-api-ruby-client/commit/2952e9ce27ca582771094cd38b852058d4344c3e

Fixed via https://github.com/sonots/gce-host/pull/6

thx!


Remove outdated information in the README

There is a line in the readme which says:
There is a settings page in the ui where you can temporarily change the REST API URL, alternatively you will need to edit the settings.js file to set the URL permanently.
This is now out of date and should be removed

Merged into develop


It does not work on Pi Zero W

The ISO used on this project doesn't even boot on Pi Zero W. Since the ISO is from 2016, and the Pi Zero W is from 2017, probably there is no support for it in this ISO.
Also, tried with an updated ISO, but when you install the .sh file it just doesn't boot anymore.
A shame, this project is quite cool but I can't make it work =/

same issue. the original iso does boot for me, but it doesn't allow me to plug in a usb hub which i need in order to plug in both an ethernet cable and a keyboard since my pi's wifi doesn't work

@fairbanksbr & @probablypablito if you know of other functioning forks or alternative rubberducky projects for PRI-0, please share.

no :(
i ended up being a pack of 2 pi picos on Amazon for $8


[FEATURE]: Change workflow branch from master to main

[FEATURE]: Change workflow branch from master to main
Description

Describe your PR

Proposed changes:

Update workflow and browserlist
Bump html-webpack-plugin

Related issue

if applicable

Fixes #
Notify
Repository owner: @marcoandre1
Thank you! 🙌

Close #17


Db folder plugin: global search and filters hidden

The global search and filters row of a db folder table is hidden. It seems that it is below the top header.
This is how a table appears with the standard theme:

The same with the Prism theme:

Hi @giancarlocella,
This is a related issue to #100 and will be addressed in a coming update. Thanks for reporting!

Hi @giancarlocella,
This issue should now be addressed in version 3.1.0.


-cuda Flag for Rocm

I have a cuda file and compiled it with
nvcc -cuda --output-file kernel.cpp kernel.cu which gives me cpp file which generates the code which can be compiled by host compiler
I would like to know is there any similar flag in hipcc which can generate similar output as in nvidia

Kernel.cu & kernel.cpp file i am attaching.
`
kernel_cu & kernel_cpp.zip
I am then compiling kernel.cpp with mpicxx to generate the kernel.o file. Can we generate a similar kernel.cpp file with hipcc/hcc ?

@kayog96 Closing ticket. Please re-open if you still need assistance with this ticket. Thanks!


Viewing existing workbook page from from dashboard returns error

There is an error on in workbooks/view.py on line 150.

addressed in https://github.com/BlinkUX/ISB-CGC/commit/69dd46b970b72cf8ca3a54c57798919da477fc3e


Enhance Support for Hardhat Network Forking

Problem
Developers utilizing Hardhat(or Foundry) for Ethereum development often rely on the forking feature to test their contracts and applications against live networks. Given that HIP-584 is final there should be no major blockers to network forking. However, when attempting to use this feature against the Hedera testnet relay, I've encountered compatibility issues. Running a fork test that deploys a contract and interacts with that contract and other contracts already live on testnet results in at least the following error:

Errors: Invalid value "0x095" supplied to : RpcBlockWithTransactions | null/transactions: RpcTransaction Array/0: RpcTransaction/gasPrice: QUANTITY

Solution
The Hedera team should conduct a thorough investigation to identify and resolve any incompatibilities or bugs that cause Hardhat fork tests to fail. This investigation should aim to ensure full support for Hardhat&Foundry network forking.
Alternatives
No response

Hi @mshakeg ,
For fork tests require batch requests to be enabled on the relay. Unfortunatelly that is not the case with the public relays, but you can run a local instance and configure it to work with testnet or mainnet. This env variable enables batch requests:
BATCH_REQUESTS_ENABLED=true

I did some research and found that fork tests work for hardhat up to version 2.20.1, but not for the latest versions. With Foundry it seems to be working fine. You can see an example in this repo

Hi @Ivo-Yankov why are batch requests not enabled on the hashio relay? It makes for a terrible dev experience, all the other EVM chains that I've interacted with have public relays that fully support&enable all the functionality required for fork tests.

Hello @Ivo-Yankov and @mshakeg ,
I can confirm that BATCH_REQUESTS_ENABLED is enabled on all production environments of HashIO.
That said, I don't think we support Forking of the blockchain at a given snapshot just yet. we are working on that feature and will update once we confirm that we have the capabilities.
Thanks

Hi @AlfredoG87 thanks for clarifying. Yes it doesn't seem like forking at a given snapshot(block height) is supported at least not fully as my fork tests fail with errors such as the one in the issue description.

This example is highly-specific to hardhat, but I think might shed some light on how this could be implemented in Hedera:
https://stackoverflow.com/a/78191980/194982
Notably:

The file hardhat/internal/hardhat-network/jsonrpc/client.js in this example performs a similar function to Hedera RPC Relay, in the sense that it proxies/ translates RPC requests.
In the case of hardhat, it's between the client and any RPC endpoint configured (including a faux EVM implementation, in memory). In the case of Hedera RPC Relay, it is between the client and a combination of Hedera Mirror Nodes and Hedera Consensus Nodes.
In that context, we could re-use some ideas from this patch to hardhat within Hedera RPC Relay.

Specifically:

See the _patchRawResult function that has been implemented.

It identifies specific differences between Ethereum's EVM implementations (e.g. geth) RPC responses, and the version the the rootstock node
Subsequently it applies some simple substitution and formatting differences to the RPC response before relaying the response to the client

In Hedera RPC Relay, we could implement an equivalent of the _patchRawResult function.

Within that, we could implement some formatting/ substitution for the already known/ identified differences (e.g. stateRoot)
We could also apply fuzz testing (or other suitable untrained discovery methodologies) to identify differences between responses between Hedera's RPC Relay and RPC endpoints from other nodes; and subsequently implement some formatting/ substitution for these too.

Refs:

Prior discussions:

hardcoded zero value for stateRoot in RPC responses for eth_getBlockBy* RPC requests
https://swirldslabs.slack.com/archives/C03EHC2H8CR/p1706056921483199

https://github.com/hashgraph/hedera-docs/issues/562

On fuzz testing:

https://github.com/hashgraph/hedera-json-rpc-relay/issues/2066
https://blocksec.com/blog/security-check-do-evm-compatible-chains-hold-up
https://blocksecteam.medium.com/systematic-approach-to-maintaining-evm-compatibility-and-security-32d0f42846f5
Patch for hardhat to enable Hardhat network forking in Rootstock: https://stackoverflow.com/a/78191980/194982

cc @Nana-EC @Neurone @a-ridley @theekrystallee

Hi @mshakeg, can you please confirm if your tests are working with<EMAIL_ADDRESS>with and without specifying the block number?

Hi @AlfredoG87 it seems like BATCH_REQUESTS_ENABLED has been set to false on the hashio mainnet relay? I'm getting the following response now:
Errors: Invalid value null supplied to : RpcBlockWithTransactions | null/transactions: RpcTransaction Array/0: RpcTransaction/gasPrice: QUANTITY, Invalid value null supplied to : RpcBlockWithTransactions | null/transactions: RpcTransaction Array/0: RpcTransaction/v: QUANTITY, Invalid value null supplied to : RpcBlockWithTransactions | null/transactions: RpcTransaction Array/0: RpcTransaction/r: QUANTITY, Invalid value null supplied to : RpcBlockWithTransactions | null/transactions: RpcTransaction Array/0: RpcTransaction/s: QUANTITY, Invalid value null supplied to : RpcBlockWithTransactions | null/transactions: RpcTransaction Array/0: RpcTransaction/type: QUANTITY | undefined

Hi @AlfredoG87 it seems like BATCH_REQUESTS_ENABLED has been set to false on the hashio mainnet relay? I'm getting the following response now:

@mshakeg I've just verified and BATCH_REQUESTS_ENABLED is seems to be enabled on mainnet
Could you share your request so I can reproduce it on my end?
I just made a batch request using this request and the response is as expected and working:
curl --location 'https://mainnet.hashio.io/api' \
--header 'Content-Type: application/json' \
--data '[
    {
        "jsonrpc": "2.0",
        "id": 1,
        "method": "eth_chainId",
        "params": []
    },
    {
        "jsonrpc": "2.0",
        "id": 2,
        "method": "eth_chainId",
        "params": []
    },
    {
        "id": 2,
        "jsonrpc": "2.0",
        "method": "eth_blockNumber"
    }
]'

Response:
[
    {
        "result": "0x127",
        "jsonrpc": "2.0",
        "id": 1
    },
    {
        "result": "0x127",
        "jsonrpc": "2.0",
        "id": 2
    },
    {
        "result": "0x3bb1f1c",
        "jsonrpc": "2.0",
        "id": 2
    }
]```

@AlfredoG87 it fails on the following line in a hardhat script. Not sure what all the json rpc requests involved in this are, but it worked fine in a fork test about 2 weeks ago.
import { ethers } from "hardhat";

// other code ...

const signers = await ethers.getSigners(); // fails on this line

@AlfredoG87 it fails on the following line in a hardhat script. Not sure what all the json rpc requests involved in this are, but it worked fine in a fork test about 2 weeks ago.
import { ethers } from "hardhat";

// other code ...

const signers = await ethers.getSigners(); // fails on this line

I am not sure what is the cause of the error but seems to me unrelated to BATCH_REQUESTS_ENABLED or even the relay. seems to me like a configuration issue is the culprit on your hardhat project.

@AlfredoG87 sure, below is a minimal reproduction of the issue, it seems like it could be a pruning issue, as forking from a block a few days ago fails but from a recent block today doesn't, so maybe the mirror node needs to keep a lot more history(at least 1 month ago)?
https://github.com/mshakeg/hardhat-template/pull/2

Hi @AlfredoG87 I just wanted to follow up on my last reply. Have you made any progress?

Hi @Ivo-Yankov just wanted to follow up on this reply. Do you know if the public mirror node prunes contract state and if so how far back does it go?

Hi @mshakeg, thanks for putting up together the repro mentioned in https://github.com/hashgraph/hedera-json-rpc-relay/issues/2161#issuecomment-2058726973. I've ran it and I've got
$ npx hardhat test
{ hasPKs: true }

  Unit tests
    1) "before all" hook in "Unit tests"

  0 passing (594ms)
  1 failing

  1) Unit tests
       "before all" hook in "Unit tests":
     Error: Missing access list

An unexpected error occurred:

[Error: Missing access list] { code: 'GenericFailure' }

are you getting the same error?
This is a known issue that we are planning to solve by publishing a Hardhat plugin.

Hey @acuarica haven't tested on my repro, but on another project that does a contract call in a fork that just reverts, however the same call directly against mainnet works.

Transaction reverted without a reason string

Does the other project use Hedera-specific services, say HTS?
If that's the case, we are building a soon-to-be-released plugin to solve this problem.

Does the other project use Hedera-specific services, say HTS?

@acuarica yes it does