Remove automatic keys

My ideal scenario, an alternate approach to #67 and #73. May not be to everyone's taste.
Also, the scenario where you need to add a key even with just one child can probably be avoided some other way. I will have a think about that one. I just wanted to show that it's pretty easy to just add keys where React tells you to.
Should yield more efficient applications too, right now I would imagine many people using this are seeing far higher CPU usage because they don't understand the key system. Which is technically undocumented!

I'm not saying "THIS IS WHAT IS HAPPENING". This is just my proposal, I wanted to make something to compare to the original PR.

Like I said in #73, I don't think it's a good idea to do this here. We're fixing the problem, but requiring users of this lib to react to the now missing keys.
We already know what the keys are going to be, since we're dealing with a mutable API. So this is imho just half way to what #73 does.
@FunkMonkey

Well, it would be a major version bump and it's good that it requires action. Those that previously got away with bad keys (and they are bad, they have no meaning and are not tied to the element's identity) would be forced to fix them, improving performance and debugging (maybe).
I just never liked the fact that I was doing something behind the users back, sneakily filling in a gap you're meant to do yourself. I'm not sure what you mean by "we already know what the keys are going to be" though.
Is it really that bad to add keys to multiple children, just like you'd have to in regular React anyway? I think not. I think it keeps the faux DOM inline with how React actually works instead of diverging from it.

Added the optimisation such that:

{[child]}

Is now:

{child}

Which does not require a key. Cleaned up that recursion call in toReact at the same time. I do worry about recursion depth there.

@Olical I do understand your intent and I admit that it is easy to add keys for code that you write yourself. But what about code that you don't write yourself? Even the simplest things like d3.svg.axis will give you React key warnings, as they produce code like this:
<g transform="translate(0,420)" class="x axis">
  <g transform="translate(0,0)" style="opacity: 1;" class="tick">
    <line x2="0" y2="-420"></line>
    <text x="0" y="3" style="text-anchor: middle;" dy=".71em">2000</text>
  </g>
  <g transform="translate(86.76724137931035,0)" style="opacity: 1;" class="tick">
    <line x2="0" y2="-420"></line>
    <text x="0" y="3" style="text-anchor: middle;" dy=".71em">2001</text>
  </g>
  <g transform="translate(173.29741379310346,0)" style="opacity: 1;" class="tick">
    <line x2="0" y2="-420"></line>
    <text x="0" y="3" style="text-anchor: middle;" dy=".71em">2002</text>
  </g>
  <path d="M0,-420V0H880V-420" class="domain"></path>
</g>

How would we deal with that?

@FunkMonkey Agreed. It's not the same to ask the user for "filling in the gaps" in faux-dom, like React can.

we already know what the keys are going to be

@Olical I mean, that faux-dom Elements are mutable. We are keeping them around and mutating them, eventually calling toReact for every render. Since these elements are not recreated inside the render method, they keep the same instance. Thus we can automatically prevent the diffing problem, that React runs into. keys in React are there for solving this exact problem: To give the diff-ing algorithm a hint of which new ReactElement is replacing which old one, so that the changes can be applied accordingly. We know which Element maps to which ReactElement, even with the changes. Thus we can keep around an ID for the FauxElement using which we can indicate the ReactElements to which DOMElement they belong.

Ah, there's an interesting difference in our use case then. I'm essentially
allergic to state, so I try to minimise it wherever possible (which is why
I wrote this! :D). So I create a new DOM on every render whenever I use
this. See:
https://github.com/QubitProducts/d3-react-sparkline/blob/master/src/d3-react-sparkline.js
I wouldn't recommend keeping the element around outside of the render call,
if you want to do that you may as well start stepping out of React and
letting D3 have full control of that element anyway. The point of the faux
DOM was that you could use it as part of a pure function.
On 21 September 2016 at 14:10, Phil Pluckthun<EMAIL_ADDRESS>wrote:

@FunkMonkey https://github.com/FunkMonkey Agreed. It's not the same to
ask the user for "filling in the gaps" in faux-dom, like React can.
we already know what the keys are going to be
@Olical https://github.com/Olical I mean, that faux-dom Elements are
mutable. We are keeping them around and mutating them, eventually calling
toReact for every render. Since these elements are not recreated inside
the render method, they keep the same instance. Thus we can automatically
prevent the diffing problem, that React runs into. keys in React are
there for solving this exact problem: To give the diff-ing algorithm a hint
of which new ReactElement is replacing which old one, so that the
changes can be applied accordingly. We know which Element maps to which
ReactElement, even with the changes. Thus we can keep around an ID for the
FauxElement using which we can indicate the ReactElements to which
DOMElement they belong.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Olical/react-faux-dom/pull/74#issuecomment-248607359,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AATPXarVtKoqJ-XBHyxBfc3uafzJ0mG_ks5qsSy-gaJpZM4KBhOt
.

@Olical What about your animate-d3-with-mixin example? It will throw key warnings due to d3.svg.axis
It is not really about keeping the the faux elements around or recreating them each time in render, but more about being able to use 3rd party code with d3.svg.axis being a very basic example that will fail.
I personally like react-faux-dom because I can mix creating SVG elements using React with extending these elements using d3, f. ex.:
function render() {
  const fauxXAxis= Faux.createElement( 'g' );
  d3.select( fauxXAxis )
    .call( d3.svg.axis() );

  const fauxYAxis= Faux.createElement( 'g' );
  d3.select( fauxYAxis )
    .call( d3.svg.axis() );

  const circles = [ 100, 200 ].map( ( val, i ) => <circle key={i} r="10" cx={val} cy={val} />
  const circleGroup = <g key="circles">{circles}</g>

  return (
    <svg height="300" width="300">
      { [ fauxXaxis.toReact( 'x' ), fauxXaxis.toReact( 'y' ), circleGroup ] }
    </svg>
  );
}

This simplified example is the main reason I use react-faux-dom and I really like that it makes things like this possible ;)

@Olical My PR also doesn't abolish creating faux-dom elements on the fly. It "falls back" to the element order in that case.
It also doesn't prevent the user from using attr('key', x)

Ah, I see what you mean with those deep structures now. I'll go over your PR again then, there's a couple of changes I made here that I would like to see in yours, only small things. Maybe I'll merge yours and just do it myself, namely the function I refactored out for the recursion.

Closing this because I now realise automatic keys are required. I don't think global automatic keys are the way though, so a middle ground would be awesome. The getChild refactor will be useful though.


Nice!

Great work, and the implementation looks really nice!
Do you have any plans on supporting kv store on top of jetstream?
(https://docs.nats.io/using-nats/developer/develop_jetstream/kv)

I would say the answer is absolutely yes.
The KV support is in our roadmap. Actually, we have an internal version already to support KV for our enterprise customers.
This project aims to provide a way for general use to access JetStream on Node-RED platform. It's porting functions from the works we did on the client-side.

Great! Looking forward to following the development!


Galaxy S7 support

Seems like Galaxy S7 is now officially supported by ARCore: https://developers.google.com/ar/discover/#supported_devices
Is it also supported by WebARonARCore? If so, do would you like me to send a PR to update the list of supported devices in README?

Thank you for the note. What is happening is that WebARonARCore is based on ARCore DP1 (were only Pixel devices and S8 were supported). DP2 now supports more devices. The biggest problem is that adding DP2 support internally requires significant changes. We are studying some possible solutions. We will update this issue as we find the right solution.

Thank you so much!
I really admire your work on this project, and I am very excited about the endless possibilities it will create for authoring cross-platform AR content for the web. I even created a detailed step-by-step tutorial guiding fellow developers how to use this magical technology:
https://medium.com/@urish/web-powered-augmented-reality-a-hands-on-tutorial-9e6a882e323e
As soon as DP2 support lands, I will update my tutorial to reflect the new supported devices.
Thanks again, and keep up the good work!

#57 has landed already. Please, be mindful that the current version of WebARonARCore is not built against the ARCore 1.0 SDK. Still, as the communication bridge between the API and the ARCore service is the same, it seems to be working well. We are working on a major reimplementation of this whole project on top of the latest Chromium and ARCore. ETA still TBD.


Update event emitter code examples for Android

Thanks for looking into this.
Would you mind adding some description on why those things changed?

Hey @charpeni , The code is out of date, (mixins aren't used anymore in React.), and DeviceEventEmitter is now deprecated. See (https://github.com/facebook/react-native/blob/1490ab12ef156bf3201882eeabfcac18a1210352/Libraries/EventEmitter/RCTDeviceEventEmitter.js#L32).  Using NativeEventEmitter seems to now work in Android, and it works exactly like the iOS version.

Additionally, the examples have references to reactContext, so I added it to the constructor so it's easy for someone to reference it in their own class, and then easily copy and paste from the examples. This is in addition to WritableMap and Arguments


Problem building cscore with Raspbian Stretch (Pi 3)

Hi all -
Background: building a RPi vision processor for use with our Roborio.  Using Raspbian stretch.
Have successfully build and published wpiutil and ntcore, would like to also build cscore.
Currently have OpenCV 2.4.13 build and installed.
When I start the build, the following occurs:
pi@raspberrypi:~/cscore ./gradlew -PreleaseBuild -PskipAllTests build
FAILURE: Build failed with an exception.

What went wrong:
A problem occurred configuring root project 'cscore'.

Exception thrown while executing model rule: DependencyConfigRules#setupDependencyDownloads(ModelMap, DependencyConfigSpec, BinaryContainer, ProjectLayout, BuildConfigSpec)
Could not resolve all files for configuration ':nativeDeps'.
Could not find opencv-cpp-linuxarm.zip (org.opencv:opencv-cpp:3.2.0).
Searched in the following locations:
http://first.wpi.edu/FRC/roborio/maven/development/org/opencv/opencv-cpp/3.2.0/opencv-cpp-3.2.0-linuxarm.zip
Could not find wpiutil-cpp-linuxarm.zip (edu.wpi.first.wpiutil:wpiutil-cpp:3.424242.0.0-beta-2-20171111160242-4-g3e4e526).
Searched in the following locations:
file:/home/pi/releases/maven/development/edu/wpi/first/wpiutil/wpiutil-cpp/3.424242.0.0-beta-2-20171111160242-4-g3e4e526/wpiutil-cpp-3.424242.0.0-beta-2-20171111160242-4-g3e4e526-linuxarm.zip

Any pointers?   Do I need OpenCV 3 for cscore?  What other dependencies?
Thanks,
Rob

Is your final goal to run java or c++? If java, you will definitely need 3.2.0, and getting a proper build of opencv for that still needs some work on our end. If c++ we are actually working on a cmake build system that works with preinstalled opencv. I'll have that working tonight and could then send it.

If we were building for Roborio we would be using java, but RPi we're using c++.

Ok. For c++ on the pi the cmake build will work just fine. I'll get a bit more docs and post a reply to this tonight.

@rtrahms
https://github.com/ThadHouse/CmakeWpilib/tree/visionTestPackage
Here is a repo for you to try if you would like. It should just work with the opencv you built on the pi, but note that opencv < 3.2 has never been tested. But it should, and I would be interested in hearing. Note it includes a copy of the CameraServer API from wpilib, so if you want everything published it should just work. You'll need to set your team number in main.cpp, and probably change the AddAxisCamera call to AddUsbCamera, but other then that everything should just work properly.

I don't believe cscore would compile against < 3.0

@ThadHouse
I cloned the repo above, had some cmake/make issues not related to the camera_server stuff - more on no make rules for wpiutil.

Hmm. You cloned the visionTestPackage branch, right? It worked on my windows desktop, and another person tested it on a TK1 and it worked. Thats weird you were getting errors. If you don't have it working later today, tomorrow I'll try installing Stretch on my Pi and try it out directly there.

pi@raspberrypi:~ $ git clone https://github.com/ThadHouse/CmakeWpilib -b visionTestPackage
Cloning into 'CmakeWpilib'...
remote: Counting objects: 76, done.
remote: Compressing objects: 100% (52/52), done.
remote: Total 76 (delta 26), reused 72 (delta 22), pack-reused 0
Unpacking objects: 100% (76/76), done.
pi@raspberrypi:~ $ cd CmakeWpilib/
pi@raspberrypi:~/CmakeWpilib $ dir
CMakeLists.txt	cscore_main.cpp  include  libraries  ntcore_main.cpp  src  wpiutil_main.cpp
pi@raspberrypi:~/CmakeWpilib $ cmake .
-- The C compiler identification is GNU 6.3.0
-- The CXX compiler identification is GNU 6.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
You have called ADD_LIBRARY for library wpiutil without any source files. This typically indicates a problem with your CMakeLists.txt file
You have called ADD_LIBRARY for library ntcore without any source files. This typically indicates a problem with your CMakeLists.txt file
You have called ADD_LIBRARY for library cscore without any source files. This typically indicates a problem with your CMakeLists.txt file
-- Configuring done
CMake Error: Cannot determine link language for target "wpiutil".
CMake Error: CMake can not determine linker language for target: wpiutil
CMake Error: Cannot determine link language for target "ntcore".
CMake Error: CMake can not determine linker language for target: ntcore
-- Generating done
-- Build files have been written to: /home/pi/CmakeWpilib
pi@raspberrypi:~/CmakeWpilib $
Am I missing something?
Rob

Ohhh. You need to clone the submodules as well.
git submodule update --init --recursive

Much further, thanks!
Made it through most of the build this time, failed right toward the end during cscore library build:
[ 96%] Built target cscore
[ 97%] Building CXX object CMakeFiles/camera_vision.dir/src/CameraServer.cpp.o
/home/pi/CmakeWpilib/src/CameraServer.cpp: In member function ‘cs::UsbCamera frc::CameraServer::StartAutomaticCapture()’:
/home/pi/CmakeWpilib/src/CameraServer.cpp:495:14: error: ‘HALUsageReporting’ has not been declared
HAL_Report(HALUsageReporting::kResourceType_PCVideoServer,
^~~~~~~~~~~~~~~~~
/home/pi/CmakeWpilib/src/CameraServer.cpp:496:32: error: ‘HAL_Report’ was not declared in this scope
camera.GetHandle());
^
/home/pi/CmakeWpilib/src/CameraServer.cpp: In member function ‘cs::UsbCamera frc::CameraServer::StartAutomaticCapture(int)’:
/home/pi/CmakeWpilib/src/CameraServer.cpp:507:14: error: ‘HALUsageReporting’ has not been declared
HAL_Report(HALUsageReporting::kResourceType_PCVideoServer,
^~~~~~~~~~~~~~~~~
/home/pi/CmakeWpilib/src/CameraServer.cpp:508:32: error: ‘HAL_Report’ was not declared in this scope
camera.GetHandle());
^
/home/pi/CmakeWpilib/src/CameraServer.cpp: In member function ‘cs::UsbCamera frc::CameraServer::StartAutomaticCapture(llvm::StringRef, int)’:
/home/pi/CmakeWpilib/src/CameraServer.cpp:516:14: error: ‘HALUsageReporting’ has not been declared
HAL_Report(HALUsageReporting::kResourceType_PCVideoServer,
^~~~~~~~~~~~~~~~~
/home/pi/CmakeWpilib/src/CameraServer.cpp:517:32: error: ‘HAL_Report’ was not declared in this scope
camera.GetHandle());
^
/home/pi/CmakeWpilib/src/CameraServer.cpp: In member function ‘cs::UsbCamera frc::CameraServer::StartAutomaticCapture(llvm::StringRef, llvm::StringRef)’:
/home/pi/CmakeWpilib/src/CameraServer.cpp:525:14: error: ‘HALUsageReporting’ has not been declared
HAL_Report(HALUsageReporting::kResourceType_PCVideoServer,
^~~~~~~~~~~~~~~~~
/home/pi/CmakeWpilib/src/CameraServer.cpp:526:32: error: ‘HAL_Report’ was not declared in this scope
camera.GetHandle());
^
CMakeFiles/camera_vision.dir/build.make:62: recipe for target 'CMakeFiles/camera_vision.dir/src/CameraServer.cpp.o' failed
make[2]: *** [CMakeFiles/camera_vision.dir/src/CameraServer.cpp.o] Error 1
CMakeFiles/Makefile2:69: recipe for target 'CMakeFiles/camera_vision.dir/all' failed
make[1]: *** [CMakeFiles/camera_vision.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2
pi@raspberrypi:~/CmakeWpilib $
Looking to see where HALUsageReporting is defined...

Hmm I thought I removed all of those. Just remove them. Its defined in wpilib, and not that project.

Success!  Snippet of the last part of the build...
Scanning dependencies of target camera_vision
[ 97%] Building CXX object CMakeFiles/camera_vision.dir/src/CameraServer.cpp.o
[ 98%] Building CXX object CMakeFiles/camera_vision.dir/src/main.cpp.o
[100%] Linking CXX executable camera_vision
[100%] Built target camera_vision
I built the usbcvstream example successfully as well, and also ran it successfully.  Not sure where the video is going yet, but it is reporting frames captured and the camera is active.
Snippet from my Makefile for that:
all:	usbcvstream
clean:
rm -f usbcvstream
INCLUDE_DIRS = -I /home/pi/ntcore/src/main/native/include -I /home/pi/wpiutil/src/main/native/include -I/home/pi/CmakeWpilib/libraries/cscore/cscore/src/main/native/include
LIB_DIRS = -L /home/pi/ntcore/build/libs/ntcore/shared -L /home/pi/wpiutil/build/libs/wpiutil/shared -L/home/pi/CmakeWpilib/libraries/cscore
...
usbcvstream: usbcvstream.cpp
g++ -o usbcvstream usbcvstream.cpp $(INCLUDE_DIRS) pkg-config opencv --cflags --libs -std=gnu++11  $(LIB_DIRS) -lwpiutil -lntcore -lcscore -lv4l2

Sweet. Note that the main example provided with a change to switch to USB cameras should stream images as well.
If you are trying to use usbcvstream, it will output the original image on ipaddr:8081, and the modified image on ipaddr:8082. If using the example in camera_vision, it will stream to ipaddr:1181 and modified image (the bgr to hsv conversion in the main looptoipaddr:1182`

I am seeing the CV page with images locally on the RPi at http://localhost:8081/.  Playing around with it, everything seems to be working well!  Thanks for the help - RPi/OpenCV2 works great with this - confirmed.

Cool!. Note I do recommend using the main one, and not the usbcvstream. It will post all the data correctly to NetworkTables, and allow all the dashboards to properly connect to the stream. It's already set up to do the same things as the usbcvstream anyway.

This is fixed with wpilibsuite/allwpilib#1015, which brings cmake builds to the merged wpilib repository.


Infinite loop when mapping unions

@PabloSzx,
I am experiencing this problem with the gqty:2.1.0.
I am mapping properties where there's a union. I want to have an ID mapped for both of them.
const properties = certificateResult.$on.Certificate.properties.map((config) => ({
  // This does not cause an inifinite loop
  id: config?.$on.CertificatePropertySimple?.property.id,

  // This causes an infinite loop
  id: (config?.$on.CertificatePropertySimple || config?.$on.CertificatePropertyDate)?.property.id

  // This as well
  id: config?.__typename === 'CertificatePropertySimple'
      ? config?.$on.CertificatePropertySimple?.property.id
      : config?.$on.CertificatePropertyDate?.property.id,
}));

return
{properties[1]?.id}
;

It prints out the ID, but still causes an infinite loop. Why is that, and how to prevent it?
This sounds similar to me as this other already closed issue https://github.com/gqty-dev/gqty/issues/236
Update:
I have tried accessing the data directly, without any mapping, and with the first part of the union it works like a charm, but with the second one, it causes the infinite loop.
// This works
return
{certificateResult.$on.Certificate.properties[1]?.$on.CertificatePropertySimple?.property.id}

// However, accessing the second union part causes the infinite loop
return
{certificateResult.$on.Certificate.properties[0]?.$on.CertificatePropertyDate?.property.id}

// This is how the properties array looks like, should the indexes on the properties cause any confusion
certificateResult.$on.Certificate.properties === [
// 0. $on.CertificatePropertyDate
// 1. $on.CertificatePropertySimple
]

Update 2:
The problem I believe is that the library doesn't resolve the cache very well, thus always things it needs to fetch more to complete it.

I'm closing this because I'm no longer using this library.
If anyone else experiences this problem, feel free to re-open it.

I also get an infinite loop when trying to access some values:

        {query.allMfes().edges?.map((e) => {
          return e.node.version;
        })}

If i drop the .version (or any other field access off of e.node), the infinite loop halts, but of course react cannot render a that object.
The infinite loop of API calls also does re-trigger the loading state. I think the cache resolve suggestion above is correct, as my queries do have the full data present!

I also get an infinite loop when trying to access some values:

        {query.allMfes().edges?.map((e) => {
          return e.node.version;
        })}

If i drop the .version (or any other field access off of e.node), the infinite loop halts, but of course react cannot render a that object (the proxy?).
The infinite loop of API calls also does re-trigger the loading state. I think the cache resolve suggestion above is correct, as my queries do have the full data present!
Inspection revealed the following. There is a for loop, processing lookup selections. The key generated is not in the cache. The key also is not representative of the query i authored--there is a 0 suffixed onto it. Thus, though the cache is fully hydrated with correct data, the Selection lookup fails, due to this wrong key:

this is not related to the union logic, it has to be a new issue with a reproduction


Feat upgrade llama-index-vector-stores-chroma and rag test coverage 100%

Features

Feat upgrade llama-index-vector-stores-chroma and rag test coverage 100%

Feature Docs

Influence

Result

Other

Codecov Report
Attention: Patch coverage is 66.66667% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 82.43%. Comparing base (90e1b62) to head (c8fe6db).
Report is 1 commits behind head on main.

Files
Patch %
Lines

metagpt/environment/minecraft_env/minecraft_env.py
0.00%
1 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1126      +/-   ##
==========================================
+ Coverage   81.33%   82.43%   +1.10%
==========================================
  Files         251      249       -2
  Lines       13980    13836     -144
==========================================
+ Hits        11370    11406      +36
+ Misses       2610     2430     -180

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Request to merge TSPLIB4J into OR project

Dear David,
I've been working on several Operations Research (OR) projects. In an attempt to simplify and speed up the implementation of these projects, I'm currently bundling several well-known algorithms, frameworks etc into an open source Java library which is hosted on Github: https://github.com/jkinable/jorlib
Currently I'm working on the integration of a Column Generation framework into the project, but that's still part of the development branch. I'll incorporate it into the master branch somewhere next week.
To extend the code base, I was wondering whether you would allow me to include the TSPLib parser from TSPLIB4J into the project. This would be a great addition as TSP related problems occur very often in the area of OR. Would you be interested in contributing the code yourself? Alternatively, I can merge it myself?
Thank you for your effort!
Joris

Joris,
I'm sorry for not responding sooner.  I never received an e-mail notification for this issue.  I'm more than happy for you to use the code (and am glad you've already merged the code without waiting for a response from me).
Best,
David


example not like xcape

Hi,
Just ran the example, but I can't really get it to work, if I tap caps it does indeed send escape, but it also lights up and then works as a caps lock key. If I use it on left_ctrl it doesn't work. Should it work like xcape ? (havn't used it in a while, before i was using caps2esc on linux).

nevermind works at advertised, I had to set xkb_options ctrl:nocaps in sway as mentioned elsewhere in the issues. Also I had accidently enabled both should_esc and should_lshift at the same time.

I had accidently enabled both should_esc and should_lshift at the same time

FYI, these are internal variables of the script that determine whether esc/lshift will be typed on the next event. They are not control knobs :)


Issue with Ingress Packets Not Reflecting Changes

I am currently working on a project where I need to remove padding from ingress packets using an ingress classifier. The logs indicate that the packet dimensions are correctly reduced, but it appears that the changes are not propagated to the upper levels.
Upon further investigation, I suspect that the packet is being propagated to the upper levels before the eBPF program completes its execution. I experimented with returning TC_ACT_SHOT at different points in the code, and I noticed that it only takes effect in the first few lines. When placed at the end of the function, it doesn’t work as expected. Could it be an issue related to asynchronicity?
This is the function I'm having problems with:
fn try_proxy_with_tc_aya(mut ctx: TcContext) -> Result<i32, i32> {
    let eth_hdr: EthHdr = ctx.load(0).map_err(|_| TC_ACT_OK)?;
    match eth_hdr.ether_type {
        EtherType::Ipv4 => {}
        _ => return Ok(TC_ACT_OK),
    }

    let ipv4_hdr: Ipv4Hdr = ctx.load(EthHdr::LEN).map_err(|_| TC_ACT_OK)?;
    let source = u32::from_be(ipv4_hdr.src_addr);

    if source != CLIENT_IP {
        info!(&ctx, "NON CLIENT_IP: {:x}", source);
        return Ok(TC_ACT_OK);
    }

    if ipv4_hdr.proto != IpProto::Tcp.into() {
        info!(&ctx, "NON TCP");
        return Ok(TC_ACT_OK);
    }

    let ipv4_len: usize =  (ipv4_hdr.ihl() << 2).into();
    let tcp_offset = EthHdr::LEN + ipv4_len;
    let tcp_hdr: TcpHdr = ctx.load(tcp_offset).map_err(|_| TC_ACT_OK)?;
    let tcp_len: usize = (tcp_hdr.doff() << 2).into();
    let http_offset = tcp_offset + tcp_len;

    info!(&ctx, "len {}", ctx.len());

    if ctx.len() <= http_offset as u32 {
        info!(&ctx, "NO HTTP, len: {} {}", ctx.len(), ctx.data_end() as usize - ctx.data() as usize);
        return Ok(TC_ACT_OK);
    }

    ctx.adjust_room(PADDING_LEN, BPF_ADJ_ROOM_NET, 0).map_err(|_| TC_ACT_OK)?;
    ctx.store(tcp_offset, &tcp_hdr, 0).map_err(|_| TC_ACT_OK)?;

    info!(&ctx, "ADJUSTED, len: {} {}", ctx.len(), ctx.data_end() as usize - ctx.data() as usize);

    Ok(TC_ACT_OK)
}

This is the link to the project.
As I am not an expert in Aya or eBPF, I might be overlooking something crucial. Any insights or suggestions would be greatly appreciated.

I have seen something similar when I was changing checksum of packet during egress using TC egress classifier.
I was modifying the checksum of TCP header during egress which was somehow getting over-written by the kernel i believe.
I didn't found the root cause, just that my changed in packet was getting over-written somehow (as those changes were not captured when TCP dump captured the egress packet).


Feature Request: Filetype recognition and/or ignore path

Most of the hiccups I had when I first started using mjml-utils had to do with the fact that it kept throwing an error on what I finally determined was my Assets folder. It was looking for typical MJML markup in every single file, not just my base .mjml files. I believe it also tries to apply such rules to folders, because even moving all assets outside of the build folder with the exception of an include (still .mjml) still threw the error because it was in its own folder.
It would be great to not be trying to look for markup in non-related files/folders, and also would be nice to recognize partials (includes) as such, since according to MJML documentation, they do not require full markup.
Another solution would be to be able to include an option for ignore path(s) added to the --build and --watch commands to set in your package.json so it just skips those files/folders entirely.

It sounds like this could be broken down into a couple of issues. I'll outline them below. Please let me know if it's in line with your thinking, then I'll work to get them resolved.

The watcher watches for changes in all files (within the input directory) when it should only watch *.mjml files
This library needs to support the <mj-include /> component and properly resolve its path attribute, regardless of the task that is being run

Sounds about right. To clarify, the current use case is having an assets
folder inside the watched folder, currently throwing errors about the lack
of proper mjml scripting. And as a corollary, any partials inside such
folders, though they be of .mjml filetype, also should not throw an error
for lacking top-level tags.
Now how about some real pain-in-the-butt suggestions? Not sure if it's
inside your scope or realm of willingness, but the following would make
this a truly powerful and unique tool.
A simple duplicate command for assets would further streamline the build
process. If that were included, I'd propose optional settings specifying a
dupe folder (for assets like images) and a partials folder in the package
run scripts (since partials need only be rendered, not living in the
completed folder). You could also just require a standard naming convention
like "assets" and "includes".
I'm currently only just getting started with mjml, so I have a master raw
folder with everything at that root level. I can imagine, though, that not
much further down the line, I'd want to split everything further into
related folders and not want to constantly change my build/watch scripts,
in which case it would be necessary for the build command to dig down into
those folders. In this case, if the duplicate/partials features were
implemented, you'd need to either go the standard naming route or note that
their personal settings must remain constant across projects.
On May 30, 2017 8:35 PM, "Justin Sisley"<EMAIL_ADDRESS>wrote:

It sounds like this could be broken down into a couple of issues. I'll
outline them below. Please let me know if it's in line with your thinking,
then I'll work to get them resolved.

The watcher watches for changes in all files (within the input
directory) when it should only watch *.mjml files
This library needs to support the  component and
properly resolve its path attribute, regardless of the task that is
being run

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/justinsisley/mjml-utils/issues/7#issuecomment-305057016,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AMELwYc7LZ092xnzQFaqqIhxHDesucnZks5r_MPigaJpZM4NqhLt
.

I apologize for the long delay in responding here. I've been preoccupied with a couple of products that I'm building.
v2.2.0 should have solved points 1 and 2 in my previous comment, therefore I'm going to close this issue.
So you know, I've separated my templates in a similar fashion to your suggestion in your last comment. My "core" email templates are in an "email" directory in my projects, and partial templates are kept in a "partials" sub-directory. Using the <mj-include> directive, this makes my email templates really easy to organize and maintain.
Thanks for the suggestions!


configure-authentication only support internal auth

We'd like to add support for the SAML options when configuring authentication. They are documented in the OpsMan docs.

PR #152

Just merged https://github.com/pivotal-cf/om/pull/152, running through CI now and we'll cut a release shortly after. Thanks for the PR!


Remove link text field from link replacement form

Short description
Disabled link text field in the link replacement form caused several issues:

The existence of the field led to confusion
The replacement form required scrolling on smaller screens
It was no longer possible to replace the src attributes of img tags

Proposed changes
Revert the previous link replacement form (without the disabled link text field)
Side effects
No?
However I am not quite sure what was meant by It was no longer possible to replace the src attributes of img tags because I think I can replace src value for images on develop.
Resolved issues
Fixes: #2307

Pull Request Review Guidelines

Oh, we should probably add a release note, since this change is visible for users 👀


Experiment with alternate search/exploration strategies

Experiment with additional search/exploration strategies.
Possibilities:

Place word in order that their overlapping letters occur, e.g. try to chain them
Alternate between long/short words
Mix-up the orientations, e.g. points towards the edges (tried this, didn't work well)

Dumb idea, already tried this


Make gaussian blur using separable convolution

Hi,
I just noticed that Gaussian blur is not implemented using separable convolutions so I implemented it. The speed up is orders of magnitude for large kernels and large images.
Basically on my machine I get a speedup of 20x for a blur radius 30 pixels and a relatively large image as shown below:
$ time bild_0.11 blur gaussian --radius 30 ~/Downloads/cat.jpg ~/Downloads/out.jpg

real	0m29.641s
user	1m28.880s
sys	0m0.320s

$ time bild-fast-gaussian blur gaussian --radius 30 ~/Downloads/cat.jpg ~/Downloads/out.jpg

real	0m1.430s
user	0m5.316s
sys	0m0.036s

$ file ~/Downloads/cat.jpg
/path/to/my/home/Downloads/cat.jpg: JPEG image data, Exif standard: [TIFF image data, little-endian, direntries=1, copyright=Image protected by copyright. Contact  National Geographic Creative at: Telephone:202.857.7537,], baseline, precision 8, 2048x1365, frames 3

Cheers,
Angelos

Looks good! I tried it on my laptop and it's a really great speed up! As a side effect, some other functions that make use of gaussian blur internally like the UnsharpMask will greatly benefit from this.
Before we merge it, could you please take a look at:

Please rename the Tranpose function to Transposed since we use that convention for immutable operations.

I think the transpose is not returning the correct result for non-square matrices, here's some test cases we could add:

func TestKernelTransposed(t *testing.T) {
	cases := []struct {
		desc     string
		kernel   *Kernel
		expected *Kernel
	}{
		{
			desc: "all zero",
			kernel: &Kernel{[]float64{
				0, 0, 0,
				0, 0, 0,
				0, 0, 0,
			}, 3, 3},
			expected: &Kernel{[]float64{
				0, 0, 0,
				0, 0, 0,
				0, 0, 0,
			}, 3, 3},
		},
		{
			desc: "one element",
			kernel: &Kernel{[]float64{
				0, 0, 0,
				0, 1, 0,
				0, 0, 0,
			}, 3, 3},
			expected: &Kernel{[]float64{
				0, 0, 0,
				0, 1, 0,
				0, 0, 0,
			}, 3, 3},
		},
		{
			desc: "diagonal",
			kernel: &Kernel{[]float64{
				1, 0, 0,
				0, 1, 0,
				0, 0, 1,
			}, 3, 3},
			expected: &Kernel{[]float64{
				1, 0, 0,
				0, 1, 0,
				0, 0, 1,
			}, 3, 3},
		},
		{
			desc: "3x2",
			kernel: &Kernel{[]float64{
				1, 1, 1,
				0, 1, 0,
			}, 3, 2},
			expected: &Kernel{[]float64{
				1, 0,
				1, 1,
				1, 0,
			}, 2, 3},
		},
		{
			desc: "5x1",
			kernel: &Kernel{[]float64{
				1, 1, 1, 0, 1,
			}, 5, 1},
			expected: &Kernel{[]float64{
				1,
				1,
				1,
				0,
				1,
			}, 1, 5},
		},
	}

	for i, c := range cases {
		actual := c.kernel.Transposed()
		if !kernelEqual(actual.(*Kernel), c.expected) {
			t.Errorf("%s case #%d: expected: %#v, actual: %#v", "KernelTransposed "+c.desc, i, c.expected, actual)
		}
	}
}

Oh, man you are so right (embarassed :) ). Let me fix these real quick and update the pull request.

Overwrote the previous commit. Added your test. I also ran the full tests this time and changed the expected outputs for effects because of the two pass algorithm some values change by 1 as was the case for the blur test.
Cheers,
Angelos

Looks great! Will prepare a release with this.


Add Prometheus statistics and an example to showcase them using Grafana

This PR takes TJ's code changes in https://github.com/fnproject/fn/pull/357 and fixes a TODO in that code so that the promhttp.Handler() is created just once.
I've also created an  example examples/grafana which shows how to run Prometheus and Grafana and install a dashboard fn_grafana_dashboard.json which showcases these metrics (queued/running/completed/failed) in a similar format to the ui tool. See screenshot....

Added screenshot. See
https://github.com/fnproject/fn/tree/add_prometheus_metrics/examples/grafana

Applied all the edits suggested by @derekschultz


prost-type: impl Hash for Duration

👋
Our proxy, which checks if we are not making the same RPC with the same payload at the same time, requires the implementation of a hash.

@LucioFranco do you think that this patch can be merged ?


Improve Language.md.  Rename it Stitch.md

Can never have too much documentation!

This is a duplicate of #390
(or is #390 a duplicate of this...?)


wallet file crack after update from 0.9.1 to 0.9.2

Wallet file crack after update from 0.9.1 to 0.9.2
also many user have this problem too

you can import from automated backups ...

locate you wallet folders according to http://wiki.bitshares.org/index.php/BitShares/Config
rename wallets/default into something else (as a backup)
launch the client and click "import wallet" from the main menu
go to the wallet folder from 1)
select the most recent backup in wallets/.backups/default/ for import
provide your passphrase ..
done

It's not clear to me what this means; if you are still having a problem, please post a more detailed report in the technical support forum here: https://bitsharestalk.org/index.php/board,45.0.html


Study inquiry status display

When a user inquires about a study the two versions of the components are out of sync until a refresh of the page (One component tells them that they've inquired and the other component still invites them to inquire).

This bug will be fixed when addressing the layout changes requested in #427


[SPARK-2960][Deploy] Support executing Spark from symlinks (reopen)

This PR is based on the work of @roji to support running Spark scripts from symlinks. Thanks for the great work @roji . Would you mind taking a look at this PR, thanks a lot.
For releases like HDP and others, normally it will expose the Spark executables as symlinks and put in PATH, but current Spark's scripts do not support finding real path from symlink recursively, this will make spark fail to execute from symlink. This PR try to solve this issue by finding the absolute path from symlink.
Instead of using readlink -f like what this PR (https://github.com/apache/spark/pull/2386) implemented is that -f is not support for Mac, so here manually seeking the path through loop.
I've tested with Mac and Linux (Cent OS), looks fine.
This PR did not fix the scripts under sbin folder, not sure if it needs to be fixed also?
Please help to review, any comment is greatly appreciated.

Merged build triggered.

Merged build started.

Test build #42204 has started for   PR 8669 at commit 69bde5c.

Other than that if it doesn't change existing behavior and lets symlinks stand a chance of working, seems good.

Test build #42204 has finished for   PR 8669 at commit 69bde5c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42204/
Test PASSed.

Merged build finished. Test PASSed.

Applying the same fix to sbin executables would be great. In fact, the whole fix would be incomplete without it.

Hi @patrungel , I've also fixed this issue for sbin executables, mind taking a look at it?

Merged build triggered.

Merged build started.

Test build #42245 has started for   PR 8669 at commit d6183b9.

Test build #42245 has finished for   PR 8669 at commit d6183b9.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42245/
Test PASSed.

Merged build finished. Test PASSed.

Hi, @jerryshao
I did a quick test with alternatives (against sbin/start-history-server.sh); the fix works to me as expected. Thanks tons.

Can the following be fixed as well?
--- a/bin/load-spark-env.sh    2015-10-08 14:51:20.874478368 +0000
+++ b/bin/load-spark-env.sh    2015-10-08 15:04:02.109674611 +0000
@@ -47,10 +47,7 @@
 if [ -z "$SPARK_ENV_LOADED" ]; then
   export SPARK_ENV_LOADED=1

-  # Returns the parent of the directory this script lives in.
-  parent_dir="$(cd "`dirname "$0"`"/..; pwd)"
-
-  user_conf_dir="${SPARK_CONF_DIR:-"$parent_dir"/conf}"
+  user_conf_dir="${SPARK_CONF_DIR:-"$FWDIR"/conf}"

   if [ -f "${user_conf_dir}/spark-env.sh" ]; then
     # Promote all variable declarations to environment (exported) variables

Woah, this PR is really big! @jerryshao, do you think that you could move this function into a common "helper library" file then source / include that file into the other scripts? That would help to reduce duplication here.

@JoshRosen yes agree. See the discussion above out the small chicken-and-egg problem there about the location of the helper script. Still may be a better solution than this much duplication, or else, declare that SPARK_HOME must be set.

I think this is a whole lot of change, and the purpose really is just to avoid setting SPARK_HOME when Spark is symlinked. But that's pretty standard procedure. RIght? I think we should resolve this as wont-fix.

The change aims to make spark binaries launch properly when symlinked, not just to avoid setting SPARK_HOME.
As of current master, SPARK_HOME value is overwritten in launchers by a
export SPARK_HOME="$(cd "`dirname "$0"`"/..; pwd)"

which is generically wrong and does not work for a symlinked set-up.
This behaviour prohibits proper ( = with alternatives) installations/packaging.
So, if 'resolve this as wont-fix' refers to the issue, I beg to differ (this discussion should rather be moved to the ticket then).
If it refers just to the change suggested, I don't protest it.

Gotcha, is the way forward really to not over-write SPARK_HOME if set? Because most of the work here seems to be about discovering the home otherwise. The symlinks themselves are fine, just not quite suitable for discovering the real location of the target.

Hi @srowen , I admitted currently solution is quite ugly, but like what you mentioned, it is a chicken-and-egg problem, hard to remove the duplication. Another solution is by setting SPARK_HOME to manually support symlink, but it still needs to change the current executables, currently we don't honor SPARK_HOME. Take beeline as a example, even SPARK_HOME is set, still we could not execute from symlink.
So if setting SPARK_HOME as a compromise solution agreed by you guys, I could update the codes.
What's your opinion?

If SPARK_HOME is the standard way, we should update the scripts to honor this, it's not a good idea to make this issue as won't fix, because lots of deployment and package actually require symlink.

Merged build triggered.

Merged build started.

Hi @srowen @patrungel @JoshRosen and @vazin, I changed this patch to honor SPARK_HOME to solve symlink issue, if users want to run the executables from symlink, they should set SPARK_HOME manually. This instead removes lots of duplications, still has the ability to run on symlinkd, please help to review, any comment is greatly appreciated.

Test build #44400 has started for PR 8669 at commit c796397.

Merged build triggered.

Merged build started.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44401/
Test FAILed.

Jenkins, retest this please.

Merged build triggered.

Merged build started.

Test build #44408 has started for PR 8669 at commit 41511b4.

Test build #44400 has finished for PR 8669 at commit c796397.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):\n  *     \"$\n  *     \"$\n  * exec \"$\n  * exec \"$\n  *       nohup nice -n \"$SPARK_NICENESS\" \"$\n  *       nohup nice -n \"$SPARK_NICENESS\" \"$\n  *   \"$\n

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44400/
Test PASSed.

Test build #44408 has finished for PR 8669 at commit 41511b4.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):\n  *     \"$\n  *     \"$\n  * exec \"$\n  * exec \"$\n  *       nohup nice -n \"$SPARK_NICENESS\" \"$\n  *       nohup nice -n \"$SPARK_NICENESS\" \"$\n  *   \"$\n

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44408/
Test PASSed.

Merged build triggered.

Merged build started.

Test build #44423 has started for PR 8669 at commit 28fd58d.

Test build #44423 has finished for PR 8669 at commit 28fd58d.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):\n  *     \"$\n  *     \"$\n  * exec \"$\n  * exec \"$\n  *       nohup nice -n \"$SPARK_NICENESS\" \"$\n  *       nohup nice -n \"$SPARK_NICENESS\" \"$\n  *   \"$\n

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44423/
Test FAILed.

Jenkins, retest this please.

Merged build triggered.

Merged build started.

Test build #44431 has started for PR 8669 at commit 28fd58d.

Test build #44431 has finished for PR 8669 at commit 28fd58d.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):\n  *     \"$\n  *     \"$\n  * exec \"$\n  * exec \"$\n  *       nohup nice -n \"$SPARK_NICENESS\" \"$\n  *       nohup nice -n \"$SPARK_NICENESS\" \"$\n  *   \"$\n

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44431/
Test PASSed.

Merged build finished. Test PASSed.

Merged build triggered.

Merged build started.

Test build #44483 has started for PR 8669 at commit 5795800.

Test build #44483 has finished for PR 8669 at commit 5795800.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):\n  *     \"$\n  *     \"$\n  * exec \"$\n  * exec \"$\n  *       nohup nice -n \"$SPARK_NICENESS\" \"$\n  *       nohup nice -n \"$SPARK_NICENESS\" \"$\n  *   \"$\n

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44483/
Test PASSed.

Merged build triggered.

Merged build started.

Test build #44988 has started for PR 8669 at commit 16ea884.

Test build #44988 has finished for PR 8669 at commit 16ea884.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):\n  *     \"$\n  *     \"$\n  * exec \"$\n  * exec \"$\n  *       nohup nice -n \"$SPARK_NICENESS\" \"$\n  *       nohup nice -n \"$SPARK_NICENESS\" \"$\n  *   \"$\n

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44988/
Test PASSed.

LGTM. There's a lot going on here but it's all really standardizing on one approach to defining the Spark base directory. I scanned all the lines and it looks correct and carefully applied, so I feel good about merging this to master.

It has been pointed out that this commit might be the cause of a regression that is possibly holding up the 1.6 release (https://issues.apache.org/jira/browse/SPARK-12345).  It would be great if those involved could take a look.

Thanks @marmbrus for your notice, I will take a look at the issues.

It's a good lead; I'm looking too. The effect of this change should be to respect SPARK_HOME if it's set to find Spark scripts and files. It's not clear why this would cause a remote machine to use the home dir set on the driver, unless something is shipping the SPARK_HOME value to the remote machine. I'll comment on the JIRA.

The MesosDispatchScheduler ships a number of environment variables.

@dragos  Did you specify SPARH_HOME manually in your environment? By default no one will specify this SPARK_HOME.

@jerryshao though the scripts will now set SPARK_HOME for you if not already set.
Should the Mesos code ship this variable across machines though? I'd have thought it's properly local to a machine only.
If this is right, one of these two things could be a way forward: don't export SPARK_HOME or don't send it.

Ohh, I see. That possibly will be the problem. One fix is to remove export in all the scripts, only using SPARK_HOME as a shell variable.

@dragos , if spark.mesos.executor.home is specified to the directory where Spark is installed in the slave node, does your problem still exist? Also from my understanding, if it is due to SPARK_HOME inconsistency between driver and executor, client mode also should have such problem, is it right?

I'll check that. I don't have Spark installed on every slave node in my test setup.

One more question @dragos , did you specify spark.executor.uri or SPARK_EXECUTOR_URI?

I believe I tried both..

@jerryshao I observed the same failure using spark.mesos.executor.home

I see, can you please confirm Spark is installed in this path spark.mesos.executor.home?

@jerryshao Yes, it is.

It is a little weird. I assume if spark.mesos.executor.home is correctly set, from the code Mesos should be able to find the right path of executables:
    val executorSparkHome = conf.getOption("spark.mesos.executor.home")
      .orElse(sc.getSparkHome())
      .getOrElse {
        throw new SparkException("Executor Spark home `spark.mesos.executor.home` is not set!")
      }

      val runScript = new File(executorSparkHome, "./bin/spark-class").getCanonicalPath
      command.setValue(
        "%s \"%s\" org.apache.spark.executor.CoarseGrainedExecutorBackend"
          .format(prefixEnv, runScript) +
        s" --driver-url $driverURL" +
        s" --executor-id ${offer.getSlaveId.getValue}" +
        s" --hostname ${offer.getHostname}" +
        s" --cores $numCores" +
        s" --app-id $appId")

Anyway I think we should not export SPARK_HOME in all the scripts, what do you think @srowen ?

I admit it is very surprising. Here's what I see
Driver page on the Mesos dispatcher (note that spark.executor.uri is set, but also SPARK_HOME -- which is wrong, since it's an absolute path on my machine):

And here is the log. Note that the executor is fetched correctly, but the submit script is calculated relative to SPARK_HOME:
I1216 09:18:44.974624   464 fetcher.cpp:369] Fetching URI '/var/spark/spark-1.6.0-bin-hadoop2.6.tgz'
I1216 09:18:44.974628   464 fetcher.cpp:243] Fetching directly into the sandbox directory
I1216 09:18:44.974639   464 fetcher.cpp:180] Fetching URI '/var/spark/spark-1.6.0-bin-hadoop2.6.tgz'
I1216 09:18:44.974652   464 fetcher.cpp:160] Copying resource with command:cp '/var/spark/spark-1.6.0-bin-hadoop2.6.tgz' '/tmp/mesos/slaves/ac7d8645-030b-4c99-a47f-2e02379d8c41-S1/frameworks/ac7d8645-030b-4c99-a47f-2e02379d8c41-0002/executors/driver-20151216101839-0001/runs/ab128426-95fa-4a3e-82f6-9c5b4baba816/spark-1.6.0-bin-hadoop2.6.tgz'
I1216 09:18:46.351866   464 fetcher.cpp:76] Extracting with command: tar -C '/tmp/mesos/slaves/ac7d8645-030b-4c99-a47f-2e02379d8c41-S1/frameworks/ac7d8645-030b-4c99-a47f-2e02379d8c41-0002/executors/driver-20151216101839-0001/runs/ab128426-95fa-4a3e-82f6-9c5b4baba816' -xf '/tmp/mesos/slaves/ac7d8645-030b-4c99-a47f-2e02379d8c41-S1/frameworks/ac7d8645-030b-4c99-a47f-2e02379d8c41-0002/executors/driver-20151216101839-0001/runs/ab128426-95fa-4a3e-82f6-9c5b4baba816/spark-1.6.0-bin-hadoop2.6.tgz'
I1216 09:18:48.625231   464 fetcher.cpp:84] Extracted '/tmp/mesos/slaves/ac7d8645-030b-4c99-a47f-2e02379d8c41-S1/frameworks/ac7d8645-030b-4c99-a47f-2e02379d8c41-0002/executors/driver-20151216101839-0001/runs/ab128426-95fa-4a3e-82f6-9c5b4baba816/spark-1.6.0-bin-hadoop2.6.tgz' into '/tmp/mesos/slaves/ac7d8645-030b-4c99-a47f-2e02379d8c41-S1/frameworks/ac7d8645-030b-4c99-a47f-2e02379d8c41-0002/executors/driver-20151216101839-0001/runs/ab128426-95fa-4a3e-82f6-9c5b4baba816'
I1216 09:18:48.625273   464 fetcher.cpp:446] Fetched '/var/spark/spark-1.6.0-bin-hadoop2.6.tgz' to '/tmp/mesos/slaves/ac7d8645-030b-4c99-a47f-2e02379d8c41-S1/frameworks/ac7d8645-030b-4c99-a47f-2e02379d8c41-0002/executors/driver-20151216101839-0001/runs/ab128426-95fa-4a3e-82f6-9c5b4baba816/spark-1.6.0-bin-hadoop2.6.tgz'
I1216 09:18:48.666476   539 logging.cpp:172] INFO level logging started!
I1216 09:18:48.667392   539 exec.cpp:134] Version: 0.25.0
I1216 09:18:48.669401   545 exec.cpp:208] Executor registered on slave ac7d8645-030b-4c99-a47f-2e02379d8c41-S1
bin/spark-submit: line 27: /Users/dragos/workspace/Spark/dev/rc-tests/spark-1.6.0-bin-hadoop2.6/bin/spark-class: No such file or directory

If you set executorUri.isDefined then it wont reach (this is what we have as well):
val executorSparkHome = conf.getOption("spark.mesos.executor.home")
.orElse(sc.getSparkHome())
.getOrElse {
throw new SparkException("Executor Spark home spark.mesos.executor.home is not set!")
}
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala

Yes, agree with @skonto. Why still honor SPARK_HOME, a little strange.

Its because Mesos Dispatcher passed that env variable to the driver at line:
builder.setEnvironment(envBuilder.build())
In 1.6 the spark-submit script has changed...
https://github.com/apache/spark/blob/master/bin/spark-submit
This line make SPARK_HOME not to change to the local path on mesos slave because it has already been set.
if [ -z "${SPARK_HOME}" ]; then
export SPARK_HOME="$(cd "dirname "$0""/..; pwd)"
fi

Ahhh, I see. The problem is that we expose SPARK_HOME in all the shell scripts.

I'm not sure it's that. It's that SPARK_HOME is copied by the mesos infrastructure to other machines. I think this in inherently local. Still, not-exporting SPARK_HOME would be a quick way to test whether this theory is true.


Revert "[stable/fluentd] Fix missing serviceName in HPA"

Reverts helm/charts#21202
The serviceName had to be added on the statefulset resource, not the HPA.
Since it has been done on another PR, we only need to revert this breaking change.

/assign @Miouge1

Re: https://github.com/helm/charts/pull/21202
I'm also getting
unknown field "serviceName" in io.k8s.api.autoscaling.v2beta2.CrossVersionObjectReference

for v2.4.1. Could we get this merged please?

/ok-to-test

/lgtm


How do I use PODE behind a (NGINX) Reverse Proxy with SSL?

I try to use PODE behind an NGINX Reverse Proxy and struggle to get this scenario working as expected.
When I use HTTP (port 80) on the Proxy, everything works fine, as soon as I use HTTPS (port 443) the behaviour changes and stuff is not working as expected.
In this case, I try to use a login (relative simple, based on some of the examples).

When access via https
<IP_ADDRESS> - - [31/Jan/2021:00:31:12 +01:00] "POST /login HTTP/1.0" 302 - "-" "-"
<IP_ADDRESS> - - [31/Jan/2021:00:31:12 +01:00] "GET /login HTTP/1.0" 200 3572 "-" "-"

This request not, it will simply reload the login (/login) page.
The reverse Proxy config is simple:
server {
   listen 80;
   listen [::]:80;
   listen 443 ssl http2;
   listen [::]:443 ssl http2;

   server_name test.hochwald.net;

   location / {
      add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;
      add_header Referrer-Policy "no-referrer, strict-origin-when-cross-origin";
      add_header X-Frame-Options SAMEORIGIN always;
      add_header X-Content-Type-Options nosniff always;
      add_header X-XSS-Protection "1; mode=block" always;
      proxy_set_header Host $http_host;
      proxy_set_header X-Forwarded-Host $host;
      proxy_set_header X-Forwarded-Server $host;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_hide_header 'Cache-Control';
      proxy_pass http://<IP_ADDRESS>:8080;
      proxy_redirect http://<IP_ADDRESS>:8080 https://vpn.hochwald.net;
   }
}

I also removed the proxy_redirect, that doesn't change the issue I have. PODE is running on port 8080 in both cases. I also reused a working SSL configuration (different port and Https as Protocol), with "-SelfSigned" and a trusted certificate, no difference.
I also tried to terminate SSL on the NGINX, makes no difference. I also removed most of the security stuff above (add_header), same result.
In general the NGINX setup works fine for several other scenarios and backends, with and without SSL. And it also works fine with plain HTTP traffic.
Any idea how to set this up, or what I did wrong?

Just to describe my setup a bit more detailed:
The reverse Proxy runs on a Debit 10 system, I use PowerShell (Core) 7.1.1 on the same system (for this test), and Pode from the Gallery.
I also tried a Remote Pode (on Windows 10, with the same PowerShell version), that didn't change.
Pode runs as Daemon, but I also tried to run it as user (in the console).
The basic example I use for this test is web-auth-form.ps1 (New-PodeAuthScheme -Form), with a simple HTML Form served by Pode.

Just to describe my setup a bit more detailed:
The reverse Proxy runs on a Debit 10 system, I use PowerShell (Core) 7.1.1 on the same system (for this test), and Pode from the Gallery.
I also tried a Remote Pode (on Windows 10, with the same PowerShell version), that didn't change.
Pode runs as Daemon, but I also tried to run it as user (in the console).
The basic example I use for this test is web-auth-form.ps1 (New-PodeAuthScheme -Form), with a simple HTML Form served by Pode.

Try building the Issue-655 branch and using it.
I tested NGINX with the same configuration on it and it works for me.

Try building the Issue-655 branch and using it.
I tested NGINX with the same configuration on it and it works for me.

Try building the Issue-655 branch and using it.
I tested NGINX with the same configuration on it and it works for me.
Hi @ili101,

thanks for the hint, the build was fine, but it didn't change the behaviour:
Pode Console:
Listening on the following 1 endpoint(s) [1 thread(s)]:
	- http://localhost:8080/
<IP_ADDRESS> - - [31/Jan/2021:16:09:13 +01:00] "GET /login HTTP/1.1" 200 3572 "-" "-"
<IP_ADDRESS> - - [31/Jan/2021:16:09:40 +01:00] "POST /login HTTP/1.1" 302 - "-" "-"
<IP_ADDRESS> - - [31/Jan/2021:16:09:40 +01:00] "GET /login HTTP/1.1" 200 3572 "-" "-"

The request will still not work as it should (the forma data seems to be missing and the request data (header) is marked as "-". With HTTP (instead if HTTPS) it still works just fine.
I have the latest NET SDK installed in that system:
.NET SDK (reflecting any global.json):
 Version:   5.0.102
 Commit:    71365b4d42

Runtime Environment:
 OS Name:     debian
 OS Version:  10
 OS Platform: Linux
 RID:         debian.10-x64
 Base Path:   /usr/share/dotnet/sdk/5.0.102/

Host (useful for support):
  Version: 5.0.2
  Commit:  cb5f173b96

.NET SDKs installed:
  5.0.102 [/usr/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 5.0.2 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 5.0.2 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

And the build was easy:
Build Build /home/josh/Pode-Issue-655/pode.build.ps1
Task /Build/BuildDeps
Done /Build/BuildDeps 00:00:00.0076283
Task /Build
Microsoft (R) Build Engine version 16.8.3+39993bd9d for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  All projects are up-to-date for restore.
  Pode -> /home/josh/Pode-Issue-655/src/Listener/bin/Release/netstandard2.0/Pode.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:00.91
Microsoft (R) Build Engine version 16.8.3+39993bd9d for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  All projects are up-to-date for restore.
  Pode -> /home/josh/Pode-Issue-655/src/Listener/bin/Release/netstandard2.0/Pode.dll
  Pode -> /home/josh/Pode-Issue-655/src/Libs/
Done /Build 00:00:02.7843523
Build succeeded. 2 tasks, 0 errors, 0 warnings 00:00:02.8146155

Try building the Issue-655 branch and using it.
I tested NGINX with the same configuration on it and it works for me.
Hi @ili101,

thanks for the hint, the build was fine, but it didn't change the behaviour:
Pode Console:
Listening on the following 1 endpoint(s) [1 thread(s)]:
	- http://localhost:8080/
<IP_ADDRESS> - - [31/Jan/2021:16:09:13 +01:00] "GET /login HTTP/1.1" 200 3572 "-" "-"
<IP_ADDRESS> - - [31/Jan/2021:16:09:40 +01:00] "POST /login HTTP/1.1" 302 - "-" "-"
<IP_ADDRESS> - - [31/Jan/2021:16:09:40 +01:00] "GET /login HTTP/1.1" 200 3572 "-" "-"

The request will still not work as it should (the forma data seems to be missing and the request data (header) is marked as "-". With HTTP (instead if HTTPS) it still works just fine.
I have the latest NET SDK installed in that system:
.NET SDK (reflecting any global.json):
 Version:   5.0.102
 Commit:    71365b4d42

Runtime Environment:
 OS Name:     debian
 OS Version:  10
 OS Platform: Linux
 RID:         debian.10-x64
 Base Path:   /usr/share/dotnet/sdk/5.0.102/

Host (useful for support):
  Version: 5.0.2
  Commit:  cb5f173b96

.NET SDKs installed:
  5.0.102 [/usr/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 5.0.2 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 5.0.2 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

And the build was easy:
Build Build /home/josh/Pode-Issue-655/pode.build.ps1
Task /Build/BuildDeps
Done /Build/BuildDeps 00:00:00.0076283
Task /Build
Microsoft (R) Build Engine version 16.8.3+39993bd9d for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  All projects are up-to-date for restore.
  Pode -> /home/josh/Pode-Issue-655/src/Listener/bin/Release/netstandard2.0/Pode.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:00.91
Microsoft (R) Build Engine version 16.8.3+39993bd9d for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  All projects are up-to-date for restore.
  Pode -> /home/josh/Pode-Issue-655/src/Listener/bin/Release/netstandard2.0/Pode.dll
  Pode -> /home/josh/Pode-Issue-655/src/Libs/
Done /Build 00:00:02.7843523
Build succeeded. 2 tasks, 0 errors, 0 warnings 00:00:02.8146155

Did another Build with .NET SDK Version 3.1.405, that didn't change it.
Just to ensure its not related to the latest .NET

Did another Build with .NET SDK Version 3.1.405, that didn't change it.
Just to ensure its not related to the latest .NET

Tested again, looks like the specific fix is actually in develop branch and Issue-655 branch is a little out of date, it worked for me on Issue-655 because I updated it before using it:
git fetch origin
git checkout Issue-655
git merge origin/develop
Invoke-Build Build

Tested again, looks like the specific fix is actually in develop branch and Issue-655 branch is a little out of date, it worked for me on Issue-655 because I updated it before using it:
git fetch origin
git checkout Issue-655
git merge origin/develop
Invoke-Build Build

@ili101 and you did that with .NET core 3.1, right?

@ili101 and you did that with .NET core 3.1, right?

Yes

Yes

After following the steps that @ili101 provided, it woks as expected:
<IP_ADDRESS> - - [31/Jan/2021:21:05:55 +01:00] "GET /login HTTP/1.1" 200 3572 "https://test.hochwald.net/login" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Mobile/15E148 Safari/604.1"
<IP_ADDRESS> - - [31/Jan/2021:21:06:08 +01:00] "POST /login HTTP/1.1" 302 - "https://test.hochwald.net/login" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Mobile/15E148 Safari/604.1"
<IP_ADDRESS> - - [31/Jan/2021:21:06:08 +01:00] "GET /login HTTP/1.1" 200 3675 "https://test.hochwald.net/login" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Mobile/15E148 Safari/604.1"

I also see the complete headers in the request and the login works just fine!
Interesting... I tried Issue-655 and develop before, but never merged it ;-)
Thanks you @ili101

After following the steps that @ili101 provided, it woks as expected:
<IP_ADDRESS> - - [31/Jan/2021:21:05:55 +01:00] "GET /login HTTP/1.1" 200 3572 "https://test.hochwald.net/login" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Mobile/15E148 Safari/604.1"
<IP_ADDRESS> - - [31/Jan/2021:21:06:08 +01:00] "POST /login HTTP/1.1" 302 - "https://test.hochwald.net/login" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Mobile/15E148 Safari/604.1"
<IP_ADDRESS> - - [31/Jan/2021:21:06:08 +01:00] "GET /login HTTP/1.1" 200 3675 "https://test.hochwald.net/login" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Mobile/15E148 Safari/604.1"

I also see the complete headers in the request and the login works just fine!
Interesting... I tried Issue-655 and develop before, but never merged it ;-)
Thanks you @ili101


Theme breaking on Visual Studio

Visual Studio version: Microsoft Visual Studio Enterprise 2022 (64-bit) - Preview (Version 17.9.0 Preview 1.0)
Issue:

I loved this theme, but after some recent updates, this is not working for me. I hope this can be fixed and I can continue to enjoy it.

Hey @anime-shed thank you for finding value in the tool and using it!
I tried to replicate this issue, but installing the extension on VS Community and VS Enterprise (preview) but was unable to.
Visual Studio Community Preview

Visual Studio Enterprise Preview

Could you check what version of the extension you have installed? If it's 0.6.0, could you try reinstalling the extension to see if that solves the problem?
You can also make sure the extension gets cleared away by making sure the extension pkgdef file is truly deleted while reinstalling by deleting the folder directly.

Hi @johnyenter-briars, I tried to reinstall it after verifying that the pkgdef file is truly deleted and it still does not work.

Sorry for not sharing the detailed info about the VS version:

Seems like I found the culprit:

Since this was working with the alternate theme I was using, I thought this might be backwards compatible with this theme too. But seems like it is not, can you try to see what might be breaking cause for the same.

@johnyenter-briars I will be glad if this is fixed.
The vs extension I was talking about:
https://github.com/jekelly/gruvboxvs
https://marketplace.visualstudio.com/items?itemName=jeffkelly.gruvboxvs10

@anime-shed Haha - I know that theme well. The fact that it wasn't quite what I was looking for was what inspired me to make / publish this theme : )
So it looks like with the "UI Refresh" the categories that control those colors are:

I'm not sure what use they serve in VS 2022 non preview.
I was able to make some progress, if you're curious you can preview the changes / build / test the extension for yourself at branch: #14

@anime-shed  Ok, did some work on the theme - I'm not a UI designer by any means, but I tried to use different grey colors to maintain the visual feeling of 'layers' on the new UI buttons.

You wanna take a look at #14 when you get a chance? You should be able to check out that branch, and then do a local build / install in your VS 2022 Preview instance.

I'll do the same, and use it for the next week locally before I publish - just to see if there are any fixes needed.

Hi, I tried to build/run locally but for some reason I am not able to load the project

@anime-shed do you have the VS theme extension I kinked above?

Sorry I did not see the extension that I uploaded, instead I was trying to build that branch on my end. I found it, will start using it and will let you know if anything breaks.

I was able to build it after editing:
https://github.com/johnyenter-briars/gruvbox-material-vs/blob/master/gruvbox-material-vs/gruvbox-material-vs.csproj#L76
 <Import Project="$(VSToolsPath)\VSSDK\Microsoft.VsSDK.targets" />

@anime-shed  Ok right on! Since that seems like an enhancement to me - as opposed to a bug fix (or in this case supporting the preview VS version) - can you open a new issue for that? (Or maybe even a PR if you see fit? ;) )
I'd like to close this issue and #14  for now and get this out on the marketplace : )


Release v4

Problem
PR for v4 release

Removed MANIFEST.in - no longer required when using setuptools-scm
Updated coveralls in CI to use github action because cannot import name 'Reporter' from 'coverage.report' when coverage[toml] added.
Renamed django-import-export-ci.yml to test.yml (name better reflects purpose of workflow)

TODO

Remove pypi and testpypi tokens (and also from github repo) - no longer required.
Remove coveralls token (if there is one) no longer required for OSS ?

coverage: 98.688% (-1.3%) from 100.0% when pulling 362016a6d7a7eafe6f68fe760a1ab23c4419326b on release-4 into 2086267a5872afcd28a13c90ed158157ea6b1949 on main.

@matthewhegarty I don't know if you're looking for feedback on v4 yet, but in case you do: calling Widget.render(..., obj) with obj != None is deprecated, but Field.export() calls return self.widget.render(value, instance) with instance != None which triggers the deprecation warning.
Other than that I didn't notice any issues :)

Thank you - feedback very welcome - I will take a look


Configurable number of lines for the subject in the e-mail list

Checklist

[X] I have used the search function to see if someone else has already submitted the same feature request.
[X] I will describe the problem with as much detail as possible.
[X] This issue only contains a request for one single feature, not multiple (related) features.

App version
6.509
Problem you are trying to solve
In the e-mail list, if the subject is long, we cannot see it totally because it is truncated to stay in one line.
Some subject have this form : "[Context for filtering] Start of..." and for this kind of e-mail, understanding the subject is hard with few useful information displayed by k9.
Suggested solution
Like we have an option to configure the number of line for the content preview (Options > General > Display > Preview lines), it can be helpful to have the same kind of option for the subject.
Personally, I prefer to have the option to see the whole subject and few or no line of content.
Screenshots / Drawings / Technical details
No response

@Massedil i set (Options > General > Display > correspondent above subject) for that reason.


Handle AzureException during storage upload

High-level description
Backport #3930

(DCOS-40442) AzureException during storage upload

@mesosphere-mergebot merge-it


[$250] Expense - On changing 1st approver, 2nd approver is not greyed out&error for not added approvers

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 9.0.78-1
Reproducible in staging?: Yes
Reproducible in production?: Yes
Issue reported by: Applause Internal Team
Device used: Redmi note 10s Android 13
App Component: Money Requests
Action Performed:
Pre-condition: 1. Have a workspace with few members
2. Enable workflows and add approval flow

Go to https://staging.new.expensify.com/home
Go to workspace settings- workflows
Go offline
Tap add approvals
Tap additional approver
Select a user & save it [user1]
Save edit approval flow
Tap add approvals
Note second approver is greyed out [user1]
Tap first approver and select different member and save it [user2]
Tap add approvals and note first approver is greyed out and second approver not greyed out now
Tap additional approver and select a new user and save it [user3]
Tap add approvals and note first+third approver greyed out & second approver is not greyed out
Remove the third approver & second approver ( note now we have only additional approver row)
Change the first approver ( to second approver who was removed [user1] ) and save
Now note after first approver is changed, edit approval flow automatically shows second approver with user selected
Now tap additional approver (  set as first approver who was selected in step 10- [ user2]  )& save it
Note fourth and fifth approver automatically added and error is displayed.

Expected Result:
After changing 1st approver, 2nd approver must be same as it was before changing 1st approver & not added approvers must not be shown with error message.
Actual Result:
After changing 1st approver, 2nd approver is not greyed out & not added approvers are shown with error message.
Workaround:
Unknown
Platforms:

[ ] Android: Standalone
[ ] Android: HybridApp
[x] Android: mWeb Chrome
[ ] iOS: Standalone
[ ] iOS: HybridApp
[ ] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos
https://github.com/user-attachments/assets/7c5b369e-59c0-45fc-9a23-d8b17db13c60
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~021871540038795816648
Upwork Job ID:<PHONE_NUMBER>795816648
Last Price Increase: 2024-12-24

Issue OwnerCurrent Issue Owner: @getusha

Proposal
Please re-state the problem that we are trying to solve in this issue.
On changing 1st approver, 2nd approver is not greyed out&error for not added approvers
What is the root cause of that problem?
When we change an approver who has a forwardTo set to another approver, the pendingFields.forwardsTo of the previous approver is not updated to reflect the new approver. As a result, when we switch to a new approver, the approver being forwarded to will not be grayed out because the pendingFields.forwardsTo of the new approver is undefined.
Logic to getApprovalPendingAction
https://github.com/Expensify/App/blob/ab40264dff9a6c21372617c8744b1013c22dd2d2/src/pages/workspace/workflows/approvals/ApprovalWorkflowEditor.tsx#L48
We updated the approver but did not update the pendingFields.forwardsTo of the new approver
https://github.com/Expensify/App/blob/d14b37f3f0273166185c33fb2bdfd959e6811731/src/libs/actions/Workflow.ts#L242
What changes do you think we should make in order to solve the problem?

To resolve this issue, we need to update the pendingFields.forwardsTo for the new approver by implementing the following steps:

We need to retrieve the pendingFields.forwardsTo of the previous approver and update it for the new approver.

https://github.com/Expensify/App/blob/d14b37f3f0273166185c33fb2bdfd959e6811731/src/libs/actions/Workflow.ts#L249
    const approvers: Array<Approver | undefined> = [...currentApprovalWorkflow.approvers];
    const previousApprover = approvers.at(approverIndex)?.email;
    const pendingFieldsOfForwardsTo = previousApprover ? currentApprovalWorkflow.members.find((item) => item.email === previousApprover) : undefined;
    const newMembers = currentApprovalWorkflow.members.map((item) => {
        if (item.email === approver.email) {
            return {
                ...item,
                pendingFields: pendingFieldsOfForwardsTo?.pendingFields ?? item.pendingFields, // Fallback to item's original pendingFields if undefined
            };
        }

        if (item.email === approvers.at(approverIndex)?.email) {
            return {
                ...item,
                pendingFields: {
                    forwardsTo: null,
                },
            };
        }

        return item;
    });

Update Onyx

https://github.com/Expensify/App/blob/d14b37f3f0273166185c33fb2bdfd959e6811731/src/libs/actions/Workflow.ts#L280
Update to:
    Onyx.merge(ONYXKEYS.APPROVAL_WORKFLOW, {approvers: updatedApprovers, members: newMembers, errors});

Update member data in convertApprovalWorkflowToPolicyEmployees when updating updateApprovalWorkflow.

https://github.com/Expensify/App/blob/d14b37f3f0273166185c33fb2bdfd959e6811731/src/libs/WorkflowUtils.ts#L243-L260
Update to:
    approvalWorkflow.members.forEach(({email}) => {
        const submitsTo = type === CONST.APPROVAL_WORKFLOW.TYPE.REMOVE ? '' : firstApprover.email ?? '';

        // For every member, we check if the submitsTo field has changed.
        // If it has, we update the employee list with the new submitsTo value.
        if (previousEmployeeList[email]?.submitsTo === submitsTo) {
            return;
        }

        updatedEmployeeList[email] = {
            ...(updatedEmployeeList[email] ? updatedEmployeeList[email] : {email}),
            submitsTo,
            pendingAction,
            pendingFields: {
                submitsTo: pendingAction,
                forwardsTo: updatedEmployeeList[email]?.pendingFields?.forwardsTo,
            },
        };
    });

POC
https://github.com/user-attachments/assets/f8574499-0732-4b3d-95e9-42a023181b07

Test branch
What specific scenarios should we cover in automated tests to prevent reintroducing this issue in the future?

Add test case for members data:

Should update pendingFields for the current approver with the previous approver’s pendingFields
Should not update pendingFields when approverIndex is invalid

What alternative solutions did you explore? (Optional)
Reminder: Please use plain English, be brief and avoid jargon. Feel free to use images, charts or pseudo-code if necessary. Do not post large multi-line diffs or write walls of text. Do not create PRs unless you have been hired for this job.


Remove canTransitionFrom override from Material/CupertinoPageRoute

Overriding this method will cause the route transition animation to be lost, so remove this method
Description
On iOS, a MaterialPageRoute should slide out with CupertinoPageTransition when a compatible PageRoute is pushed on top of it.  Overriding canTransitionFrom method will cause the route transition animation to be lost when a CupertinoPageRoute push on another route. So I remove this method from material/pages.dart and cupertino/pages.dart.
Related Issues
ref：#44864
Tests
I added the following tests:
/packages/flutter/test/material/page_test.dart
Checklist
Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

[x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
[x] I signed the [CLA].
[x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement].
[ ] I updated/added relevant documentation (doc comments with ///).
[ ] All existing and new tests are passing.
[ ] The analyzer (flutter analyze --flutter-repo) does not report any problems on my PR.
[x] I am willing to follow-up on review comments in a timely manner.

Breaking Change
Does your PR require Flutter developers to manually update their apps to accommodate your change?

[ ] Yes, this is a breaking change (Please read [Handling breaking changes]). Replace this with a link to the e-mail where you asked for input on this proposed change.
[x] No, this is not a breaking change.

Thanks @AlexVincent525 and @LongCatIsLooong for helping me complete my test case！

@googlebot I signed it!

@LongCatIsLooong /cc


Radix Sort in java

🛠 Issue #321

Issue no #321 solved

✔ Check List

[x] My code follows the code style of this project.
[x] This PR does not contain plagiarized content.
[x] The title of my pull request is a short description of the requested changes.

📄 Note to reviewers
Added radix sort in java in sorting algorithms in java, time complexity of the algorithm is O(nk) where n is the number of elements in the array and k is the number of digits in the largest number in the given array.

📷 Screenshots

Updated the code now we can take user's input, added comments and explained and deleted the .class file


sioclient: unstable-2023-02-13 -> 3.1.0-unstable-2023-11-10

Description of changes
https://github.com/socketio/socket.io-client-cpp/commits/master/
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[x] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 278763 run on aarch64-linux 1

2 packages built:

freedv
sioclient

@nixpkgs-merge-bot merge


[Relay] GetItem

GetItem is a relay construct for getting field out of a tuple

@junrushao1994 @srkreddy1238 @masahi please review.

@junrushao1994 thx for reviewing!
@jroesch the relay doc/evaluator should be updated with tuplegetitem
@joshpoll the relay prettyprinter/parser/text format should be updated with tuplegetitem
I can take the responsibility of adding tuplegetitem if you two are busy - in that case I can make pr to you two's branch.


Fork in private repository

Hello,
The following structure should work (I'm missing some information in the docs)?

Root
composer.json has a private repository (a forked one)

The forked one
composer.json has a private asset repository (a forked one)

...
"require": {
  "bower-asset/my-repo": "1.0.0"
},
"extra": {
  "asset-directories": [
    {
      "type": "vcs",
      "url": "xxxx:repo/my-repo.git",
      "name": "bower-asset/my-repo",
    }
  ]
}

The repo my-repo is a published package, but I want my version of it.
Composer understands it and get my forked version, does this plugin do it too?

Yes you can use your forked repo. You must create and used a new branch for example.
{
    "require": {
        "bower-asset/my-repo": "dev-my-new-branch"
    },
    "extra": {
        "asset-directories": [
            {
                "type": "vcs",
                "url": "xxxx:repo/my-repo.git",
                "name": "bower-asset/my-repo",
            }
        ]
    }
}

Thanks for the response. Actually, I miss the bower-vcs and the information that composer doesn't resolve repositories recursively.


Translated Render & Commit Page

Updated the Render & Commit Page with Hindi Translations.
render-and-commit.md

@arshadkazmi42  Thank you, I have done the changes you have asked for till line 65, also  I made sure that whatever word changes or spelling changes you have suggested I have changed it globally throughout the file, awaiting your response for some doubts will commit once they are cleared.

@arshadkazmi42 updated the Render and Commit page with all the changes you mentioned. I think the file is perfect now. Awaiting your review.

@arshadkazmi42 Done with the changes you mentioned.
I think this pull request #179 is ready to be merged, awaiting your review.
Thank you.


Added the ability to set the host of the server

Options now allows .host to indicate the IP to listen on. See line 122 and here:
https://nodejs.org/api/http.html#http_server_listen_port_hostname_backlog_callback

thanks!.


Changes on CompleteDone not properly undone if new line inserted after completion

Text changes on the CompleteDone event may not be properly undone if new line is inserted after completion. Here's a command that reproduces the issue:
vim -Nu NONE \
    -c "autocmd CompleteDone * call append(0, ['prepend'])" \
    -c "call feedkeys(\"iline\<ESC>o\<C-X>\<C-L>2\<CR>\<ESC>u\", 't')"

The first part of the command creates a CompleteDone  autocommand that add the line prepend at the start of the buffer. The second part consists of the following steps:

enter insert mode, write line, and leave insert mode. The buffer looks like:
line

enter insert mode on the next line, start line completion (i_CTRL-X_CTRL-L), type 2, insert a newline, and leave insert mode. The buffer now looks like:
prepend
line
line2

finally, undo the changes. This results in:
line2

while the expected buffer is
line

In the CompleteDone autocommand, it should be call append(0, ['prepend']) instead of call append(1, ['prepend']) (the line must be added before line, not after). Updated the post.

Fixed by patch 8.1.0256


Generator Gadgets

This PR implements the concept of gadgets into Revizor's program generator. Here, a gadget refers to a small list of assembly instructions placed in a specific order for a specific reason. I've made this PR description somewhat lengthy to make sure I document my thought process in implementing this feature.
The need for Gadgets
Considering Spectre v1, the below gadget (in pseudocode), which features two memory load instructions, would be useful for causing an extra line in the CPU cache to be allocated during speculative execution.
LOAD REG1, [REG2] # REG1 is destination, REG2 is source address
LOAD REG3, [REG1] # REG1's result from previous load is used as the next load's address

Or consider another gadget, potentially useful for increasing the CPU's speculative execution window by creating a chain of data-dependent instructions:
MUL REG1, REG2, REG3
MUL REG4, REG1, REG3 # depends on REG1
MUL REG2, REG4, REG3 # depends on REG4
MUL REG3, REG2, REG1 # depends on REG2
# ...

Because Revizor's program generator randomly places assembly instructions, specific assembly sequences like these won't easily be generated. Gadgets make this possible.
Gadget File
At runtime, the user can specify the path to a gadget file through a config field (gadget_file). This file is formatted in JSON and uses the same syntax for specifying assembly instructions as Revizor's ISA specification parser. Within this file, the user can place interesting assembly gadgets he/she wants to see in Revizor's generated programs. Each gadget has a name and an optional weight (used to tune how often gadgets appear in generated programs). Here's an example of what one gadget might look like:
{
    "name": "double_subtraction",
    "weight": 0.5,
    "instructions":
    [
        {
            "name": "SUB", "category": "general", "control_flow": false,
            "operands": [
                {"dest": true, "src": false, "comment": "", "type_": "REG", "width": 32, "values": ["GPR", "SP"]},
                {"dest": false, "src": true, "comment": "", "type_": "REG", "width": 32, "values": ["GPR", "SP"]},
                {"dest": false, "src": true, "comment": "", "type_": "REG", "width": 32, "values": ["GPR"]}
            ],
            "implicit_operands": [
                {"type_": "FLAGS", "width": 0, "src": false, "dest": false, "values": ["w", "", "", "w", "w", "", "", "", "w"]}
            ]
        },
        {
            "name": "SUB", "category": "general", "control_flow": false,
            "operands": [
                {"dest": true, "src": false, "comment": "", "type_": "REG", "width": 32, "values": ["GPR", "SP"]},
                {"dest": false, "src": true, "comment": "", "type_": "REG", "width": 32, "values": ["GPR", "SP"]},
                {"dest": false, "src": true, "comment": "", "type_": "REG", "width": 32, "values": ["GPR"]}
            ],
            "implicit_operands": [
                {"type_": "FLAGS", "width": 0, "src": false, "dest": false, "values": ["w", "", "", "w", "w", "", "", "", "w"]}
            ]
        }
    ]
}

If the generator chooses to place this gadget into a program, three randomized SUB consecutive instructions will be placed within a basic block.
(I've created an example gadget file at src/arm64/tests/example_gadgets.json).
Gadget Selection
As the program generator fills basic blocks with instructions, it may choose to instead select a random gadget from the gadget file and place it into the program. This pseudocode describes the basic idea of how it decides:
while (program isn't full yet)
{
    bb = choose_random_basic_block();
    if (random_chance_to_use_gadget())
    {
        gadget = select_random_gadget_that_fits();
        bb.insert(gadget.instructions);
    }
    else
    {
        instruction = generate_random_instruction();
        bb.insert(instruction);
    }
}

The randomness of gadget placement is configurable using a few new config fields:

CONF.avg_gadgets_per_bb dictates, roughly, how many gadgets are selected and placed into each basic block of the test case.
CONF.max_gadgets_per_bb forces a maximum number of gadgets that can be placed into each basic block.

Gadget placement will also not occur if:

A gadget is too big (i.e. won't fit in CONF.program_size)
The generator is in the middle of placing two memory access instructions
A gadget would exceed the average number of memory accesses (CONF.avg_mem_accesses)

Hi @OleksiiOleksenko - have you had the chance to look at this? I had another idea that could be added to this PR; what do you think of this?
Currently I've implemented this gadget system to forbid any control-flow instructions. The motivation here was to prevent the disruption of your test case's DAG; if control-flow instructions were allowed in gadgets, the user could force more basic blocks to be added to the test case that aren't a part of the official DAG.
However, what do you think about allowing small, self-contained, loops to be present? A "loop gadget" might look like this:
MOV X0, #4
LOOP_GADGET_START:
SUBS X0, X0, #1
B.PL LOOP_GADGET_START

Something like this might allow for forced misprediction to occur, without creating infinite loops or doing anything unreasonable. If this isn't worth implementing as a gadget, perhaps implementing loops would be best as a separate feature of the program generator. (I might actually prefer the latter.) What do you think?

I'm also implementing a way to "alias" operands in a gadget, to allow for the same unique operand to be used in two different places in the same gadget. I'll push this up as soon as it's done, but for now, I'll mark the PR as a draft.


Loss of accuracy in weights of large Gauss-Jacobi rules

While using some large Gauss-Jacobi rules, I found that for highly singular integrands ($\alpha < -0.5$ or so), I'm losing digits in the resulting integrals. As a minimal example, consider
$$\int_{-1}^1 (1-x)^\alpha dx = \frac{2^{\alpha+1}}{\alpha+1}.$$
For a range of $n$'s, we approximate the above integral by summing the weights of an $n$-point Gauss-Jacobi rule.
using FastGaussQuadrature

a = -0.9
ns = 2 .^ (1:16)
I_true = 2^(a+1)/(a+1)

Is_quad = [sum(gaussjacobi(n, a, 0)[2]) for n in ns]
rel_errs = (I_true .- Is_quad) / I_true

which yields the following errors
n       relative error
----------------------
2       -1.657e-16
4       0.000e+00
8       0.000e+00
16      0.000e+00
32      0.000e+00
64      1.657e-16
128     -5.310e-13
256     7.170e-12
512     5.277e-12
1024    1.242e-11
2048    -1.904e-10
4096    1.459e-09
8192    2.030e-09
16384   -1.294e-08
32768   -4.663e-08
65536   6.886e-08

Is this expected behavior? Or perhaps related to this issue posted by @JamesCBremerJr?
Figure 4.5 in Hale and Townsend 2012 (on which I believe this routine is based?) shows 13 to 14 digits of accuracy in the weights for $\alpha = 2, \beta = -0.75$ up to $10^4$ nodes. But if we rerun the above code with $\alpha = -0.75$, we get only 10 digits in the integral for $n = 2^{16}$.
If we consider larger rules with $\alpha = -0.75$, we get only 8 digits for $n=2^{20}$. And if we take a harsher singularity, e.g. $\alpha = -0.99$, we get only 6 digits for $n=2^{16}$. So it looks like this loss of accuracy eventually (but long before any nodes coincide to machine precision) appears for any $\alpha < -0.5$, and becomes more severe as $\alpha \to -1$. Are such cases known limitations?

If we take the chebfun weights as ground truth (given that they integrate $f \equiv 1$ to 15 digits) and plot the relative error between the chebfun and FastGaussQuadrature.jl weights, we see that the error is highly concentrated near the endpoints. For example, with $\alpha = -0.99$ and $n = 2^{16},$ we have

Markers are sized and colored by error size for emphasis. This is a more exaggerated case of exactly what @JamesCBremerJr noted in the aforementioned issue.

The issue is that in the Newton iterations at gaussjacobi.jl:382, the recursive formula innerjacobi_rec is used instead of the boundary asymptotics, which are implemented as feval_asy2 in Chebfun. I now have a fork here with the low order terms in these asymptotics implemented. This gives us back 5 digits or so. See below.

However, to get 16 digit accuracy, we need the higher order terms. The higher order coefficients $A_m$ and $B_m$ in equation (3.24) of Hale and Townsend 2012 are computed numerically in Chebfun from recursive integro-differential formulae using Chebyshev interpolation, integration, and differentiation. But these routines are not readily available in FastGaussQuadrature.jl.
Do these Chebyshev routines exist elsewhere in JuliaApproximation? Or (at the risk of bloating this package) should we implement them here?

Thank you for the detailed comments!
PRs are always welcomed.

Do these Chebyshev routines exist elsewhere in JuliaApproximation?

I'm not much familiar with Chebyshev polynomials, but maybe FastChebInterp.jl helps here?

It should all be doable with ClassicalOrthogonalPolynomials.jl but that depends on this package./.

Thanks for your responses!
Unless I'm mistaken, it doesn't appear that Chebyshev integration is currently implemented in ClassicalOrthogonalPolynomials.jl. As an alternative, ApproxFunOrthogonalPolynomials.jl has both Chebyshev integration and differentiation implemented. However, it is similarly a circular dependency with this package...
In my view, part of the value of FastGaussQuadrature.jl is that it is extremely lightweight, with relatively few dependencies. I can use ApproxFunOrthogonalPolynomials.jl to solve this issue, but it's a heavy (not to mention circular) dependency, so from a software engineering perspective it doesn't seem like a great path forward. It's a hefty price to pay for some basic Chebyshev routines on $N = 10$ points.
The alternative is to use some lookups and reimplement very barebones routines for the necessary computations. This is the path I've taken in my fork. I've saved the 10-point Chebyshev integration and differentiation matrices as constants in gaussjacobi.jl, and reimplemented only the necessary barycentric interpolation routine. This is pretty painless and keeps the repository lightweight.
@dlfivefifty as a contributor to all the involved libraries, let me know if you think this is a reasonable solution. If so, I'll try to iron out the remaining numerical issues (all the existing package tests pass, but I still get ~11 digits on worst-case tests related to this issue, not 16) and submit a pull request.

Ah, I see - I was looking for an integrate-like function instead of a cumsum.
Your ChebyshevTransforms.jl solution makes good sense to me. Given my lack of familiarity with the relevant interfaces in JuliaApproximation.jl, I've submitted a pull request in the meantime which resolves this issue and #58. It uses saved integration and differentiation matrices from Chebfun as I mentioned above. It should be easy to substitute more elegant Chebyshev routines once the restructuring of the JuliaApproximation.jl package ecosystem that you suggest occurs. But I imagine you may want to engineer that yourself as the manager of those packages.

Fixed by https://github.com/JuliaApproximation/FastGaussQuadrature.jl/pull/131.


progressive step

Hi. I've added option called "progressive step" if it is true step will be calcualted automatically depending for value.
For example:
if (1 < value < 60 ) step = 1;
if (60 < value < 600) step = 10;
and so on
This is usefull for big values.

This would prevent setting some values. It's better to leave setting step to user.