add insert to DispatcherBuilder to not consume it

This makes it possible to store the dispatcher builder in another builder, and to pass it around a bit more easily to plugins.
I also snuck in a small iter_stages() function for debugging, I can pop that out and make a separate PR if you prefer.

Do you think it would be preferable to use only the insert methods going forward? If the add* method aren't being used and instead only the insert* methods would be used, we could replace the current functions with yours?

I think that add is nice; there's some very unergonomic sides to &mut Self as the return type of a builder. And for smaller projects it's just easier to .add(...).add(...) when you don't run multiple dispatchers or use too many systems.

I thought about this for a while. Can we rename .insert to .with and also add a test that uses chaining? :
DispatcherBuilder::new().with_system(...).with(...).build(&mut world)

Thanks :)

@jojolepro I guess it'd be add that goes to with? You can't chain insert into build since it returns &mut Self, which can't be sunk.

I think I'm missing the point. What is the purpose of .insert that isn't covered by .add ?

Currently, it is impossible to do something like:
mod audio {
   fn submit_clips(...) -> SystemResult { /* snip */ }

   pub fn add_functions(builder: &mut DispatcherBuilder) {
       builder.add(submit_clips);
   }
}

So that's why I need a function that doesn't consume self.
In theory, this case could work as:
mod audio {
   fn submit_clips(...) -> SystemResult { /* snip */ }

   pub fn add_functions(mut builder: DispatcherBuilder) -> DispatcherBuilder {
       builder.add(submit_clips);
   }
}

but I use multiple builders; like so:
struct RuntimeBuilder {
    begin_frame_builder: DispatcherBuilder,
    update: DispatcherBuilder,
    /* etc */
    component_types: Vec<Box<dyn ComponentDescriptor>>,
}

and then you need pass that whole thing around, all members need to be public to be able to destruct/construct etc.

I see. Would this do the trick for your use case? https://crates.io/crates/planck_ecs_bundle

Hmm. I could implement the trait for another builder structure and then inject them all in one go. But that's pretty much just replacing the dispatcher builder.

Making it public sounds like a great idea, then I can either keep it on my end or make an extension crate/repo (or make planck_ecs a workspace in a single repo?)

Best option would seem to have an extension crate, the same way plank_ecs_bundle adds to the base crate :)

Closing in favor of the other PR.


Services UI

Implement UI for Services. This will probably be broken down as the definition becomes clearer.
┆Issue is synchronized with this Jira Story
┆Issue Type: Story
┆Fix Versions: Dockstore 1.7
┆Sprint: Seabright Sprint 11 Kayak
┆Issue Number: DOCK-713
┆Epic: Dockstore  My Services

This should be broken out, but we need

https://dockstore.org/sitemap (services added here)
https://dockstore.org/services (for debugging, should re-use as much code as possible)
https://dockstore.org/search?_type=services&searchMode=files

So...for the sitemap, there is the somewhat hidden/deprecated /tools page that tools links to.  For services, should it just link to https://dockstore.org/search?_type=services&searchMode=files ?

I like the hidden page (using the old table) for debugging things actually (like if elastic search had an issue). Let's get that working for services

Done
Done
Punted

#2576 follows


Convert plugin to use Typescript and overall refactor

Tried to spread it out a bit and remove a lot of redundant code.  Should make it easier to maintain in the future.
Still need to do some more testing but hoping to get some thoughts.

Thanks for the PR! Unfortunately it won't have time to look at it until the weekend

Thanks, Updated review. Just using 2 space for the config to keep it more compact.


Remove unnecessary repetions on main file.

Although this is not the final step, I needed an easier way to make
changes to the auxiliary functions to make them library friendly.
I think the functionality of the script has not changed.

Hi @AlbertAlonso
The idea is good, but there are some issues I'm trying to fix (to.lower() --> tolower() ) and it seems that there is something wrong with the definition of wrk_files.
After that I'll compare the results with the old ones, if they are equal, I'll merge this branch.
FRA.

Good job!
After a couple of minor bugs, it is working and the results are the same as in the old code :)
FRA.


Modify /dataset POST method to perform INSERT or UPDATE of the DatasetRecord

LGTM

Request to merge the pull request to linkedin branch


Fix GSLIB tests

Running make test in miniapps/gslib fails for one of the tests:
    GSLIB miniapp [ mpirun -np 4 schwarz_ex1p ... ]: FAILED  (0.78s 11916kB)

with the message
Verification failed: (np_list.Sum() == num_procs) is false:
 -->  The individual mpi ranks for each of the meshes do not add up to the total ranks specified.
 ... in function: int main(int, char **)
 ... in file: schwarz_ex1p.cpp:163

Another small deficiency: the CMake file in miniapps/gslib does define any tests -- ideally, it will define the same tests as the makefile.

Discussion continued in @2945


Add 'insecure' option to enforce use of HTTP protocol for registry requests

#76

Thanks!
However, whatever I do to test this it does not have any effect. It still tries https. Moreover, it tries https and then http even w/o that option. May be because I am on mac...

That's a bit strange. Did you set registry.insecure to true when you were testing?

Yes. Can you reliably test this on your end?

Ok, the code is working despite it will still try https first no matter what in case it is available. Otherwise, http.

😂Strangely, it works quite well on my end. I previously tested it on Windows and Debian systems, and it worked as expected. I just tested it on an Intel CPU Mac: if registry.insecure is set to false, the program throws an exception 'http: server gave HTTP response to HTTPS client' and then exits. If set to true, it connects successfully.


[DROOLS-1150] fix infinite recursions and other issues found by FindBugs

+1, looks good. If all checks pass, could be merged.

Looks good, +1 if all checks pass.

Merged by https://github.com/droolsjbpm/drools/commit/a493b58a889cb71258d664dac011263394ddda5a


adl: init at 3.0.1

Description of changes
https://github.com/RaitaroH/adl

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Once again, this was largely based on the flake pieces of @hlissner (https://github.com/hlissner/dotfiles/blob/master/packages/adl.nix) and @danielphan2003 (https://github.com/danielphan2003/flk/blob/main/pkgs/applications/video/adl/default.nix).

P.S. This package is "pending" as a Draft until frece gets merged: #179939

./results/adl/bin/adl doesn't show anything.

Should be something like this: https://asciinema.org/a/0ZGkTdpYtk0dbihEKkcxxszvN
But I think for it to work you should have logged in with trackma. That's why in my example it got pre-populated with those animes.


cmd/link: paths in debug_line contain a mix of forward and backward slashes with Go 1.14beta1/Windows

What the title says. It used to be that debug_line uniformly used forward slashes / but now some paths have forward slashes / while others have backward slashes \.
This happens reliably when building with go build path_to_go_file.go, I'm not sure about other build modes.
Existing versions of Delve will have a problem with this but the fix is simple and I have already implemented it. However the behavior seems unintentional enough that I thought I'd report it.
I'm blaming cmd/link entirely without proof, it could be something else.
Since Windows doesn't have a handy utility to examine debug_line I wrote this script:
package main

import (
	"debug/dwarf"
	"debug/pe"
	"fmt"
	"io"
	"os"
)

func must(err error) {
	if err != nil {
		panic(err)
	}
}

func main() {
	exe, err := pe.Open(os.Args[1])
	must(err)
	dw, err := exe.DWARF()
	must(err)

	rdr := dw.Reader()

	seen := make(map[string]bool)

	for {
		e, err := rdr.Next()
		must(err)
		if e == nil {
			break
		}

		if e.Tag != dwarf.TagCompileUnit {
			continue
		}

		lnrdr, err := dw.LineReader(e)
		must(err)

		if lnrdr != nil {
			var lne dwarf.LineEntry
			for {
				err := lnrdr.Next(&lne)
				if err == io.EOF {
					break
				}
				must(err)

				if !seen[lne.File.Name] {
					seen[lne.File.Name] = true
					fmt.Printf("%s\n", lne.File.Name)
				}
			}
		}

		rdr.SkipChildren()
	}

}

@gopherbot please add label Debugging

Does the @gopherbot thing not work anymore?

CC @thanm @cherrymui @jeremyfaller

Thanks for the report.  Tentatively marking as a release blocker for 1.14.


TypeError Cannot read property 'freezeResults' of null

Version
v4.0.0-rc.5
Reproduction link
https://codesandbox.io/s/codesandboxnuxt-kdgpg?fontsize=14
Steps to reproduce
clean install - implement @nuxtjs/apollo as described

What is expected ?
apollo to be implemented
What is actually happening?
TypeError
Cannot read property 'freezeResults' of null

This bug report is available on Nuxt community (#c202)

For now, I'm not pretty sure what happen. freezeResults in apollo-cache-inmemory supposed to set to false by default.
The current solution, you can manual set freezeResults like this.
   apollo: {
    clientConfigs: {
      default: {
        httpEndpoint:
          "https://api.graph.cool/simple/v1/cj1dqiyvqqnmj0113yuqamkuu",
        inMemoryCacheOptions: {
          freezeResults: false
        }
      }
    }
  }

Fixed on new release version.


Fuse.Controls.CameraView needs ImageTools package

Should probably include Fuse.ImageTools as a transient package? Won't build without it.

The Fuse.ImageTools dependency is lingering from the old implementation. I think it can be removed all together


Preliminary multi-window support for Obsidian 0.15

This is a draft of multi-window support for Obsidian 0.15.1+.  It currently works (mostly) except for the interactjs parts, and I suspect some of the bounding box calculations need to be fixed for secondary windows as well.
Basically, with this PR you can get hover editors in all windows, but you can't move/resize/header-double-tap the ones in secondary windows.  Unlike in the main window, the cursor also doesn't change to show move/resize options.
I tried adding code to ensure interact.addDocument() is called for every window's document (and removeDocument() on shutdown), but that did not fix the issue.  I may see if I can figure something else out later, but if you have any ideas I'd love to hear them.

Thanks pjeby! I'll try and look at this shortly.

I got it working!  Had to create a separate interactStatic per window, and copy the default plugins to it.  There might be a cleaner way, but I dunno what it is.  I also haven't looked at the clipping calculations yet, but hover editors are now interactable in all windows with this version.  You probably have a better idea of what it's supposed to do, so have a look-see.
I'm pretty sure that one thing that's wrong right now is the calculation of the top -- I think it's subtracting the title bar size even though the area for secondary windows doesn't include the title bar in the first place.  There might be other issues.

After a bit more playing, I've identified two things that are broken in (unmaximized) secondary windows:

Resizing upward sometimes allows you to place the header slightly off-screen, or sometimes doesn't show the resize until you release
Snapping to the right doesn't work, even though snapping left does

I think, though I'm not sure, that this might actually be an issue with resizing?
I've checked in a change to do reflows when secondary windows resize, but it doesn't fix snapping to the right or sizing to the top when a secondary window isn't maximized.  A second change to fix getting the body size info from the wrong document, and that seems to have fixed right-side snapping.
So, basically, sometimes dragging the top to resize in a secondary window still acts funny, not sure why.  Especially since it doesn't happen every time.

One other weird issue: sometimes, if you hover a target in a secondary window, then move the mouse into the popover, leave it there for a few seconds and move it some more, the popover will disappear.  It's especially noticeable when the window is small enough that the popover overlaps the original target rectangle: in that case, just moving outside the target rectangle (while still over the poover) will close it.
I have no clue what to do with that one short of double-checking there aren't any more hidden references to the main document or window such that there are handlers being set up differently between the main and secondary windows.

Ok, I think this is done for now.  I've scoured the code and they're probably some dumb little thing, but given that both problems (top-resize and sometimes-early-disappearance) are intermittent and only happen in secondary windows, this is probably good enough to use as a beta for multi-window support, and shouldn't break for people on the standard Obsidian version.

Thank you for all of the work you did on this!
I'm trying to test this out and am seeing issues with "Open in new window" on a leaf spawning empty windows rather than loading the document from the target leaf. The exception thrown was n.matchParent is not a function.
On doing a hard reload, it's preventing app startup do to w.document.body.findAll is not a function:

I'm still looking into whether this is some weird interaction on my end but just seeing if you have any ideas


Incorporate ng e2e into the lab

Angular CLI now makes it easy to run Protractor and end-to-end (e2e) tests, and we should incorporate that into the lab. This would involve at least the following:

[x] Add reasonable e2e tests (that pass) for the user part of the lab so they have examples to work from.
[x] Add a gradle task to run the e2e tests (if there's not one that I'm just not aware of)
[ ] Add the e2e tests to the "run all tests" task in Gradle
[ ] Make sure those tests are run in Travis (so failing e2e tests should cause the build to fail)
[ ] Figure out how to include those in code coverage calculations
[x] Update the write-up to include some discussion of e2e tests

This is largely resolved in pull request #29. Thanks to @carav008 for doing a lot of good work on this!
I'm not sure we're running the e2e tests on Travis, though, so I don't know if they affect whether builds succeed.
We've determined that Protractor (which runs the E2E tests) doesn't provide code coverage, and they seem to see this as a feature not a bug, so I'm not sure we're ever getting that one item to work.

Currently, the e2e tests fail, so that needs to be resolved before we can call this finished.


Support new multi-objective API from Optuna v.2.4.0

Multi-objective optimization (e.g. specifying directions instead of direction in optuna.create_study) is becoming a first class citizen in Optuna. C.f. https://github.com/optuna/optuna/issues/1980. Some private APIs that we rely on here were updated during the course. This PR aims to fix those compatibility issues.
Let me know if I'm missing something since I'm not that familiar with the internals of Kurobako. Thanks.

Ready for review. I think these are the minimum changes required to support Optuna's upcoming v2.4.
Verified with the following toy example.

binh_and_korn_problem.py

from kurobako import problem

class BinhAndKornProblemFactory(problem.ProblemFactory):
    def specification(self):
        params = [
            problem.Var("x", problem.ContinuousRange(0, 5)),
            problem.Var("y", problem.ContinuousRange(0, 3)),
        ]
        return problem.ProblemSpec(
            name="Binh and Korn Function",
            params=params,
            values=[
                problem.Var("4 * x ** 2 + 4 * y ** 2"),
                problem.Var("(x - 5) ** 2 + (y - 5) ** 2"),
            ],
        )

    def create_problem(self, seed):
        return BinhAndKornProblem()

class BinhAndKornProblem(problem.Problem):
    def create_evaluator(self, params):
        return BinhAndKornEvaluator(params)

class BinhAndKornEvaluator(problem.Evaluator):
    def __init__(self, params):
        self._x, self._y = params
        self._current_step = 0

    def current_step(self):
        return self._current_step

    def evaluate(self, next_step):
        self._current_step = 1
        return [4 * self._x ** 2 + 4 * self._y ** 2, (self._x - 5) ** 2 + (self._y - 5) ** 2]

if __name__ == "__main__":
    runner = problem.ProblemRunner(BinhAndKornProblemFactory())
    runner.run()

random_solver.py  # Just Kurobako's official example.

from kurobako import problem
from kurobako import solver
from kurobako.solver.optuna import OptunaSolverFactory
import numpy as np

import optuna

class RandomSolverFactory(solver.SolverFactory):
    def specification(self):
        return solver.SolverSpec(name="Random Search")

    def create_solver(self, seed, problem):
        return RandomSolver(seed, problem)

class RandomSolver(solver.Solver):
    def __init__(self, seed, problem):
        self._rng = np.random.RandomState(seed)
        self._problem = problem

    def ask(self, idg):
        params = []
        for p in self._problem.params:
            if p.distribution == problem.Distribution.UNIFORM:
                params.append(self._rng.uniform(p.range.low, p.range.high))
            else:
                low = np.log(p.range.low)
                high = np.log(p.range.high)
                params.append(float(np.exp(self._rng.uniform(low, high))))

        trial_id = idg.generate()
        next_step = self._problem.last_step
        return solver.NextTrial(trial_id, params, next_step)

    def tell(self, trial):
        pass

if __name__ == "__main__":
    runner = solver.SolverRunner(RandomSolverFactory())
    runner.run()

botorch_solver.py

from kurobako import problem
from kurobako import solver
from kurobako.solver.optuna import OptunaSolverFactory
import numpy as np

import optuna

import argparse

parser = argparse.ArgumentParser()
parser.add_argument("--directions", nargs="+")

args = parser.parse_args()

def create_study(seed):
    sampler = optuna.integration.BoTorchSampler()
    return optuna.create_study(directions=args.directions, sampler=sampler)

if __name__ == "__main__":
    factory = OptunaSolverFactory(create_study)
    runner = solver.SolverRunner(factory)
    runner.run()

I think that I'll publish the next version of kurkobako-py once Optuna-v2.4 is released.
Please let me know if you need the new version soon.

Thanks for checking, it's not a hurry actually. With v2.4 sounds good, since this change depends on it. It should be released rather soon.


Fix file naming in the player module

And this is why all changes should go through review.

/botio reftest

/botio reftest

r=me, straight-forward renaming of files.


RigidJoint and RevoluteJoint don't align correctly

https://discord.com/channels/964330484911972403/1074840524181217452/1183060124353433641
b1 = Box(10, 10, 1)
b2 = Box(5, 5, 1)
j1 = RigidJoint("j1", b1, Pos(-4, -4, 0.5))
j2 = RevoluteJoint("j2", b2, Axis((2.5, 2.5, 0), (0, -1, 0)))

j1.connect_to(j2, angle=30)
show(b1, b2, render_joints=True)

Three issues I don't get:

Revolute axis ignored when adding an angle
Absolute positions not the same after connect_to
Wrong order of joints when I think about connecting like a carpenter does
Admittedly, the third one might be subjective, and is of minor importance. I can flip the order in my mind. But the first two are unclear to me

print(j1.parent.location * j1.relative_location)
print(j2.parent.location * j2.relative_axis.location)
# Location: (position=(-4.00, -4.00, 0.50), orientation=(-0.00, 0.00, -0.00))
# Location: (position=(-4.00, -4.00, 0.50), orientation=(-90.00, -60.00, -0.00))

I added to RevoluteJoint, LinearJoint, CylindricalJoint
    @property
    def absolute_location(self):
        """Absolute location of joint"""
        return self.parent.location * self.relative_axis.location

and to RigidJoint, `BallJoint``
    @property
    def absolute_location(self):
        """Absolute location of joint"""
        return self.parent.location * self.relative_location

and replaced _connect_to in class Joint by
    def _connect_to(self, other: Joint, **kwargs):  # pragma: no cover
        """Connect Joint self by repositioning other"""
        if kwargs.get("angle") is not None:
            transform = Rotation(0, 0, kwargs["angle"])
        else:
            transform = Location()

        self.parent.location = (
            other.absolute_location * transform * self.absolute_location.inverse()
        ) * self.parent.location
        self.connected_to = other

Then j2.connect_to(j1, angle=45) results in

and a following j2.connect_to(j1, angle=0) results in

What I'd like to demonstrate here is that working with the absolute locations for calculating the new location of j2 can be done very simply.
And since self.parent is relocated to the global origin by self.absolute_location.inverse() the angle rotation can be easily applied and respects the direction of the axis (always automatically the z-axis)

I think applying this pattern will both simplify and correct the joint handling.
Let's generalize the example:
loc1 = Location((1, 2, 3), (10, 20, 30))
loc2 = Location((16, 4, 0), (-30, -40, 50))

b1 = Box(10, 10, 1)
b2 = Box(5, 5, 1)
RigidJoint("j1", b1, Pos(-4, -4, 0.5))
RevoluteJoint("j2", b2, Axis((2.5, 2.5, 0), (0, -1, 0)))

b1 = loc1 * b1
b2 = loc2 * b2

show(b1, b2)

j2.connect_to(j1, angle=0) resuts in

Something is currently already broken, I think:

Shouldn't the arm's edge aligned with the base edge?

And a third issue is around LinearJoint, see https://discord.com/channels/@me/1083392680039555192/1185620998876303450

slider_arms are oriented the same in world coords, but not relative to the LinearJoint

For RigidJoint and RevoluteJoint alignment, I think the issue is in RevoluteJoint:relative_to, around below line where the transformation is calculated.
self.relative_axis.location * rotation * other.relative_location.inverse()
I had a fix as below. I tried several different manual tests locally and it seems working as expected. But I am new to build123d for like just 3 days, so please take a close look.
https://github.com/easonke/build123d/commit/389da70b70d9a47000fb40811f5a446ed5400d36

For reference: https://github.com/gumyr/build123d/tree/joints-v2, an attempt to fix these issues

I had been struggling with this problem for a while before coming across this issue, the joints-v2 branch fixed it for me. @bernhard-42, are there any plans to merge that branch?

I'm pretty sure that the Joints systems didn't have the current problems in the past and suffered from a change(s) that passes all of the tests (code coverage is at 100% IIRC) but doesn't cover all of the scenarios/math. Adding tests that currently fail is a high priority to me to ensure that the code stays functional going forward. So, if you have any scenarios that currently fail that could be a unittest, please share.
... and yes, I'm going to but this fix in my high priority list - so much to do.

@zackyancey I am kind of already waiting a bit for the joints stuff being looked at 😉
But having my own open source project, I totally understand the issue of time and priorities ...

Here's what I was doing:
from build123d import *

A = Box(6, 4, 2)
B = Cone(2, 1, 2)
jA = RevoluteJoint("jA", A, Axis((3, 0, 1), (0, -1, 0)))
jB = RigidJoint("jB", B, Location((-2, 0, -1), (90, 0, -90)))

A.joints["jA"].connect_to(B.joints["jB"], angle=30)

print(B.location)

# Lazy version of assertAlmostEqual:
assert str(B.location) == "Location: (position=(4.23, -0.00, 2.87), orientation=(0.00, -30.00, 0.00))"

Hoping for this

Got this

I'm pretty sure this is basically the same code that's in this comment, but bernhard-42's version also throws in an additional move of the objects before jointing them.
No worries, I get that there's a big todo list (brief appreciation aside since I've got the maintainers of two of my recent favorite open source projects here—build123d and ocp-vscode is basically my dream CAD environment, very quickly after finding this it's already easier to make complex (for me) parametric models than it ever was in F360/OpenSCAD. amazing work!)

I wrote up the failing case as a unit test, in case that's helpful (#613). I wasn't sure if I should put it on the joints branch or if there's a convention for expected failures, I can move/modify it if needed.

@gumyr Is there anything I can help to get the joints stuff prioritised (it is >5 months old in the meantime)?
The joints-v2 branch is fully functional, with all tests passing (I only tweaked some of them, where the joints logic was broken before). If you review it and let me know what to fix I am happy to work on it.

@bernhard-42 Sorry this is taking so long.  I've got a fix for this that passes all of the existing tests - instead of this (in RevoluteJoint.relative_to:
        rotation = Location(
            Plane(
                origin=(0, 0, 0),
                x_dir=self.angle_reference.rotate(self.relative_axis, angle),
                z_dir=self.relative_axis.direction,
            )
        )
        return (
            self.relative_axis.location * rotation * other.relative_location.inverse()
        )

this:
        return (
            self.relative_axis.location
            * Rotation(0, 0, angle)
            * other.relative_location.inverse()
        )

but I'd like to extend the tests to try to get better coverage - 100% coverage by line numbers doesn't verify that the math is correct necessarily.
Here is a slightly modified version of your example from the top:
b1 = Box(10, 10, 1)
b2 = extrude(Trapezoid(5, 5, 70, 90), 1)
j1 = RigidJoint("j1", b1, Pos(-4, -4, 0.5))
j2 = RevoluteJoint("j2", b2, Axis((2.5, 2.5, 0), (0, -1, 0)))

j1.connect_to(j2, angle=30)

Before connect_to:

After:

No worries, I was just  curious.
I would still recommend to look at my branch. I spent quite some time to simplify/harmonize the logic. And I took care that after joining the absolute location of both joints are identical. This was broken earlier => candidate for some tests?
Do you have a branch with your changes? I could try to break it and write further tests while doing so.

This test breaks with the current dev code:
    def test_revolute_joint_absolute_locations(self):
        b1 = Box(10, 10, 1)
        b2 = Box(5, 5, 1)
        j1 = RigidJoint("j1", b1, Location((-4, -4, 0.5)))
        j2 = RevoluteJoint("j2", b2, Axis((2.5, 2.5, 0), (0, -1, 0)))

        j1.connect_to(j2, angle=0)

        a1 = j1.parent.location * j1.relative_location
        a2 = j2.parent.location * j2.relative_axis.location

        self.assertEqual(a1, a2)

btw. it would be good if Joints would have a location property that allows to write the test like:
```python
    def test_revolute_joint_absolute_locations(self):
        b1 = Box(10, 10, 1)
        b2 = Box(5, 5, 1)
        j1 = RigidJoint("j1", b1, Location((-4, -4, 0.5)))
        j2 = RevoluteJoint("j2", b2, Axis((2.5, 2.5, 0), (0, -1, 0)))

        j1.connect_to(j2, angle=0)

        self.assertEqual(j1.location, j2.location)

@bernhard-42 I did look at your branch.  Thanks for all the work you've put into it but I was hoping to keep the existing design as I think it will be helpful for some of the future features I'm planning.  It's super useful to have as a reference though.
I like the idea of a Joint location and I'll add the test you've provided.

Your fix makes this test successful. However, I wanted to write a test with angle:
It has a bit of a surprising result:
a = Pos(0, 0, 3) * Box(1, 5, 5)

b1 = Box(10, 10, 1)
b2 = Box(5, 5, 1)
j1 = RigidJoint("j1", b1, Pos(-4, -4, 0.5))
j2 = RevoluteJoint("j2", b2, Axis((2.5, 2.5, 0), (0, -1, 0)))

angle = 30
j1.connect_to(j2, angle=angle)

a1 = j1.parent.location * j1.relative_location
a2 = j2.parent.location * j2.relative_axis.location

show(Rot(0, 0, angle) * a, b1, b2, render_joints=True)

returns

with angle = 0 they are oriented the same way.

So the rotation angle seems to behave differently for normal rotation (as expected counter clockwise) and when angle is provided

I like the idea of a Joint location and I'll add the test you've provided.
And I think the absolute location assertion should be added as a default to every joint test

This test also fails:
    def test_linear_revolute_joint(self):
        base = Box(10, 10, 2)
        arm = Box(2, 1, 2, align=(Align.CENTER, Align.CENTER, Align.MIN))

        base_top_edges = (
            base.edges().filter_by(Axis.X, tolerance=30).sort_by(Axis.Z)[-2:]
        )
        j1 = LinearJoint(
            "slot",
            base,
            axis=Edge.make_mid_way(*base_top_edges, 0.33).to_axis(),
            linear_range=(0, base_top_edges[0].length),
        )
        j2 = RevoluteJoint("pin", arm, axis=Axis.Z, angular_range=(0, 360))

        j1.connect_to(j2, position=6, angle=60)

        a1 = j1.parent.location * j1.relative_axis.location
        a2 = j2.parent.location * j2.relative_axis.location
        self.assertTupleAlmostEquals(list(a1.position), list(a2.position), 6)
        self.assertTupleAlmostEquals(list(a1.orientation), list(a2.orientation), 6)

btw. to be tolerant, I changed the assertion to assertTupleAlmostEqual
The main reason for thew failure is that the two joint axis are orthogonal after joining. While I understand that this is the result wanted, I feel it breaks the logic. I solved it by introducing a parameter called y_angle to rotate the aligned object around the y-axis. I think this is a bit of a design challenge ...

Another thing you could take over is to show the x-axis in the symbol. I found that really helpful

Your fix makes this test successful. However, I wanted to write a test with angle:
It has a bit of a surprising result:
a = Pos(0, 0, 3) * Box(1, 5, 5)

b1 = Box(10, 10, 1)
b2 = Box(5, 5, 1)
j1 = RigidJoint("j1", b1, Pos(-4, -4, 0.5))
j2 = RevoluteJoint("j2", b2, Axis((2.5, 2.5, 0), (0, -1, 0)))

angle = 30
j1.connect_to(j2, angle=angle)

a1 = j1.parent.location * j1.relative_location
a2 = j2.parent.location * j2.relative_axis.location

show(Rot(0, 0, angle) * a, b1, b2, render_joints=True)

returns
with angle = 0 they are oriented the same way.
So the rotation angle seems to behave differently for normal rotation (as expected counter clockwise) and when angle is provided

Interesting.  The Axis might be in two different directions here so the angle or rotation is backwards?  This is why I wanted more testing...

Another thing you could take over is to show the x-axis in the symbol. I found that really helpful

Excellent idea

def test_revolute_joint_absolute_locations(self):
        b1 = Box(10, 10, 1)
        b2 = Box(5, 5, 1)
        j1 = RigidJoint("j1", b1, Location((-4, -4, 0.5)))
        j2 = RevoluteJoint("j2", b2, Axis((2.5, 2.5, 0), (0, -1, 0)))

        j1.connect_to(j2, angle=0)

        self.assertEqual(j1.location, j2.location)

Got all the changes in (including adding a location property) but this test fails: AssertionError: (p=(-4.00, -4.00, 0.50), o=(-0.00, 0.00, -0.00)) != (p=(-4.00, -4.00, 0.50), o=(-0.00, 0.00, 0.00)) i.e -0.00 != 0.00 sigh .. now I have to look at location comparisons again :(

The issue is finally fixed.  In addition all the joints have a location property and I've enhanced the rotation based symbols to show the x direction.


PCA analysis doesn't look correct after upgrading

The PCs are no longer in order after upgrading to v.1.4.5.post1

Just to clarify, are you referring to 3 plots in the middle (PCA loading plots)? In new scanpy release, we render both positive and negative genes.

no, the variance ratio, first PC is PC21 and the last two are PC50 and PC20

@aopisco, I'm having some trouble reproducing your plots. Could you show us some more details about your environment?


Cannot use anything but wisp v0.1.0

I am completely new to gleam & wisp. I was trying to follow an article to setup my project for a web app. I did the following:
gleam new app
then
gleam add wisp lustre mist gleam_http gleam_erlang dot_env gleam_json
I then followed the article until I had to do  import wisp.{type Request, type Response} in router.gleam.
This resulted in the following:
  Compiling app

error: Unknown module field

  ┌─ C:\dev\FYP\app\src\app\router.gleam:3:14
  │
3 │ import wisp.{Request, Response}
  │              ^^^^^^^ Did you mean `main`?

The module `wisp` does not have a `Request` value.

error: Unknown module field
  ┌─ C:\dev\FYP\app\src\app\router.gleam:3:23
  │
3 │ import wisp.{Request, Response}
  │                       ^^^^^^^^ Did you mean `main`?

The module `wisp` does not have a `Response` value.

error: Unknown type
  ┌─ C:\dev\FYP\app\src\app\router.gleam:5:28
  │
5 │ pub fn handle_request(req: Request, ctx: Context) -> Response {
  │                            ^^^^^^^

The type `Request` is not defined or imported in this module.

I finally realized I was on version 0.1.0 but cannot get a higher version. gleam update doesn't do anything, running
gleam add wisp

  Resolving versions
      Added wisp v0.1.0

doesn't help and if I specify a version I get something like the following:
gleam add<EMAIL_ADDRESS>  Resolving versions
error: Dependency resolution failed

An error occurred while determining what dependency packages and
versions should be downloaded.
The error from the version resolver library was:

Unable to find compatible versions for the version constraints in your
gleam.toml. The conflicting packages are:

- app
- gleam_json
- wisp

This is the gleam.toml I am now trying to use:
name = "app"
version = "1.0.0"

[dependencies]
gleam_stdlib = "~> 0.34 or ~> 1.0"
wisp = ">= 0.1.0 and < 1.0.0"
gleam_erlang = ">= 0.25.0 and < 1.0.0"
mist = ">= 1.2.0 and < 2.0.0"
dot_env = ">= 1.1.0 and < 2.0.0"
lustre = ">= 4.3.0 and < 5.0.0"
gleam_json = ">= 2.0.0 and < 3.0.0"
gleam_http = ">= 3.6.0 and < 4.0.0"

[dev-dependencies]
gleeunit = "~> 1.0"

I don't know if I am just making a simple mistake or doing something completely wrong but I can't seem to get a version other than 0.1.0

to quickly add, my gleam.toml was
[dependencies]
gleam_stdlib = "~> 0.34 or ~> 1.0"
wisp = "~> 0.14"
gleam_erlang = "~> 0.25"
mist = "~> 1.0"
dot_env = "~> 0.5"
lustre = "~> 4.1"
gleam_json = ">= 1.0.0 and < 2.0.0"
gleam_http = ">= 3.6.0 and < 4.0.0"

[dev-dependencies]
gleeunit = "~> 1.0"

but then it changes to what I previously commented when I run gleam add

For some reasons I've not determined the instructions in that guide no longer work. I believe this may be due to one of the packages being out of date and conflicting with the others. Add them individually with Wisp first and see how that goes.

Wisp doesn't support gleam_json v2, you'll need to use the v1 line.

Thank you!


What is download_client.rb and meeting_id.mp4 in meeting_id directories

Hello,
First of all I should say thanks for this work which is highly appreciated:)
My question is

what is the role of file download_client.rb and is that necessary to place it somewhere?
I mean I cloned this git repo in my server in a path like /root/, and only moved the files export_presentation.rb and lib/interval_tree.rb to mentioned directories with executive rights, and since the day I did it's making mp4 files in /var/bigbluebutton/published/presentation/meeting_id/meeting.mp4 with no issue.
Is that all? Or I should do any other things?
I also see many of these when I run this command:

find . -name '*.mp4' -exec ls -lh {} \;
-rw-r--r-- 1 bigbluebutton bigbluebutton 210M Aug 12 09:45 ./5b47d382b32ad48b12492577000a574c154e0465-1628745881813/meeting.mp4
-rw-r--r-- 1 bigbluebutton bigbluebutton 185M Aug 12 07:16 ./5b47d382b32ad48b12492577000a574c154e0465-1628745881813/5b47d382b32ad48b12492577000a574c154e0465-1628745881813.mp4
-rw-r--r-- 1 bigbluebutton bigbluebutton 194M Aug  8 13:00 ./4f3eb13ed936e91b407524f56b6c8b52640c690a-1628406180177/meeting.mp4
-rw-r--r-- 1 bigbluebutton bigbluebutton 175M Aug  8 10:58 ./4f3eb13ed936e91b407524f56b6c8b52640c690a-1628406180177/4f3eb13ed936e91b407524f56b6c8b52640c690a-1628406180177.mp4
-rw-r--r-- 1 bigbluebutton bigbluebutton 178M Aug  7 16:10 ./c865285389c7c51b6f12652d054faa21be005222-1628341197035/meeting.mp4
-rw-r--r-- 1 bigbluebutton bigbluebutton 155M Aug  7 14:40 ./c865285389c7c51b6f12652d054faa21be005222-1628341197035/c865285389c7c51b6f12652d054faa21be005222-1628341197035.mp4
-rw-r--r-- 1 bigbluebutton bigbluebutton 234M Aug  7 08:36 ./cf10e9a265eb2b487e47f61f4d5d83ab3bf44494-1628314280877/meeting.mp4
-rw-r--r-- 1 bigbluebutton bigbluebutton 207M Aug  7 06:51 ./cf10e9a265eb2b487e47f61f4d5d83ab3bf44494-1628314280877/cf10e9a265eb2b487e47f61f4d5d83ab3bf44494-1628314280877.mp4

I do not mean meeting.mp4 files, but I do mean /var/bigbluebutton/published/presentation/meeting_id/meeting_id.mp4. What are these files generated by bbb-recording-exported script?
Thanks in advance

Hi @SaeedFazlollahzadeh,
The download_client.rb file was accidentaly pushed to the main branch. It's only needed for the client-side version of the script. Thanks for mentioning it, it's now gone.
As for the other files, they do not look like something my script would create. Did you try any other exporting scripts like tilmanmoser's bbb-video-download? I believe his script outputs files in that format.
If not, could you please provide some more information, such as BBB's version and so on?
Regards,
Daniel

Hi @SaeedFazlollahzadeh,
The download_client.rb file was accidentaly pushed to the main branch. It's only needed for the client-side version of the script. Thanks for mentioning it, it's now gone.
As for the other files, they do not look like something my script would create. Did you try any other exporting scripts like tilmanmoser's bbb-video-download? I believe his script outputs files in that format.
If not, could you please provide some more information, such as BBB's version and so on?
Regards,
Daniel

Hi @danielpetri1 and thanks for the reply,
What is the client-side version? Is that something before existed and it's gone now?
I use tilmanmoser's bbb-video-download in another server, but it only generates video.mp4 and yours generates meeting.mp4. But in this server I only user your script, and I'm not sure why, but I will investigate more for these files.
Regarding information, this server uses BBB 2.2 with Ubuntu 16. More info needed?
Thanks,
Saeed

@SaeedFazlollahzadeh the client-side version is on another branch of this repo. It's meant for people that do not have access to a server, such as students wanting to download lectures they attended.
Since you're on Ubuntu 2.2, you need to set REMOVE_REDUNDANT_SHAPES to true in the script. Otherwise, the annotations will look weird in your videos.
Regards
Daniel

@danielpetri1 could you please make and example of annotations looking weird? There are many videos and I did not see all of them, but I just saw some of them and I did not find any strange things.

@SaeedFazlollahzadeh Here's an example.


Implement Rosetta API Tests

Create a test framework where upon we could easily test our rosetta implementation.

Time Estimate: 1 hour


Confirmation email link fails

Should have opened this on marketing site https://github.com/joshmn/caffeinate-marketing


Simple fix to resolve byte arrays being slammed into array groupings.

Very basic pull request regarding #1431

Thanks for PR! I submitted this one instead https://github.com/querydsl/querydsl/pull/1440 which should solve the problem in a more general fashion.


Fix support for downloading to subfolder

This is a proposed fix for https://github.com/alainbryden/bitburner-scripts/issues/23 . It modifies git-pull.js to rewrite files as they are downloaded, updating path references to include the optional subfolder. This includes references to 'helper.js', as well as all the tools created in daemon.js.
I've had to include a workaround for what I believe is a bug in bitburner - importing a path relative to '.' doesn't seem to work. So the rewrite step in git-pull.js changes that to an absolute path.

Note this also fixes https://github.com/alainbryden/bitburner-scripts/issues/30 with the introduction of pathJoin.

Added a couple small fixes that I hadn't noticed weren't uploaded to github (they were on my local bitburner scripts only).

Almost there, I think there's a couple other spots where we'll need to add a const subfolder to be procedurally updated to avoid breaking paths:

https://github.com/alainbryden/bitburner-scripts/blob/b2d716a8265c39ddae96872854f7e080b4a29307/Tasks/contractor.js#L2
https://github.com/alainbryden/bitburner-scripts/blob/b2d716a8265c39ddae96872854f7e080b4a29307/crime.js#L2

OK, I moved the 'const subfolder' munging to helpers.js, with a new helper getFilePath to be used when procedurally generating paths. I did a grep for '.js' and found a couple paths I had missed. I also took your suggestion for handling generic imports. Hopefully that's everything - thanks for your help!

Thank YOU. Good final iterations, much more contained.


[HOLD for payment 2023-03-09] [$1000] GIF attachments are either displayed as blank or there is infinite loading on attachment preview

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Open the app and login with user A
Open user B report
Send any gif
Open the app in android and login with user B
Open user A report
Click on gif to preview the gif

Expected Result:
GIF should be displayed normally on preview screen
Actual Result:
Most of times, blank screen or infinite loading is displayed for GIF preview
Workaround:
unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native

Version Number: 1.2.74-0
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos:
https://user-images.githubusercontent.com/43996225/219974498-e8b1824e-211a-4f33-b819-68bb86d57740.mp4
https://user-images.githubusercontent.com/43996225/219974505-241bfd84-2145-4f86-bcfd-d46edbc49ea3.mp4
Expensify/Expensify Issue URL:
Issue reported by: @dhanashree-sawant
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1676741569824349
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~018f1c677402b17651
Upwork Job ID:<PHONE_NUMBER>837296640
Last Price Increase: 2023-02-21

Triggered auto assignment to @davidcardoza (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

Bug0 Triage Checklist (Main S/O)

[ ] This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
[ ] This bug is not a duplicate report (check E/App issues and #expensify-bugs)

If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead

[ ] This bug is reproducible using the reproduction steps in the OP. S/O

If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
If the reproduction steps aren't clear and you determine the correct steps, please update the OP.

[ ] This issue is filled out as thoroughly and clearly as possible

Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.

[ ] I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync

cc @Beamanator @trjExpensify Might want to fold this into image iprovements.

Hm yeah, is this not a dupe? https://github.com/Expensify/App/issues/14751 the PR was approved today?

Not quite dupes, https://github.com/Expensify/App/issues/14751 is only about uploading a gif as an avatar, not a chat attachment

Gotcha, okay. This is inconsistently reproducible on browserstack:

I asked @Julesssss to try on a physical Android device for good measure, and he's equally getting inconsistent results.
Seems like something we should investigate further. @thomas-coldwell you might have an idea from the image caching improvements on native perhaps?

Job added to Upwork: https://www.upwork.com/jobs/~018f1c677402b17651

Current assignee @davidcardoza is eligible for the External assigner, not assigning anyone new.

Triggered auto assignment to Contributor-plus team member for initial proposal review - @Santhosh-Sellavel (External)

Triggered auto assignment to @bondydaa (External), see https://stackoverflow.com/c/expensify/questions/7972 for more details.

Proposal
Please re-state the problem that we are trying to solve in this issue.
GIF attachments are either displayed as blank or there is infinite loading on attachment preview
What is the root cause of that problem?
on ImageView there is a failure of calc of the zoom sizes due to undefined state of height at the moment of execution.
this height is set onLayout of super view of this component and not happening before the images is loaded (depending of the speed of image loading), those the image is stuck loading/not appearing, on dev it showing quite a bug.
What changes do you think we should make in order to solve the problem?
I am proposing to ether use promises or useEffect to wait for two initial conditions to happen in ImageView native,

basic load of image (to get width, height)
layout event to get height of container
after those variables reached we can run a calc and the image will show

What alternative solutions did you explore? (Optional)
none

Contributor details
Your Expensify account email<EMAIL_ADDRESS>Upwork Profile Link: (https://www.upwork.com/freelancers/michaelchasovin)

✅ Contributor details stored successfully. Thank you for contributing to Expensify!

Proposal
Please re-state the problem that we are trying to solve in this issue.
Image attachment modal show infinite loading when we open a GIF attachment.
What is the root cause of that problem?
We are using react native fast image component to show the image and the loading indicator will be shown when onLoadStart is called.
https://github.com/Expensify/App/blob/9d04b46456253955c96981db9ef38245953baf18/src/components/ImageView/index.native.js#L173-L189
https://github.com/Expensify/App/blob/9d04b46456253955c96981db9ef38245953baf18/src/components/ImageView/index.native.js#L82-L84
The loading indicator will be hidden after the onLoad is called.
https://github.com/Expensify/App/blob/9d04b46456253955c96981db9ef38245953baf18/src/components/ImageView/index.native.js#L93-L115
However, from what I tested, onLoadStart is not always called before onLoad. Sometimes, onLoad is called before onLoadStart. So, we are setting the isLoading to false and then to true afterwards, which make the loading indicator shown indefinitely. This issue also happens to any type of image, but it happens more often to GIF image.
What changes do you think we should make in order to solve the problem?

Before I explain the solution for this issue, currently, every time I open an image attachment modal, I always get an error that is caused by undefined this.state.containerHeight but configureImageZoom is being called which use that state.
https://github.com/Expensify/App/blob/9d04b46456253955c96981db9ef38245953baf18/src/components/ImageView/index.native.js#L99
We can simply solve this by calling the configureImageZoom after both this.state.containerHeight and image size from onLoad is ready.
Now, the solution for this issue is to not set the loading to false inside configureImageZoom and set it after onLoadEnd (a new props to be added to RN fast image) is called instead. onLoadEnd will always be called after onLoadStart. We can also set the initial state of isLoading to true to avoid image blinking.

I'll just unassign myself I think I will not be able to get to this sooner, thanks! @bondydaa @davidcardoza Can you reapply external label here to get a new C+ or assign @s77rt If they are willing to take it as C+

📣 @Santhosh-Sellavel! 📣
Hey, it seems we don’t have your contributor details yet! You'll only have to do this once, and this is how we'll hire you on Upwork.
Please follow these steps:

Get the email address used to login to your Expensify account. If you don't already have an Expensify account, create one here. If you have multiple accounts (e.g. one for testing), please use your main account email.
Get the link to your Upwork profile. It's necessary because we only pay via Upwork. You can access it by logging in, and then clicking on your name. It'll look like this. If you don't already have an account, sign up for one here.
Copy the format below and paste it in a comment on this issue. Replace the placeholder text with your actual details.

Format:
Contributor details
Your Expensify account email: <REPLACE EMAIL HERE>
Upwork Profile Link: <REPLACE LINK HERE>

@bondydaa @davidcardoza I'd be happy to take over this issue.
Thanks! @Santhosh-Sellavel

📣 @s77rt You have been assigned to this job by @davidcardoza!
Please apply to this job in Upwork and leave a comment on the Github issue letting us know when we can expect a PR to be ready for review 🧑💻
Keep in mind: Code of Conduct | Contributing 📖

Great @s77rt you’re assigned to the issue.

@Litande @bernhardoj Thank you both for your proposals. Please give me sometime to learn the issue, will get back to you asap.

Quick Update: I think @bernhardoj is correct about onLoad being called before onLoadStart and seems to be the root cause of this issue and if so then we must fix this upstream.

It may be related to the fact it is cached rnfastimage cashes the file those may mismatch the order of load events

Adding Help Wanted since this job is still open for hiring and hid the comment about hiring you, @s77rt since you're the C+, but you haven't been hired to work on this job :D (just to clarify why i'm seemingly randomly doing things in this issue)

Explanation on why the event could possibly not in order. So, on Android, RN fast image are doing 3 request asynchronously.
The first request below to check whether there is a cache or not and will fire onLoadStart after it complete. I am not sure why it's trying to send the cache path because on the documentation, onLoadStart does not have any parameter.

The second request is the request to load the image and put (basically show) it into the Android ImageView.

If the request complete successfully, it will fire onLoadEvent only, but if there is an error, it will also fire onLoadError before onLoadEnd.

Now, the third request to get the target image size and will fire onLoad event after it complete.

Because all those 3 requests are executed asynchronously, we can't guarantee the order of the event.
On iOS, all the requests are executed sequentially.

I have updated my proposal a bit to overcome the asynchronous issue.

@Litande Thanks once again for your proposal.
Your RCA is accurate but not good enough, there is indeed a bug where containerHeight is referenced before being set onLayout causing aspectRatio to be NaN and yields the bug on dev. This will need to be fixed as well but it's not the root cause of the reported issue.
You are proposing to fix this using promises, I think that's a bit overengineered. Can you propose a more straightforward solution?

Your proposal is well structured but please next time use permalinks to refer to the code you are talking about and use the proposal template as is (respecting the heading levels).

@bernhardoj Thank you for your proposal.
I agree with your RCA. The root cause is correct where onLoadStart is called after onLoad resetting the isLoading to true causing the infinite loading indicator. However your solution of using onLoadEnd won't work either as I have encountered a case where onLoadEnd is not called in the end either.
I see that you also brought up the containerHeight bug but it's not clear how you will fix that, would you please elaborate?

The proposal is also well structured but embedding code blocks makes it a little harder to read, it would be better if you use permlinks instead.

@s77rt
First of all, thanks for providing the proposal link, didn't notice it.
Regarding the permalinks will do still learning ) only 4 days here in such capacity.
Regarding the RCA, you are totally right, in my initial analysis after fixing the first issue, i didn't notice the second imiditaly, and was thinking the first error is what causes the main issue.
I also think that promises aren't the simpliest of solutions, that said there is here 2 indepenedent events.

the loading of the image.
the layout event which i believe is caused by transition of AttachmentView
both of those events are happening simultaniously, those the initial idea is to wait for both of the to complete.
ofcouse we can since them in the following order:
layout
render image
but this will create an effect of the image loader to load later, and if the image is loaded fast, it will cause a flicker effect there.

@bernhardoj I see that you have updated your proposal.
I think that would work, but we can just do it in two steps:

Set initial isLoading to true
Remove onLoadStart

I see that you also brought up the containerHeight bug but it's not clear how you will fix that, would you please elaborate?

The general idea is to wait for both containerHeight and the image size is ready. We already have containerHeight as state, so we need to make a new state to hold the image size from onLoad instead of calling configureImageZoom immediately. We can use componentDidUpdate to know whether both state is ready or not.
if (containerHeight and imageSize is not empty) and (prev containerHeight != containerHeight or prev imageSize != imageSize) then call configureImageZoom

I think that would work, but we can just do it in two steps:

Yep, you are right.

 Unless I'm missing something?

@s77rt that what i proposed in the second part there, but i do believe it may cause loader flickering effect

@Litande I'm sorry but your second part was not clear about that, please use pseudo-code if necessary or permalinks to link where your propose the change to be. You need to provide the bare minimum of technical details. Can you elaborate how will this cause flickering effect?

Ah yes, I think that will works. It is much simpler.
Can you explain why it would cause flickering effect? I haven't test it but looking at the flow, I don't think it will cause flickering effect. The containerHeight is initially undefined and after the onLayout of the outer View is complete ImageZoom will be rendered. Even though the ImageZoom is rendered, the image won't be visible because the opacity is set to 0 while the loading is set to true. The image will be shown after the configureImageZoom is complete.

@s77rt will prep a video in hour with an example
But in theory, the attachment view will transition first, and finish layout.
Then the image will render and show loader till load is finished, this may be fast and in case of cached image really fast those effect of flicker

@Litande I still don't see how will this cause the flickering effect, here is the flow:

containerHeight is not set
Show loading indicator
onLayout callback is executed
containerHeight is set
Loading indicator is still shown
Render ImageZoom and Image
Image is loaded, set isLoading to false
Loading indicator is hidden

Perhaps I'm missing something here?

@bernhardoj In the meantime, please update your proposal based on https://github.com/Expensify/App/issues/15288#issuecomment-1441517895 and https://github.com/Expensify/App/issues/15288#issuecomment-1441561055 and make the proposal a bit clearer (replace embedded code blocks with permlinks)

@s77rt if i am not wrong loader indicator is part image not image view, but let verify it

@Litande It's not, the indicator is a separate component rendered here.

@s77rt yep my bad

Updated the proposal.
Do you mean the embedded code here?

@Litande In that case, I think we will move forward with @bernhardoj's proposal, you clearly have the skills to fix bugs given your accurate RCA while still being new here! You are most welcome to work on others bugs (there is plenty) checkout issues with Help Wanted label.

@bernhardoj I mean the embedded code on your proposal.

@s77rt all good didn't expect my proposal will go through @bernhardoj initial one is better.
One thing the i believe the item that needs to be wrapped is Image and not ImageZoom.
Image is the one calling configureImageZoom
also off topic can you check if there is anyone that can add me to slack, trying to get there since friday with 3 emails...
Thanks.

@Litande Thanks for understanding! I think there is no use to render ImageZomm without Image so we can wrap the whole ImageZoom component.
As for the slack, I have a left a message on Slack, hopefully you will get access soon. Thanks for your interest!

@s77rt aaa again ) you are right it contains Image )))

@Litande Whats your email address used for the request?

@mountiny<EMAIL_ADDRESS>

Thanks, getting someone to look into this!

I suppose we are okay to go with @bernhardoj's proposal. Technically the bug is upstream where onLoad and onLoadStart events are not always in order, we can wait for the fix to land upstream but in our case this is not really required since the use of onLoadStart is actually redundant where it will always be called thus we can just set the initial state of isLoading to true and only relay on one event (onLoad) to set isLoading to false.
What do you think? @bondydaa

I think I'm following @bernhardoj's proposal so going to assign it to you.
One thing I'm not 100% sure on is if we update to this this.state.containerHeight && <ImageZoom>...</ImageZoom> is there potential to cause unneeded re-renders?
I see we set the state of containerHeight here in onLayout
https://github.com/Expensify/App/blob/9d04b46456253955c96981db9ef38245953baf18/src/components/ImageView/index.native.js#L133-L138
which per the docs https://reactnative.dev/docs/view#onlayout looks like it is called

Invoked on mount and on layout changes.

so maybe that is what we want?

📣 @bernhardoj You have been assigned to this job by @bondydaa!
Please apply to this job in Upwork and leave a comment on the Github issue letting us know when we can expect a PR to be ready for review 🧑💻
Keep in mind: Code of Conduct | Contributing 📖

is there potential to cause unneeded re-renders?

@bondydaa I have checked, there are 3 renders that are all necessary (first render before setting container height, second after setting it, third after loading the image) and the onLayout callback is only called once.

Thanks for checking that @s77rt. Yes, there will be 3 renders only, mount and update twice.
PR is ready for view.

Reviewing label has been removed, please complete the "BugZero Checklist".

The solution for this issue has been :rocket: deployed to production :rocket: in version 1.2.77-4 and is now subject to a 7-day regression period :calendar:. Here is the list of pull requests that resolve this issue:

https://github.com/Expensify/App/pull/15472

If no regressions arise, payment will be issued on 2023-03-09. :confetti_ball:
After the hold period is over and BZ checklist items are completed, please complete any of the applicable payments for this issue, and check them off once done.

[ ] External issue reporter
[ ] Contributor that fixed the issue
[ ] Contributor+ that helped on the issue and/or PR

As a reminder, here are the bonuses/penalties that should be applied for any External issue:

Merged PR within 3 business days of assignment - 50% bonus
Merged PR more than 9 business days after assignment - 50% penalty

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

[ ] [@s77rt / @bondydaa] The PR that introduced the bug has been identified. Link to the PR:
[ ] [@s77rt / @bondydaa] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:
[ ] [@s77rt / @bondydaa] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:
[ ] [@bernhardoj] Propose regression test steps to ensure the same bug will not reach production again.
[ ] [@davidcardoza] Link the GH issue for creating/updating the regression test once above steps have been agreed upon:

Regression Test Step

Open any chat
Send an image attachment
Open the image you just sent
Verify the image loaded and shown successfully

Do we 👍 or 👎 ?

The PR that introduced the bug has been identified:

https://github.com/Expensify/App/pull/6442 for the introduction of react-native-fast-image (which is bugged)
https://github.com/Expensify/App/pull/8238 for triggering the bug in the upstream repo (the use of multiple async callbacks)
https://github.com/Expensify/App/pull/8804 for the mishandled containerHeight

The offending PR has been commented on:

https://github.com/Expensify/App/pull/6442#issuecomment-1453184136
https://github.com/Expensify/App/pull/8238#issuecomment-1453188431
https://github.com/Expensify/App/pull/8804#issuecomment-1453202571

A discussion in #expensify-bugs has been started: https://expensify.slack.com/archives/C049HHMV9SM/p1677831828579259

cc @bondydaa

@bernhardoj Please update the regression test to take into account that this must be tried 3-5 times given that the reproduction of this issue is somehow random.

@s77rt Is it better to mentioned to repeat a fixed number of times (3-5) or just multiple times?

@bernhardoj Any will do :+1:

Hi @s77rt , @bondydaa , @davidcardoza, The issue has hold for payment of 9th march, can you have a look once you all are available?

I can process payment. Is it $1000 to @s77rt and @bernhardoj  and $250 to @dhanashree-sawant ?

@davidcardoza I believe @bernhardoj and I are eligible for 50% timeline bonus.

@bernhardoj and @dhanashree-sawant offers sent. @s77rt payment was sent.

@bondydaa Checklist ^ https://github.com/Expensify/App/issues/15288#issuecomment-1453205786

oh sorry 👍 looks good to me

are we good to close? i think payments and everything else is set right?


Image resizer

Description of the new feature / enhancement
Add option to add a coloured background.
Scenario when this would be used?
Turning a landscape or portrait photo into a square picture for things like instagram post. This will add a black/white background to it. instead of filling or cropping a picture.
Supporting information
No response

Image Resizer is meant to be "light and fast", adding extras like this opens the door to many more extras and that will lead to a heavier app. Like there are countless of already. I'll mark as suggestion for now, but personally I wouldn't expect too much.

I don't think PowerToys will create this utility


fix: Memory validation fix + core_memory_replace runaway content repeating fix

Please describe the purpose of this pull request.
Is it to add a new feature? Is it to fix a bug?
Fixes bug (Also see note at bottom)
How to test
How can we test your PR during review? What commands should we run? What outcomes should we expect?
Make agent core memory (human or persona be 1999 characters with a 2000 limit) Ask agent to save something more to the memory. If it does actually do it, the validation and function call would fail but the data was still applied to the agent. On next agent load it will throw an error because the memory is over the limit.
For next item, just have the AI update a memory block with core_memory_replace and have it send "" for old content. It will replace the text with a huge blob (size based on # characters on the existing content in that bank)
Have you tested this PR?
Have you tested the latest commit on the PR? If so please provide outputs from your tests.
I am now able to trigger a save, function fail, and then the length remained at (1999).
I was also able to have AI repair its own persona by replacing it entirely with old_content = "" and new_content = "lorem ipsum etc".
Related issues or PRs
Please link any related GitHub issues or PRs.
Is your PR over 500 lines of code?
If so, please break up your PR into multiple smaller PRs so that we can review them quickly, or provide justification for its length.
Additional context
Add any other context or screenshots about the PR here.
Idk if the validation system being used normally recommends set_attr to be overridden like it is but I'm not an expert on the lib being used. I needed to move super call to be after the validation, and to run validation on a copy so the old value is reserved if it fails (exits before super called).
[Action item #1] - Also, there was a bug with core_memory_replace. If the AI sends a "" for old content, it will replace every existing character with the new_content value. This leads to trying to save repeating text in the hundreds of thousands of characters long (depending on the existing length of old_content on agent already. I updated the function in code to replace all if no old content was specified. I don't know if that's ideal per design or not.. if not then please update this PR with a validation rule so the AI can at least be aware of an issue.
[Action item #2] - My sqlite database toolmodel table for core_memory_append and replace referenced the memory as self.memory.memory[name] = ...    but the code I edited in this PR would only apply to new agents (I believe) So there may need to be a migration script added to this PR to repair existing agents if you guys deem the code update necessary. Also, the code I edited for the append referenced memory as self.memory[name].value instead of self.memory.memory[name].value. I'm not sure of the reasons why there is a difference but memory.memory (in the db) is what works for me.
I added the two action items above for some minor help/adjustments to this PR if needed. I fixed this to get past some blockers I had using local llms and updated the tool in my own DB but the migration stuff I want to leave to you guys. I assume it's sqlite, postgres, etc or something. Overall I think this contains most of a critical fix. Someone else in the chats were having agents break from some of the things happening here. (huge text sizes and it saving huge text sizes in the first place)

I'm having the AI test the function with me throwing a ValueError when old_content is "". Works as intended but then it's a little more complex to have it do a full replace if it wanted. But making that too easy is also kind of scary. What are you guys' thoughts on the logic design of this situation? I can update PR if it's straightforward.

Anything new with this? For my use case, I haven't run into any new blockers since I fixed the issue.

HI sorry for the delay - I added a few comments. Could you also please resolve the merge conflicts?


Update to lodash 3.0

I looked over the changes and I don't think that anything broke on this but an extra pair of eyes would be good: https://github.com/lodash/lodash/wiki/Changelog#compatibility-warnings

By the way, for now I can't use both bower restangular and lodash 3.x in my projects due to the "Warning: ECONFLICT Unable to find suitable version for lodash"

I have had no problems upgrading to 3.1.0 so far.

No issues on 3.1.0 so far, any plans for upgrading?

:+1:

:+1:

Thanks guys for the PR. We will double check it does not break anything and update the build once completed.

Any update on that? We already have lodash 0.3.5.

:+1:

+1

+1

+1

+1

+1

+1

:+1:

👍🏻

+1

+1

It's there guys :bomb:


Where is the  post-processing code in this repository?

Hi, thanks for the interesting points in your paper and the code you provided.
Could you help me to find where is the linear interpolation post-processing code described in Table 10 of the paper located?

Hi,
Please check the code at:
tools/interpolation.py and tools/gp_interpolation.py


Chart style

add some chart style, such as linePlusBarChart, multiBarHorizontalChart, multiChart

PR against master branch please.


BUG ViewableData::__isset reports false for getters (PHP 7)

In php 7.0.6 a recent fix to isset means that certain behaviour that framework relies on no longer works.
For the time being we have allowed failures for 7.0 at https://github.com/silverstripe/silverstripe-framework/pull/5449
My suggestion is that we update ViewableData::__isset to also respect method getters, and not only fields on the current object.
However, even the behaviour of __isset for fields isn't quite correct; Currently we report that a field is set even if it is null, which isn't correct.
The method should probably look something like this:
public function __isset($property) {
	if($this->hasMethod($method = "get$property")) {
		return $this->$method() !== null;
	}
	if($this->hasField($property)) {
		return $this->getField($property) !== null;
	}
	if($this->failover) {
		return $this->failover->__isset($property);
	}
	return false;
}

However, this might need some discussion.

I think your example of what the method might look like looks fine, what regression risks are there? Any test failures if that change is made?
I suppose in some scenarios we’d be doubling the number of calls to get{$property}() and getField($property), but the risk of issues arising from that should be pretty tiny. It might also be a slight performance hit but there’s nothing we can do about that if we’re going to do this properly.

This is definitiely a master change, especially since it's here to fix PHP7 support.
I think we put this in, and if it starts getting problematic for performance reasons, we refactor so as to reduce the use of this function.

Checking a method for isset seems like bad semantics. We should be checking magic-fields that are designed by methods (eg getState()) but not direct method calls.

 Is this actually the case? We should be using whatever __get uses

Checking directly for get$field assumes that this is the only way of defining a getter method. Is this actually the case? We should be using whatever __get uses

The only reason I didn't use this is because I'd prefer to use $this->failover->__isset rather than $this->failover->$field !== null, since it might have better logic.

Yep, now that I'm not on my phone, I've read the code in more detail and I'm fine with it.
More specifically: any change to __get() would also require a change to __isset(), but that's expected behaviour.

I say get this in a PR and see what Travis tells us!

I've raised #5452 as a suggestion for how we can reduce the number of isset() calls...

Sam has started a fix for this at https://github.com/sminnee/silverstripe-framework/commits/php706-isset-fix

Is this issue fixed in master now? @sminnee 's fix wasn't merged yet was it?

I wasn't aware of a fix for this.

Fixed with https://github.com/silverstripe/silverstripe-framework/pull/6232


Wip/redsign xml handling

There is the issue with the search function. I will try and fix it before merge. If I can't figure it out in the morning, I suggest we merge and fix in a new PR so we can keep working on the deduplication.

Codecov Report
Base: 42.86% // Head: 41.13% // Decreases project coverage by -1.73% :warning:

Coverage data is based on head (df3c008) compared to base (97020eb).
Patch coverage: 10.86% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #114      +/-   ##
==========================================
- Coverage   42.86%   41.13%   -1.74%
==========================================
  Files          25       25
  Lines        1570     1658      +88
==========================================
+ Hits          673      682       +9
- Misses        897      976      +79

Impacted Files
Coverage Δ

src/lib/database/database.py
34.66% <0.00%> (ø)

src/lib/datahandler.py
22.61% <7.14%> (+1.83%)
:arrow_up:

src/lib/metadata.py
8.66% <9.91%> (-0.05%)
:arrow_down:

src/lib/tamer.py
12.50% <10.74%> (-1.93%)
:arrow_down:

src/lib/database/db_init.py
30.90% <21.05%> (+0.29%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Typo

Fix typo, false (weekly builds) are the default. Set jenkins_prefer_lts: true to use the current LTS version.

Hello, I'm still interested in getting this minor PR merged. Let me know if any changes are necessary.

Hello, I'm still interested in getting this minor PR merged. Let me know if any changes are necessary.

This fixes issue  #336.

Hello, I'm still interested in getting this minor PR merged. Let me know if any changes are necessary.


Arguments for simple H3 model than could learn Causal or Masked LM modeling?

Hi all! I've been trying to get the training experiments to run and struggling with some errors in which hydra cannot parse the configs given at /experiment/pile/h3 (I was following the instructions in experiments.md for the Pile). I'm actually hoping to train on a different dataset entirely though, for which I already have a working pipeline. Given correct installation of the dependencies in this repo, is there a best way to instantiate an H3 model that would be suitable for Causal language modeling, and/or for Masked LM?
For example, hoping to come up with something comparable to this, and just using it in my existing pipeline:
config = AutoConfig.from_pretrained(
        "roberta-base",
        vocab_size=tokenizer.vocab_size,
        random_init=True,
        is_decoder=True
    )

model = RobertaForCausalLM(config)

Here is what I have for Causal LM, though I am still trying to sort out the dependencies to get it to run.
model = ConvLMHeadModel(
        d_model=768, n_layer=12, d_inner=768 * 4,
        vocab_size=tokenizer.vocab_size, resid_dropout=0.0, embed_dropout=0.1,
        layer="h3", attn_layer_idx=[1, 8],
        attn_cfg=None,
        fused_mlp=True,
        fused_dropout_add_ln=True,
        residual_in_fp32=True,
        pad_vocab_size_multiple=8,
    )

Are these reasonable choices for these arguments, and/or are there others I would need to specify? Are there important pieces of the training or data preparation specified in the configs, that I would need to replicate in another pipeline designed for a HF transformer LM? Sorry if these are obvious/documented, was just having a hard time reading through the configs.

That is mostly correct, but you would also need to pass in an attn_cuff to
specify how many heads you want in the two attention layers, like specified
here:
https://github.com/HazyResearch/safari/blob/9ecfaf0e49630b5913fce19adec231b41c2e0e39/configs/experiment/pile/h3.yaml#L17
You can use this as an example of how to construct the model in Python:
https://github.com/HazyResearch/H3/blob/8ebedd61275770b1fca6e0f8a31e642529d8aa97/examples/generate_text_h3.py#L37
The SSMLMHeadModel there has the same constructor as ConvLMHeaodel in this
repo.
We haven’t worked too much with masked LM yet, but let us know if you get
something working!
On Sun, Mar 12, 2023 at 5:56 AM Brendan King @.***>
wrote:

Hi all! I've been trying to get the training experiments to run and
struggling with some errors in which hydra cannot parse the configs given
at /experiment/pile/h3 (I was following the instructions in experiments.md
http://./experiments.md for the Pile). I'm actually hoping to train on
a different dataset entirely though, for which I already have a working
pipeline. Given correct installation of the dependencies in this repo, is
there a best way to instantiate an H3 model that would be suitable for
Causal language modeling, and/or for Masked LM?
For example, hoping to come up with something comparable to this, and just
using it in my existing pipeline:
config = AutoConfig.from_pretrained(
"roberta-base",
vocab_size=tokenizer.vocab_size,
random_init=True,
is_decoder=True
)
model = RobertaForCausalLM(config)
Here is what I have for Causal LM, though I am still trying to sort out
the dependencies to get it to run.
model = ConvLMHeadModel(
d_model=768, n_layer=12, d_inner=768 * 4,
vocab_size=tokenizer.vocab_size, resid_dropout=0.0, embed_dropout=0.1,
layer="h3", attn_layer_idx=[1, 8],
attn_cfg=None,
fused_mlp=True,
fused_dropout_add_ln=True,
residual_in_fp32=True,
pad_vocab_size_multiple=8,
)
Are these reasonable choices for these arguments, and/or are there others
I would need to specify? Are there important pieces of the training or data
preparation specified in the configs, that I would need to replicate in
another pipeline designed for a HF transformer LM? Sorry if these are
obvious/documented, was just having a hard time reading through the configs.
—
Reply to this email directly, view it on GitHub
https://github.com/HazyResearch/safari/issues/3, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABDDIIR3JDP6E6X6PSU6AELW3WMULANCNFSM6AAAAAAVYAQTXE
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Awesome, thanks! This worked great. Just for my understanding, is the ConvLMHeadModel a synonym for SSMLMHeadModel, or do they just share the same constructor signature?
I also wanted to ask if there are any necessary steps to apply attention masks/prevent forward leakage of information in causal LM training for this model. Based on my experiments so far results would suggest there is no such leakage, but I was surprised no attention_mask or similar arguments are accepted in forward.  Apologies if I'm missing something that should be obvious for SSMs/H3, I'm not very familiar with how they work and am mostly trying to assist some students in setting up/debugging the model for one of their experiments

Hi I had forgotten to come back to this, but this was very helpful and everything works as expected! Thanks for the help and the insights


Fix help text not rendering as "safe"

Fixes issue #92

Could the test be corrected rather than deleted so we can capture a regression in future?

Sure thing, I'll edit the PR

Complete.

Thank you! 👍


Change the OE dataset for training on custom data

While training the fcdd using run_custom.py I'm getting this error:
FileNotFoundError: [Errno 2] No such file or directory: '/home/OneClass/fcdd/data/datasets/imagenet22k/fall11_whole_extracted'
I guess for training on custom data, ImageNet22k is used as an OE dataset. I want to change this to ImageNet1k dataset. I have downloaded and placed the appropriate files in the datasets folder. @liznerski Could you please help me how to change the OE dataset to Imagenet1k?

I solved this error by specifying the noise-mode argument='imagenet'. But I want to now change the input size of the images. There is no input size argument available. Could you please tell me where can I change this parameter?

Here. All images are resized to raw_shape and cropped (random at training, center at testing) to shape. You can also customize this behavior here.

Thanks @liznerski  for the answer. I want to save the trained model. Can you please tell how to do that?

Hi @liznerski, for training with ground truth maps, what should the training images masks be like? Should they be just plain black images? Since you had mentioned we should use 255 for anomalous regions, I had this doubt.

@liznerski assert len(set(gts.reshape(-1).tolist())) <= 2, 'training process assumes zero-one gtmaps'
Does this error mean the ground truth maps I'm using are not in [0, 1] range?

Actually not in {0,1}. They need to be binary.

@liznerski for the first three batches of first epoch, I'm getting NaN values for errors. Is this normal or there's some issue with my training setup?

Oh that's alright. It's because the logger outputs a moving average and that is None as long as the windows isn't filled.

@liznerski transforms.Normalize(mean, std)
Is this transform necessary? Can I use some other values for normalisation, or can I directly omit the normalisation before training?

Per default, the code computes the mean and std of the training data and uses them in a transforms.Normalize(mean, std) as part of the data preprocessing pipeline. You can customize this behavior to your liking. I don't see a reason why FCDD shouldn't work with other kinds of normalization strategies (or even without normalization).


AssumeDefaultVersionWhenUnspecified is not working as expected

I have been using asp net core versioning component for my WebAPI. Need your help in understanding how AssumeDefaultVersionWhenUnspecified is working. (tried searching for documentation, but couldn't find one)
My startup looks like below
services.AddApiVersioning(o => {
            o.ReportApiVersions = true;
            o.AssumeDefaultVersionWhenUnspecified = true;
            o.DefaultApiVersion = new ApiVersion(2, 0);
            o.ApiVersionReader = new UrlSegmentApiVersionReader();
        });

When the route attribute is something like below
[ApiVersion("2.0")]
[Route("api/v{version:apiVersion}/values")]
[ApiController]
public class ValuesV2Controller : ControllerBase
{
...
}

The above route works only when the api version is specified. ie: http://localhost:55401/api/v2/values If I call like http://localhost:55401/api/values, getting 404 error
My question is this. How AssumeDefaultVersionWhenUnspecified works. Wouldn't it ignore the version in Route? Looks like Route attribute takes precedence over AssumeDefaultVersionWhenUnspecified. If I choose QueryString or Header versioning and when the Route looks like
[ApiVersion("2.0")]
[Route("api/values")]

the default routing reaches the API (when the version is not specified in the query string or header)
Am I missing anything or is my understanding wrong? How shall I achieve default routing to the latest version API using url versioning?

In short, you can't. Since this style of versioning uses a URL segment, there is no way to omit part of the path. By definition of the Uniform Interface REST constraint, the path is the identifier so it doesn't really make sense that you can default or omit part of it.
There is a way to make it work as explained here in the wiki. You will have to define multiple routes, effectively an alternate identiifer, where both routes map to the same action. One will have the API version in the path and the other will not. When the path without the API version is requested, the lack of an API version will be allowed and assumed to be the configured, default API version (as you have shown).
It also appears that you want to float this behavior to your latest version. Keep in mind that if you do that, you run the risk of breaking older clients because they won't know they've been redirected to a nwe API version. Any API change, even additive, is not a guarantee that a client will not break. You will also have to move the route declaration without the API version segement to the latest controller implementation. You probably want to set the option:
options.ApiVersionSelector = new CurrentImplementationApiVersionSelector().
This will select the maximum API version (e.g. the current one) so that you don't have to keep changing options.DefaultApiVersion which is used in several scenarios, not just for assuming the default version.
This limitation only applies to the URL segment method because it uses the path. All of the other supported versioning methods can support your requirements without having to do any other special setup.
I hope that helps.

Fantastic @commonsensesoftware. Thank you so much for the reply. This helps me a lot

You sure can. It looks like you're using Swashbuckle. My understanding is that you'll need to do this with your own, custom IDocumentFilter.
It will look something like:
public class DefaultApiVersionFilter : IDocumentFilter
{
  public DefaultApiVersionFilter( ApiVersion defaultApiVersion, string apiVersionFormat )
  {
    DefaultApiVersion = defaultApiVersion;
    ApiVersionFormat = apiVersionFormat;
  }

  // this should the same as the DefaultApiVersion option
  protected ApiVersion DefaultApiVersion { get; }

  // this should be the same as the SubstitutionFormat api explorer option
  protected string ApiVersionFormat { get; }

  public void Apply(
    SwaggerDocument swaggerDoc,
    SchemaRegistry schemaRegistry,
    IApiExplorer apiExplorer)
  {
    // take the guess work out of formatting the version the same way as the api explorer
    var versionSegment = DefaultApiVersion.ToString( ApiVersionFormat );

    foreach ( var apiDescription in apiExplorer.ApiDescriptions.OfType<VersionedApiDescription>() )
    {
      if ( apiDescription.ApiVersion == DefaultApiVersion &&
           apiDescription.RelativePath.Contains( versionSegment ) )
      {
        // remove the path that contains the versioned url segment
        var path = "/" + apiDescription.RelativePath.TrimEnd( '/' );
        swaggerDoc.paths.Remove( path );
      }
    }
  }
}

I hope that helps.

Thank, I help but do I'm struggling one more thing. I am using Swashbuckle.AspnetCore. The IDocument filter do not implement IApiExplorer, so I modify the code and my Apply looks like:
    public void Apply(
        SwaggerDocument swaggerDoc,
        DocumentFilterContext context)
    {
        var versionSegment = DefaultApiVersion.ToString(ApiVersionFormat);

        foreach (var apiDescription in context.ApiDescriptions.OfType<ApiDescription>())
        {
            if (apiDescription.GetApiVersion() == DefaultApiVersion &&
                apiDescription.GroupName.Contains(versionSegment))
            {
                // remove the path that contains the versioned url segment
                var path = "/" + apiDescription.GroupName.TrimEnd('/');
                swaggerDoc.Paths.Remove(path);
            }
        }
    }

But now, I have problem with start the app. I get
Unable to resolve service for type 'Microsoft.AspNetCore.Mvc.ApiVersion' while attempting to activate 'Nofo.WebApi.Web.Core.Extensions.Swagger.DefaultApiVersionFilter'.
I used Microsoft.AspNetCore.Mvc.Versioning.ApiExplorer 2.2.0 and Swashbuckle.AspnetCore 3.0 and my usings are:
using System.Linq;
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Mvc.ApiExplorer;
using Swashbuckle.AspNetCore.Swagger;
using Swashbuckle.AspNetCore.SwaggerGen;
My AddSwaggerGen looks like:
        services.AddSwaggerGen(c =>
        {
            var provider = services.BuildServiceProvider()
                .GetRequiredService<IApiVersionDescriptionProvider>();

            foreach (var description in provider.ApiVersionDescriptions)
            {
                c.SwaggerDoc(description.GroupName, CreateApiVersionInfo(description));
            }

            c.DocumentFilter<DefaultApiVersionFilter>();

This is due to how your filter is being registered. It is configured with dependency injection and the runtime is unable to resolve your constructor's parameters. You should be able to update it like this:
private readonly IOptions<ApiExplorerOptions> options;

public DefaultApiVersionFilter( IOptions<ApiExplorerOptions> options ) => this.options = options;

protected ApiVersion DefaultApiVersion => options.Value.DefaultApiVersion ;

protected string ApiVersionFormat => options.Value.SubstitutionFormat;

@commonsensesoftware Many thanks for your help! Now everything works great!
I've changed a little logic to be working with .NET Core. Now if the version is default swagger document do not show path with the version, else only paths with version are shown. Please find what the final solution, maybe someone will be interested.
using System.Text.RegularExpressions;
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Mvc.ApiExplorer;
using Microsoft.Extensions.Options;
using Swashbuckle.AspNetCore.Swagger;
using Swashbuckle.AspNetCore.SwaggerGen;

namespace MyProject.Core.Extensions.Swagger
{
    public class DefaultApiVersionFilter : IDocumentFilter
    {
        private readonly IOptions<ApiExplorerOptions> _options;

        public DefaultApiVersionFilter(IOptions<ApiExplorerOptions> options)
        {
            _options = options;
        }

        protected ApiVersion DefaultApiVersion => _options.Value.DefaultApiVersion;
        protected string ApiVersionFormat => _options.Value.SubstitutionFormat;

        public void Apply(
            SwaggerDocument swaggerDoc,
            DocumentFilterContext context)
        {
            var versionSegment = DefaultApiVersion.ToString(ApiVersionFormat);

            foreach (var apiDescription in context.ApiDescriptions)
            {
                //If the version is default remove paths like: api/v1.0/controller
                if (apiDescription.GetApiVersion() == DefaultApiVersion)
                {
                    if (apiDescription.RelativePath.Contains(versionSegment))
                    {
                        var path = "/" + apiDescription.RelativePath;
                        swaggerDoc.Paths.Remove(path);
                    }
                }
                //If the version is not default remove paths like api/controller
                else
                {
                    var match = Regex.Match(apiDescription.RelativePath, @"^api\/v\d+");

                    if (!match.Success)
                    {
                        var path = "/" + apiDescription.RelativePath;
                        swaggerDoc.Paths.Remove(path);
                    }
                }
            }
        }
    }
}


Namespace-indepent GetElementById

I was using XMLDSIGjs to validate a specific XML file see here
The code path inside XMLDSIGjs is as follows:

SignedXml class
Verify()
VerifyReferences() see here
DigestReference() see here
XmlCore.XmlObject.GetElementById() called here

Continuing in XmlCore:
6. GetElementById() : It tries to find an element using Xpath see here
It was not able to get the following element:
  <wstxns8:Body wsu:Id="id-1-9aa320963c34df6165073aa90a42581a"     xmlns:wstxns8="http://schemas.xmlsoap.org/soap/envelope/"     xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
I reasoned that the namespace of the Id attribute was the cause, so I changed GetElementById() to allow looking up elements by id regardless of any namespace on the id attribute.
I don't know enough about the XMLDSIGjs and XmlCode codebase or XML files in general to judge whether this is a good approach, so do with this as you see fit. I'm curious to hear your thoughts on this.
In any case, "it works" for me and I'm able to verify the file now by supplying the CryptoKey from the SPKI in the embedded certificate in the XML file.

The new version<EMAIL_ADDRESS>has been published
@ledgitbe thank you for your update


Error as I follow the "Usage with spaCy" example

Hi,
On macosx 10.13.5 I got an error at the 'nlp.add_pipe(BeneparComponent("benepar_en"))' statement:
`
$ python
Python 3.6.4 |Anaconda custom (64-bit)| (default, Dec 21 2017, 15:39:08)
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import benepar
/Users/colingoldberg/anaconda3/lib/python3.6/site-packages/h5py/init.py:34: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type.
from ._conv import register_converters as _register_converters
benepar.download('benepar_en')
[nltk_data] Downloading package benepar_en to
[nltk_data]     /Users/colingoldberg/nltk_data...
True
import spacy
from benepar.spacy_plugin import BeneparComponent
nlp = spacy.load('en')
nlp.add_pipe(BeneparComponent("benepar_en"))
Traceback (most recent call last):
File "", line 1, in
File "/Users/colingoldberg/anaconda3/lib/python3.6/site-packages/benepar/spacy_plugin.py", line 73, in init
super(BeneparComponent, self).init(filename, batch_size)
File "/Users/colingoldberg/anaconda3/lib/python3.6/site-packages/benepar/base_parser.py", line 163, in init
tf.import_graph_def(graph_def, name='')
File "/Users/colingoldberg/anaconda3/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 316, in new_func
return func(*args, **kwargs)
File "/Users/colingoldberg/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/importer.py", line 541, in import_graph_def
raise ValueError('No op named %s in defined operations.' % node.op)
ValueError: No op named ClipByValue in defined operations.

`
I also tried this on an AWS EC2 (t2.small) instance (2GB memory), and got the following error:
`
python
Python 3.6.3 |Anaconda custom (64-bit)| (default, Oct 13 2017, 12:02:49)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

import benepar
benepar.download('benepar_en')
[nltk_data] Downloading package benepar_en to /root/nltk_data...
[nltk_data]   Package benepar_en is already up-to-date!
True
import spacy
from benepar.spacy_plugin import BeneparComponent
nlp = spacy.load('en')
nlp.add_pipe(BeneparComponent("benepar_en"))
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/share/anaconda3/lib/python3.6/site-packages/benepar/spacy_plugin.py", line 73, in init
super(BeneparComponent, self).init(filename, batch_size)
File "/usr/local/share/anaconda3/lib/python3.6/site-packages/benepar/base_parser.py", line 163, in init
tf.import_graph_def(graph_def, name='')
File "/usr/local/share/anaconda3/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 432, in new_func
return func(*args, **kwargs)
File "/usr/local/share/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/importer.py", line 486, in import_graph_def
with c_api_util.tf_buffer(graph_def.SerializeToString()) as serialized:
MemoryError

`
I look forward to having this issue resolved so that I can continue testing.
Thank you.
Colin Goldberg

The ValueError: No op named ClipByValue in defined operations. is caused by running an older version of tensorflow. Tensorflow 1.8.0 is required to run the benepar_en model. I'm sorry that the tensorflow installation situation is so messy, but it's difficult to solve at the package-manager level because there are separate tensorflow packages for cpu and gpu.
I can't help about the MemoryError: the parsers are actually quite big when uncompressed. You can download and use benepar_en_small instead of benepar_en if your machine has insufficient RAM.

Thank you. Updating tensorflow worked on my mac, and using benepar_en_small on the 2GB EC2 instance worked as well (even as it gave some memory warnings).
Do you have any recommendations, or experience, for (minimum) memory requirements for benepar_en?
As a newcomer to this space, I appreciate any documentation and help, especially examples showing the kinds of things I can do with the results of a parse.

Thanks for pointing out the memory situation. The benepar_en model is around 430 MB when uncompressed. In theory this should be most of the memory that's needed to run the model, but unfortunately there seems to be quite a bit of overhead inside Tensorflow. Steady-state memory consumption on my machine is around 3GB, though I'm not sure how much of that can be garbage collected. I anticipate that a 1-2 GB of RAM (for the ~430MB parameters plus active computations) plus a generous amount of swap space should suffice to run the model. EC2 instances don't normally have swap enabled, which explains why the full-size model won't run.
My recommendation for now is to run benepar_en_small if you need to reduce memory consumption.
Another thing to note is that if your spaCy documents contain many sentences, or if you're using parse_sents from the NLTK integration, benepar will parse a batch of sentences at a time. Both benepar.Parser and BeneparComponent accept a batch_size argument, which can be lowered from the default value of 64 (sentences) to reduce memory usage.

What do you mean by transforming the parse tree to a dict?
A parse tree is a tree data structure, which has very different properties from a dict. For example, children in a parse tree are ordered (while keys in a dict are not, and also can't repeat).
Here's one way to convert the parse tree to a dict, but this is mostly just an example of how to navigate the parse tree:
import spacy
from benepar.spacy_plugin import BeneparComponent
nlp = spacy.load('en')
nlp.add_pipe(BeneparComponent("benepar_en"))
doc = nlp(u"The time for action is now. It's never too late to do something.")
sent = list(doc.sents)[0]
print(sent._.parse_string)

# Note: doesn't work for the last sentence in a document.
# The bug will be fixed in a new release of benepar, shortly
def to_dict(span, idx=0):
    if len(span) == 1:
        res = str(span)
    else:
        res = {}
        for i, child_span in enumerate(span._.children):
            res.update(to_dict(child_span, i))

    labels = list(span._.labels)
    if len(span) == 1:
        labels.append(span[0].tag_)

    for label in reversed(labels[1:]):
        res = {"0/" + label: res}

    res = {str(idx) + "/" + labels[0]: res}
    return res

print(to_dict(list(doc.sents)[0]))

I accept the limitations you point out. The reason I want to do this is that I need to report results back to a non-python caller using a json structure, and this is best done via a dict (I believe). As long as I manage what I am submitting to be parsed, this should be ok.


Create global transforms using geocoder api resources and publish to FME Gallery

Each transform loops through a list and applies a geocoder resource to the list item and saves the response.
There should be transforms for geocoder/addresses, geocoder/intersections/nearest, and geocoder/sites/nearest

Not needed. Use CPF instead.