Create GenerativeTextTemplate that streamlines usage of AiClient, Prompts, and OutputParsers

There is an analogy to make between GenerationTemplate and JdbcTemplate that will help simplify the use of the lower level classes such as AiClient, Prompts, and OutputParsers.  Note: AiClient should likely be renamed GenerationClient as it is focused on generation, specifcally text, so perhaps even TextGenerationClient as we haven't yet experimented with multi-modal apis.
DataSource <-> AiClient

String Query with placeholdres <-> Prompt with placeholders

RowMapper <-> OutputParser

The simplest method (which is a default method in AiClient) is string input and string output, that could be removed from the AiClient interface and used in GenerateTemplate instead.  Here are a few sample signatures for consideration.
public interface GenerateOperations {

	String generate(String message);

	String generate(String message, Map<String, Object> model)

	String generate(PromptTemplate promptTemplate, Map<String, Object> model);

	<T> T generate(String message, Class<T> elementType, Map<String, Object> model);

	<T> T generate(String message, OutputParser<T> parser, Map<String, Object> model);

	<T> List<T> generateList(String message, Class<T> elementType, Map<String, Object> model);

	<T> List<T> generateList(String message, OutputParser<T> parser, Map<String, Object> model);

        AiResponse generateResponse(PromptTemplate promptTemplate, Map<String, Object> model);
}

Example usage
ActorsFilms actorsFilms = generateTemplate.generate("Generate the filmography for the actor {actor}",
 		                                    ActorsFilms.class, Map.of("actor", "Tom Hanks"));

Simple "chains" for flows can be done with standard Java functional programming, and we can see how our needs for a "chain" or a "flow" evolve over time.  e.g.  the example of a chain from langchain using functional programming
	@Test
	void functionalChains() {
		Function<String, String> combinedFunction = generateSynopsis.andThen(generateReview);
		System.out.println(combinedFunction.apply("Tragedy at sunset on the beach"));
	}

	private Function<String, String> generateSynopsis = title -> {
		String synopsisInput = """
				You are a playwright. Given the title of play, it is your job to write a synopsis for that title.

				Title: {title}
				Playwright: This is a synopsis for the above play:""";

		return generateTemplate.generate(synopsisInput, Map.of("title", title));
	};

	private Function<String, String> generateReview = synopsis -> {
		String synopsisInput = """
				You are a play critic from the New York Times. Given the synopsis of play, it is your job to write a review for that play.

				Play Synopsis:
				{synopsis}
				Review from a New York Times play critic of the above play:""";

		return generateTemplate.generate(synopsisInput, Map.of("synopsis", synopsis));
	};

As a note, a new issue to create a 'GenerateTextOperation' class that has a fluent api, akin to SimpleJdbcCall and company

this has been addressed with the fluent api in the updated chatclient.


Suggest looking here

This is all because you are using a fixed width and height for you images... MAJOR NO NO in the age of responsive design.
You either need to use square images or .img-fluid.


2 errors in Aimbot.

nevermind, i fixed it as soon as i posted this thread.

Can you please tell me how you fixed it?


Andrea, are you planning to continue the contribution to this project?

The reason I ask is because I would like to present this project as a solution for  CMIS UI. However, if we decide to go with CmisJS, it would be used for many years.

Hi, I do not plan to stop contributing to this project, but keep in mind that I cannot guarantee any kind of support for this project

Fair enough. Thanks!


Prepare 2.7.2 release

Looks good overall. This adds a vendors/tezos-context-hash submodule that I believe we don't need in 2.7.
For the future, when cherry-picking to these backport branches it can be a good idea to use -x to add some context about where the commit came from – not worth worrying about now.

For the future, when cherry-picking to these backport branches it can be a good idea to use -x to add some context about where the commit came from – not worth worrying about now.

Noticed those only after opening the PR, I added them too.
Because of the progress library, I think we need to also upgrade the index version (which I did here) let me know if I got this wrong.

I think we need to also upgrade the index version (which I did here) let me know if I got this wrong.

This looks fine :+1: Requiring 1.4.1 over 1.4.0 seems like a good idea anyway -- we'll just need to add the latest version of progress to the Tezos opam-repository when we make the upgrade.

Thanks! Now released here: https://github.com/ocaml/opam-repository/pull/19088
The master changelog will need updating accordingly.


Intermediate compute() fails even though whole graph works fine

I am new to dask and started by rewriting a small python script that generates some matplotlib plots from some hdf5 files rather slowly. I've successfully used dask to parallelize this script with bags. I tried to add a plot that is generated with a delayed function instead of bags. Based on my naive understanding of dask I thought it might be better to avoid calling compute more than once if all the computations share the same data source (no matter whether they use bags or delayed). So I tried using dask.bag.from_delayed and then dask.bag.concat to create a 'mixed bag' of 'natural' bags and bags from delayed. Whilst doing this I encountered some confusing errors. I was able to create a series of py.test tests that reproduce the problem while eliminating unrelated code and any dependency on my data.
I've attached 4 tests. 2 of them pass as expected, 2 of them fail and I expected them to pass (which is
noted in the docstrings). The comments contain my 3 questions.
import dask
import dask.bag

def func(x):
    return type(x) == int

def inc(x):
    return x + 1

def ident(x):
    return x

def test_mixed_bag():
    """
    This test develops code that mixes `delayed` and `bags` into the same
    computation graph to avoid calling compute() more than once. We avoid
    calling compute more than once here because the 'mixed bag' of
    computations depend on the same data and thus would like to exploit any
    caching as much as possible. Perhaps this goal is arising from an
    improper understanding of how dask works. But, it has given rise to a
    perplexing issue I'd like to understand nonetheless. Note that this test
    PASSES as expected. However, a small change that shouldn't break it,
    does. That change is captured in
    test_mixed_bag_with_intermediate_compute_on_oddball
    """
    x = dask.bag.from_sequence([1, 3, 5])
    # The following line may seem pointless but its purpose will become
    # apparent.
    x = x.map(ident)
    x_d = x.to_delayed()

    c_d_0 = dask.delayed(lambda x: x[0])(x_d[0])
    assert c_d_0.compute() == 1

    func_d = dask.delayed(func)(c_d_0)

    func_l = dask.delayed(lambda x: [x])(func_d)

    oddball = dask.bag.from_delayed([func_l])

    mixed_bag = dask.bag.concat([
        x.map(inc),
        oddball
    ])

    assert mixed_bag.compute() == [2, 4, 6, True]

def test_mixed_bag_on_oddball_with_x_map_ident():
    """
    See test_mixed_bag for background on this test. This test should pass based
    on my understanding of dask, but it FAILS.
    """
    x = dask.bag.from_sequence([1, 3, 5])
    x = x.map(ident)
    x_d = x.to_delayed()

    c_d_0 = dask.delayed(lambda x: x[0])(x_d[0])
    assert c_d_0.compute() == 1

    func_d = dask.delayed(func)(c_d_0)

    func_l = dask.delayed(lambda x: [x])(func_d)

    oddball = dask.bag.from_delayed([func_l])
    # The following assertion is the only difference with test_mixed_bag.
    # QUESTION 1: Why does this _intermediate_ computation fail when the whole
    # computation does not?
    # Note: I am really only interested in the final computation; but, I often
    # need to insert these intermediate computations to debug results. I
    # encountered this issue when debugging a similar construction and got very
    # confused by the traceback it produces.
    # QUESTION 2: How do I fix this test so it will pass, without removing the
    # line x = x.map(ident). Because in the actual code that inspired this,
    # the equivalent of ident does real work I can't remove (it's a pluck).
    assert oddball.compute()[0]

    mixed_bag = dask.bag.concat([
        x.map(inc),
        oddball
    ])

    assert mixed_bag.compute() == [2, 4, 6, True]

def test_mixed_bag_on_oddball_without_x_map_ident():
    """
    For some reason, if I remove the x = x.map(ident) line from
    test_mixed_bag_on_oddball_with_x_map_ident then the intermediate
    oddball.compute()[0] works as expected and this test PASSES as it should.
    """
    x = dask.bag.from_sequence([1, 3, 5])
    # The following commented out line is the only difference with:
    # test_mixed_bag_on_oddbal_with_x_map_ident
    # x = x.map(ident)
    x_d = x.to_delayed()

    c_d_0 = dask.delayed(lambda x: x[0])(x_d[0])
    assert c_d_0.compute() == 1

    func_d = dask.delayed(func)(c_d_0)

    func_l = dask.delayed(lambda x: [x])(func_d)

    oddball = dask.bag.from_delayed([func_l])
    assert oddball.compute()[0]

    mixed_bag = dask.bag.concat([
        x.map(inc),
        oddball
    ])

    assert mixed_bag.compute() == [2, 4, 6, True]

def test_mixed_bag_on_oddball_with_x_pluck_instead_of_ident():
    """
    In this test I replaced the x.map(ident) with two lines that reduce to
    an identity. The first replaces values with singleton tuples. The second
    uses pluck(0) to transform back into the original list of integers. I
    created this version of the test because the x.map(ident) operation
    actually represents a pluck(0) in my code that inspired these tests. This
    test FAILS and I expect it to pass based on my possibly flawed understanding
    of dask.

    I've included this version out of thoroughness as it produces a different
    error message, namely:
    `TypeError: 'map' objects is not subscriptable`
    Rather than:
    `TypeError: 'generator' objects is not subscriptable`

    Although, I strongly suspect whether it is pluck, or ident doesn't matter.
    The problem is somewhere else.

    QUESTION 3: What can be changed to get this test to pass?
    """
    # The following 4 lines are changed from
    # test_mixed_bag_on_oddball_without_x_map_ident
    data = dask.bag.from_sequence([1, 3, 5])
    x = data.map(lambda x: (x,))
    x = x.pluck(0)
    assert x.compute() == data.compute()

    x_d = x.to_delayed()

    c_d_0 = dask.delayed(lambda x: x[0])(x_d[0])
    assert c_d_0.compute() == 1

    func_d = dask.delayed(func)(c_d_0)

    func_l = dask.delayed(lambda x: [x])(func_d)

    oddball = dask.bag.from_delayed([func_l])
    assert oddball.compute()[0]

    mixed_bag = dask.bag.concat([
        x.map(inc),
        oddball
    ])

    assert mixed_bag.compute() == [2, 4, 6, True]

Here is a dump of the conda environment I'm running this code under.
channels:
- defaults
dependencies:
- bzip2=1.0.6=h9a117a8_4
- ca-certificates=2017.08.26=h1d4fec5_0
- cairo=1.14.12=h77bcde2_0
- certifi=2018.1.18=py36_0
- cycler=0.10.0=py36h93f1223_0
- dbus=1.12.2=hc3f9b76_1
- expat=2.2.5=he0dffb1_0
- fontconfig=2.12.4=h88586e7_1
- freetype=2.8=hab7d2ae_1
- glib=2.53.6=h5d9569c_2
- graphite2=1.3.10=hf63cedd_1
- graphviz=2.40.1=h25d223c_0
- gst-plugins-base=1.12.4=h33fb286_0
- gstreamer=1.12.4=hb53b477_0
- harfbuzz=1.7.4=hc5b324e_0
- hdf5=1.8.18=h6792536_1
- icu=58.2=h9c2bf20_1
- intel-openmp=2018.0.0=hc7b2577_8
- jpeg=9b=h024ee3a_2
- libedit=3.1=heed3624_0
- libffi=3.2.1=hd88cf55_4
- libgcc=7.2.0=h69d50b8_2
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran-ng=7.2.0=h9f7466a_2
- libiconv=1.14=0
- libpng=1.6.34=hb9fc6fc_0
- libstdcxx-ng=7.2.0=h7a57d05_2
- libtiff=4.0.9=h28f6b97_0
- libtool=2.4.6=h544aabb_3
- libxcb=1.12=hcd93eb1_4
- libxml2=2.9.4=0
- lzo=2.10=h49e0be7_2
- matplotlib=2.1.2=py36h0e671d2_0
- mkl=2018.0.1=h19d6760_4
- ncurses=6.0=h9df7e31_2
- numexpr=2.6.4=py36hc4a3f9a_0
- numpy=1.13.1=py36h5bc529a_2
- openssl=1.0.2n=hb7f436b_0
- pandas=0.22.0=py36hf484d3e_0
- pango=1.41.0=hd475d92_0
- pcre=8.41=hc27e229_1
- pip=9.0.1=py36_5
- pixman=0.34.0=hceecf20_3
- pyparsing=2.2.0=py36hee85983_1
- pyqt=5.6.0=py36h0386399_5
- pytables=3.4.2=py36hc5f0ea4_2
- python=3.6.4=hc3d631a_1
- python-dateutil=2.6.1=py36h88d3b88_1
- pytz=2018.3=py36_0
- qt=5.6.2=h974d657_12
- readline=7.0=ha6073c6_4
- scipy=0.19.1=py36h9976243_3
- setuptools=38.5.1=py36_0
- sip=4.18.1=py36h51ed4ed_2
- six=1.11.0=py36h372c433_1
- sqlite=3.22.0=h1bed415_0
- tk=8.6.7=hc745277_3
- tornado=4.5.3=py36_0
- wheel=0.30.0=py36hfd4bba0_1
- xz=5.2.3=h55aa19d_2
- zlib=1.2.11=ha838bed_2
- pip:
  - attrs==17.4.0
  - click==6.7
  - cloudpickle==0.5.2
  - dask==0.17.1
  - distributed==1.21.1
  - heapdict==1.0.0
  - locket==0.2.0
  - msgpack-python==0.5.5
  - partd==0.3.8
  - pluggy==0.6.0
  - psutil==5.4.3
  - py==1.5.2
  - pytest==3.4.1
  - sortedcontainers==1.5.9
  - tables==3.4.2
  - tblib==1.3.2
  - toolz==0.9.0
  - zict==0.1.3

I haven't read through everything in detail yet, but I tried running one of your tests, and got this exception:
TypeError: 'generator' object is not subscriptable

Is this the type of error that you're seeing and are concerned by?  Or, more generally, can I ask you about what isn't working for you?

Yes, the other failing test produces almost the same error. It just has 'map' in place of 'generator'.

Currently the delayed objects you get point to Iterator objects, not necessarily lists.  You might want to call
b.map_partitions(list).to_delayed()


Strange behaivor after fall arango

Version: 3.3.17
Storage: RocksDB
Deployment Mode: Active Failover
Deployment Strategy: Manual Start
Operating System: Ubuntu 16.04
Total RAM in your machine: 32Gb
I have server configuration 3 agents and 3 DB(1 leader and 2 followers). In night become strange behavior: arango stop accept requests for reading and data recording, can't access to web monitoring in leader. But in logs i see error like in screenshot.

Agent see db and communicate with this. It's very bad, that agent don't change leader.
After restart it's become good working.
Why this happen? Is this correct or it's a bug?? How can be safity in this situation? About what resource it's said? What is Scheduler thread?

Hi,
could it be that this cgroups feature hits you:
https://unix.stackexchange.com/questions/253903/creating-threads-fails-with-resource-temporarily-unavailable-with-4-3-kernel
(at least systemd v 228 , kernel v4.4)
Can you try setting [TasksMax](https://www.freedesktop.org/software/systemd/man/systemd.resource-control.html#TasksAccounting=) in /lib/systemd/system/arangodb3.service ?
Please note that arangodb 3.3.21 is available for download and may contain fixes you need.

It's repeat 2 times, that is no feature but bug

Hi,
Can you share some insights with your monitoring about when this happenes? Screenshots from graphana or whatever? Like number of running processes, used memory, threads that ArangoDB is using at that time?
Another reason for threads not being able to start may be that there is no available memory to allocate their stack, so whats your 'overcommit memory setting' ?

this is time falling.

Hi,
can you check your long running queries log? is there something locking up resources since we see the threads oscilating?

Hi, Dothebart. No long query in log. Every  query run <1sec. Many query <0.3 sec

Hi, can you run this in a loop
date; ps H -o 'tid comm' <PID>; sleep 5 > /tmp/somefile
(or maybe in one file each 5 seconds)
so that we may see which threads are spawned at that time?
A figure missing in the screenshot above is disk I/O, do you have that too?
Can you provide a bit more information, on which machine this is? coordinator? DB-Server?
Also it would be nice to have a bit more information on your load - Do you use foxx? what type of queries do you run? Joins? Whats your shards layout? do you have collections with many shards? Whats your documents? Rather small? ~1k Rather Big > 100k?

Some information about linux machine and disks operation:

Threads are in graph about 150-200.  Machine is VM. In DB-Server 32 cpu and 32 ram. In agent machine 8 cpu and 16 gb
Foxx we not using. In graph you can see a lot of PUT query and a bit GET. No shards beacuse we use active failover.

Could you enable request logging : https://docs.arangodb.com/3.3/Manual/Administration/Configuration/Logging.html#log-outputs to find out which sort of requests are causing the rise of the threads?
We're talking about the leader-arangod which is misbehaving, right?

Problem was with memory leak with parameter vm.overcommit_memor = 2. Decision is in issue https://github.com/arangodb/arangodb/issues/8163


compiled .adf please!

would be great having a bootable .adf imagedisk available as well, with the game compiled, for lazy people like me! :D - thanks!

Hi there,
I was recently thinking about removing binaries entirely from repo and setting up some automation for generating binary/adf files on remote server. This game is still heavily under development and about 20% done, so there are no official releases yet. ;)
Until it's done I'll try to add some pre-release releases on github from time to time after each milestone being completed. One thing to wait for is having basic AI implemented, which should arrive during this month. Also note that running from floppy is not really encouraged with game built using debug config - loading times will be very long and game performance will be poor since writing to log on floppy disk takes ages.
So I guess I should supply game in two variants - one with logging enabled and one without it, so that ppl without hdd may test it.

I've recently set up Bebbo's GCC toolchain on my home server, so after https://github.com/tehKaiN/openFire/issues/7 is solved I can automate builds and put them on the net in LHA and ADF formats.


[Bug]: Popover component tests

Environment
Developement/Production OS: Windows 10 19043.1110
Node version: 20.0.0
Package manager<EMAIL_ADDRESS>Radix Vue version: 1.7.1
Vue version: 3.4.21

Link to minimal reproduction
https://stackblitz.com/edit/github-zsxecm?file=src%2Fcomponents%2F__tests__%2FRadixPopover.spec.ts
Steps to reproduce
I hope you are able to see the problem in the reproduction test-suites. It took me quite long to understand where the problem is coming from (whether it is test-utils, Teleport or testing-library), but I believe now it is coming from radix-vue.
Teleport-testing-library.spec.ts shows how testing of a ref within a Popover component is supposed to work.
RadixPopover.spec.ts contains a test that proves that this approach does not work when using Popover. The test fails, because it can never find {{ user }}.
The test suite is using vitest and happy-dom. I've swapped out the latter for jsdom, which doesn't change the situation.
Describe the bug
I think the test-suite shows two problems:

I think, it is currently not possible to stub out Teleport for tests. When doing so, the popover content never gets rendered which makes testing a bit harder than it could be.

If opening the Popover triggers an async operation, which sets a ref in the background that should be rendered within PopoverContent the ref doesn't seem to get re-rendered in a test environment.

I run into this case in a real-world application, when user-data is loaded in the background once the Popover is opened, but once the data is resolved it never gets rendered in the test, although I can see (through a console.log) that it has been resolved.
Expected behavior

It would be nice if we could stub Teleport. I see this more as a nice-to-have. In any case it would probably be good to add some guidance on how to test these components to the docs or a better error message, because at the moment it is a bit unclear what the proper way of testing is.

I would expect to be able to test refs within PopoverContent like I can for other Teleport components aswell.

Context & Screenshots (if applicable)
No response

Had a quick test with the reproduction link, seems like happy-dom doesn't behave as expected.
I've added a console.log of the state of html after clicking Toggle, and realized that something was off. The data-state somehow was closed.
I tested the same on the current repo (using jsdom), seems to be working fine.

Oh you are right 🤦🏼 Apologies I must have overlooked that jsdom works without the Teleport stub.
Do you have any idea what kind of APIs might not be implemented in happy-dom? I've had a look, but my feeling is that it must come from floating-ui already, because neither Popover, Popper or Teleport contain any DOM related APIs that might be missing.

No sure tho, not quite familiar with happy-dom. In jsdom there's error being logged when DOM API were missing.
Perhaps there's a way to log that in happy-dom too?

I'll have a look 👍🏼 So far I couldn't make it log any error which makes me thing floating-ui swallows it somewhow.
What's your position on the Teleport stub? If you don't consider that a bug I'd go ahead and close this one and make sure to create a follow-up either for happy-dom or floating ui.

I'll take a look too. Let's keep this open for now so anyone using happy-dom for testing can comment if they have any resolution for this 😁

As this doesn't seems like an issue with radix-vue, as I've tested with jsdom and it works. Moving this to discussion instead 😁


pipelines is ready test is failing

https://k8s-testgrid.appspot.com/sig-big-data#kubeflow-gcp-blueprints-master-periodic
@Bobgy could you please take a look? It could be the case that the test needs to be updated.

@jlewi Is this the entrypoint of the test? https://github.com/kubeflow/kfctl/blob/7bfe692bdfb42002073c0ea196c5942e606ed48c/py/kubeflow/kfctl/testing/pytests/kf_is_ready_test.py#L96

@Bobgy yes; if you set your current kubectl context to point at a cluster you should be able to run that test locally which may help debug it.

@jlewi I looked at auto deployed clusters and found the root cause is:
persistence disks were created in us-central1-f zone, while nodes were in us-central1-{a,b,c} zones, so PVs cannot be mounted to pods.
Do you have any ideas why they are deployed to different zones?

@Bobgy Its because we are using a regional cluster so it has nodes it multiple zones. It looks like kubeflow/gcp-blueprints#6 we still have some work to do to make this work with regional clusters.
The short term fix would be to change the autodeployments to use a zonal cluster.
I think we could change the defaults here
https://github.com/kubeflow/testing/blob/637d1cd5fe33d03ee5646380d960fabe8a230d0a/py/kubeflow/testing/create_kf_from_gcp_blueprint.py#L69
I don't think the blueprint reconciler is overwriting the defaults

@Bobgy Would you mind submitting a PR to try to change the auto-deployer to use a zonal cluster?

It's been Chinese holidays for three days, I will return to work on Sunday (a working day).

@Bobgy thanks for the heads up; I've been OOO as well; enjoy the holiday; we can fix this next week.

e

Lets not close this until we have a passing green.

@jlewi The test is still failing, I looked at the log and found that the test script no longer matches current deployment name.
https://github.com/kubeflow/kfctl/blob/baf59c2692f45847bbd042c78a751a761b2b7eaa/py/kubeflow/kfctl/testing/pytests/kf_is_ready_test.py#L96-L106
ml-pipeline-viewer-controller-deployment is now called ml-pipeline-viewer-crd. Can I go ahead and update that test? Why is it in kfctl repo? Will there be backward compatibility concerns?

Also we should probably update that list with new deployments in KFP service.

@jlewi friendly ping

@Bobgy Yes please go ahead and update the tests is necessary to work with the current version of KFP. You can consider the code location to be a historical accident. So feel free to move the pipelines code somewhere else if it makes more sense.

I'm not very familiar with how to move tests so I'm going to send a PR to fix tests first.

@Bobgy moving the tests just means

Moving the python code to where every you want it to live
Update the appropriate Tekton task e.g. https://github.com/kubeflow/testing/blob/c37f4bb06e268022f41214d5995df7e738e9800b/tekton/templates/tasks/kf-ready-task.yaml#L67

Updating the tekton task might mean

Adding/updating the resources to include the repository where the code now lives
Updating the working directory for the task
Updating the pythonpath for the task

let me leave the issue open to try moving that python code.
This seems a good issue for me to get better idea of the test infra.

actually, I've tracked the task in https://github.com/kubeflow/pipelines/projects/5.
I think we can close this.
/close


[CI] Enable sccache in docker build and publish ci job

Issue summary

Since we have switched sccache backend to DO space, we can enable sccache in build stage with below changes

[ ] Implement a build arg flag (e.g. SCCACHE_ENABLED=true) for enabling sccache
[ ] Set RUSTC_WRAPPER=sccache when the flag above is on
[ ] Download and install proper prebuilt sccache binaries for amd64-linux and arm64-linux respectively
[ ] Pass sccache s3 env args via build arg

To convert build arg to env var
ARG SCCACHE_ENDPOINT
ENV SCCACHE_ENDPOINT=$SCCACHE_ENDPOINT

To disable/enable sccache
ARG SCCACHE_ENABLED
# https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html
ENV RUSTC_WRAPPER=${SCCACHE_ENABLED:+sccache}

Other information and links

@hanabi1224 Is it the same as https://github.com/ChainSafe/forest/issues/2519? If so, let's close one of them.

@hanabi1224 This is done, right?

@hanabi1224 bump

@lemmih @LesnyRumcajs No I don't think we have enabled sccache in docker build. I can prioritize this task

Do we need it? I mean, is there a compelling reason to have this? What time gains are we talking about?

@LesnyRumcajs  I think the major benefit is that, we could save some $ by switching from buildjet to GH hosted runners

@LesnyRumcajs I believe this has been implemented in https://github.com/ChainSafe/forest/pull/3510 but superseded by the buildjet to fuzzy switch.

We're not running this on fuzzy anymore, given it introduced some nasty race conditions.

If this task had been easy, it would already have been done. Closing as our time is worth more than what we spend on buildjet.


Incremental build still rebuilding everything

TypeScript Version:  3.7.x-dev.201xxxxx or 3.6.3

Search Terms: incremental
Code
https://github.com/singerb/ts-incremental-test is a small example repo based on the structure of the private repo I'm working with, which reproduces the problem. Even with typescript@next (originally tested with 3.6.3), I'm seeing the following output:
% rm built/tsconfig.tsbuildinfo
% time ./node_modules/.bin/tsc --listEmittedFiles
TSFILE: <repo>/built/lib/common.js
TSFILE: <repo>/built/lib/library.js
TSFILE: <repo>/built/bin/script.js
TSFILE: <repo>/built/bin/task.js
TSFILE: <repo>/built/tsconfig.tsbuildinfo
./node_modules/.bin/tsc --listEmittedFiles  4.03s user 1.05s system 168% cpu 3.016 total
% time ./node_modules/.bin/tsc --listEmittedFiles
TSFILE: <repo>/built/lib/common.js
TSFILE: <repo>/built/lib/library.js
TSFILE: <repo>/built/bin/script.js
TSFILE: <repo>/built/bin/task.js
TSFILE: <repo>/built/tsconfig.tsbuildinfo
./node_modules/.bin/tsc --listEmittedFiles  4.08s user 0.62s system 161% cpu 2.907 total

tsconfig.json is here: https://github.com/singerb/ts-incremental-test/blob/master/tsconfig.json
Expected behavior:
Subsequent builds are faster, and emit no files if no input files were changed.
Actual behavior:
Subsequent builds take the same time, and are emitting all the output files again. My guess is that something undocumented about combining JS and TS is breaking the dependency checking and causing the incremental build to just redo it all.

Can confirm, second runs are fast and only emit the buildinfo file when I remove the rootDir setting. I'm not sure why I added that when originally setting up this project, and at least in my minimal test example, removing it still puts the built files in the correct places, so maybe I can just remove it for now.


zero-based FoldingRange

from monaco.d.ts:
export interface FoldingRange {
        /**
         * The zero-based start line of the range to fold. The folded area starts after the line's last character.
         */
        start: number;

zero-based is a mistake? asking because Position is 1-based

It's a mistake, should say 1-based, will fix

@akosyakov @aeschli Am I misreading something? Isn't Position zero-based?
https://github.com/Microsoft/vscode-languageserver-node/blob/5f9c993ff38f5c369949aeb359b3e9b178172dbc/types/src/main.ts#L19-L24
https://github.com/Microsoft/language-server-protocol/blob/9d64e8cb88f66832a0b262af1dfc187aa3e6d3d2/specification.md

Never mind, false alarm. I talked to Anton and he explained to me that Monaco is one-based but the LSP is zero-based.
Sorry for the noise!


"The path is not of a legal form"

!MESSAGE RequestMessage
{
  "jsonrpc": "2.0",
  "id": "1",
  "method": "initialize",
  "params": {
    "rootUri": "file:/home/mistria/runtime-EclipseApplication-with-patch/EclipseConEurope/",
    "capabilities": {},
    "clientName": "Eclipse IDE"
  }
}
[...]
!ENTRY org.eclipse.lsp4e 1 0 2017-02-09 12:06:43.923
!MESSAGE NotificationMessage
{
  "jsonrpc": "2.0",
  "method": "telemetry/event",
  "params": {
    "type": "log",
    "message": "Arguments: --stdio,--zero-based-indices,-s,,--hostPID,22622,--dotnet:enablePackageRestore\u003dtrue"
  }
}
[...]
!ENTRY org.eclipse.lsp4e 1 0 2017-02-09 12:06:44.073
!MESSAGE NotificationMessage
{
  "jsonrpc": "2.0",
  "method": "window/logMessage",
  "params": {
    "type": 3,
    "message": "{\"Type\":\"error\",\"Event\":\"error\",\"Seq\":-1,\"Body\":{\"Message\":\"\\n\"}}"
  }
}

!ENTRY org.eclipse.lsp4e 1 0 2017-02-09 12:06:44.074
!MESSAGE NotificationMessage
{
  "jsonrpc": "2.0",
  "method": "telemetry/event",
  "params": {
    "type": "error",
    "message": "Unhandled Exception: System.ArgumentException: The path is not of a legal form.\n   at System.IO.Path.GetFullPath(String path)\n   at OmniSharp.Program.Main(String[] args)\n   at Program.Main(String[] args)\n"
  }
}

Does omnisharp-node-client conform to latest version of the Language Server Protocol? It seems like it's failing at reading the rootUri field and at setting the -s argument for OmniSharp.

The issue seems to be at https://github.com/OmniSharp/omnisharp-node-client/blob/master/languageserver/server.ts#L140 , only rootPath is dereferenced. This method could include additional logic to look for rootUri instead/additionally to rootPath.
I'm not a TypeScript developer so cannot provide a patch at the moment.
I have a demo planned next week for which one I'd like to show Eclipse LSP4E integration with OmniSharp-node-client. Would you be so kind to fix it ASAP so I can have my demo working simultaneously with VSCode language servers and OmniSharp? The LSP has changed in non backward-compatible way on this topic, and the client APIs too; so I don't have a way to properly work around it on client side.

PR https://github.com/OmniSharp/omnisharp-node-client/pull/382 should fix it (not 100% sure though as I'm not familiar with TypeScript and all the npm machinery).

All that seems  fixed since the migration to newer VSCode modules.


Disable debugger module selector when no modules available

@elharo @patflynn this is what this patch makes it look like.

closes #228

Cool. Can we hide the ' please select a module' when there are none to select?

The validation system kind of drops that in (and it toggles the Ok/Attach button enable/disable). Is there a message you would rather have? It'll start to get real hacky if we take it out.

Do we need the PLease select a mopdule label at all? Or maybe it shoudln't be where it is. Instead it should be the default unselectable item in the popup, and is shown only when we're not showing "No debuggable modules found" If you want to do that in a later PR, I'm OK with this PR; but if not I agree withy Patrick that we should hide the label in the case where no module can be selected.

I guess if it's a validation error it should be disabled when the field is
disabled. If it's a label I agree with elliotte that it's redundant and can
be removed.
On Wed, Dec 9, 2015 at 3:59 PM, Elliotte Rusty Harold <
<EMAIL_ADDRESS>wrote:

Do we need the PLease select a mopdule label at all? Or maybe it shoudln't
be where it is. Instead it should be the default unselectable item in the
popup, and is shown only when we're not showing "No debuggable modules
found" If you want to do that in a later PR, I'm OK with this PR; but if
not I agree withy Patrick that we should hide the label in the case where
no module can be selected.
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/gcloud-intellij/pull/305#issuecomment-163390232
.

Yeah, it's a strange message, but it's part of the Validation system built into Dialogs into intellij.  You can either 1. have a message and disable the ok button or 2. return null and enable the ok button.
We don't have direct control over it.  I mean we could write our own validation system, but right now we use it control the attach button's enabled-state.

In that case for this particular case maybe it's better to have reinforce
the message that instructs the user what to do in the case of no debuggable
modules.
something like "Please select a project with debuggable modules"
On Wed, Dec 9, 2015 at 4:11 PM, Appu Goundan<EMAIL_ADDRESS>wrote:

Yeah, it's a strange message, but it's part of the Validation system built
into Dialogs into intellij. You can either 1. have a message and disable
the ok button or 2. return null and enable the ok button.
We don't have direct control over it. I mean we could write our own
validation system, but right now we use it control the attach button's
enabled-state.
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/gcloud-intellij/pull/305#issuecomment-163392913
.

Might be a good place to link to Cloud Debugger docs on deploying
debuggable modules..
On Wed, Dec 9, 2015 at 4:14 PM, Patrick Flynn<EMAIL_ADDRESS>wrote:

In that case for this particular case maybe it's better to have reinforce
the message that instructs the user what to do in the case of no debuggable
modules.
something like "Please select a project with debuggable modules"
On Wed, Dec 9, 2015 at 4:11 PM, Appu Goundan<EMAIL_ADDRESS>wrote:

Yeah, it's a strange message, but it's part of the Validation system
built into Dialogs into intellij. You can either 1. have a message and
disable the ok button or 2. return null and enable the ok button.
We don't have direct control over it. I mean we could write our own
validation system, but right now we use it control the attach button's
enabled-state.
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/gcloud-intellij/pull/305#issuecomment-163392913
.

here's what it looks like now

ship it!
On Wed, Dec 9, 2015 at 4:52 PM, Appu Goundan<EMAIL_ADDRESS>wrote:

[image: image]
https://cloud.githubusercontent.com/assets/1304826/11699999/32424a1c-9e95-11e5-8c0b-3e3b40a2e0c1.png
here's what it looks like now
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/gcloud-intellij/pull/305#issuecomment-163405852
.

is that an lgtm?

lgtm.
On Thu, Dec 10, 2015 at 10:47 AM, Appu Goundan<EMAIL_ADDRESS>wrote:

is that an lgtm?
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/gcloud-intellij/pull/305#issuecomment-163665108
.


[Docs] Create/Update the index list of the CONCEPT section in the spanish docs site

Current State
The spanish docs site doesn't have the CONCEPTS section nor the index page
English page to translate: https://docs.meshery.io/concepts
Desired State
The spanish docs site has the CONCEPT section and the index list of spanish docs in the section

Contributor Guides and Resources

🛠 Meshery Build & Release Strategy
📚 Instructions for contributing to documentation

Meshery documentation site and source

🎨 Wireframes and designs for Meshery UI in Figma
🙋🏾🙋🏼 Questions: Layer5 Discussion Forum and Layer5 Community Slack

Thanks, @shubham-tyagi!

Thanks! @shubham-tyagi


Releasing the model on torch.hub?

Hi @GaoShuang98 thank you for open sourcing your model!
Are there any plans to release the trained model on torch.hub? It is quite simple to do and allows people to use your model with two lines of code, allowing more people to use your model and helping to spread your work!
For example I did it for CosPlace, and the trained model can be automatically downloaded from anywhere without cloning the repo just like this
import torch
model = torch.hub.load("gmberton/cosplace", "get_trained_model", backbone="ResNet50", fc_output_dim=2048)

I'd like to add your model to the VPR-methods-evaluation repo and having the model on torch.hub would greatly help

Hello, thank you for your recognition and constructive suggestions. I am also grateful that you are considering adding our work to the VPR-methods-evaluation repo. I will release the model on torch.hub as soon as possible🚀.

Hi @gmberton , the trained model has been released on torch.hub. I hope this meets your requirements and expectations.
Code to load the ViTb14-mix model from torch_hub:
`
import torch
model = torch.hub.load("GaoShuang98/DINO-Mix", "dino_mix", pretrained=True)
`
Thank you again for your valuable suggestions.

Hey thank you for your quick reply!
I tried to add it to my repo but I'm having two issues:

If I use it on a machine with only one GPU, it says
RuntimeError: Attempting to deserialize object on CUDA device 1 but torch.cuda.device_count() is 1. Please use torch.load with map_location to map your storages to an existing device.
I think the model's weight were saved while on GPU 1, instead of moving them to CPU.

Even when I run the model on a machine with 2 GPUs, I get low results (R@1 68.0 on St Lucia, 74.6 on Pitts30k). I tried with both normalizations of [0.5,0.5,0.5] and ImageNet's, but they both have low results. I resized the images to 224x224 otherwise it breaks. Do you know what the issue could be?

Hello, I conducted a loading test on the hubconf file in DINO-Mix and evaluated the accuracy of the loaded model. The test results indicate that the model loading is normal. For your reference, please see the test log and the hubconf_model_eval. I hope this can help you solve the issues.  Thank you！


Does the framework intend to continue to be updated?

Does the framework intend to continue to be updated?

@ccworld1000 Yes I am working on a newer version but to be honest with very little time at disposal due to other work commitments, I can’t commit to an ETA.
Any specific features you can mention would be useful. Thanks

Does it support FBX < < = > > obj conversion?
Mutual conversion

Whether the library file can be cropped can be customized and imported in pod as needed

@smhk Thanks for the appreciation 👍
Despite the best of intentions, I am unable to find time to make the necessary updates due to other conflicting priorities. However, if you can submit a PR with the necessary changes, I will be more than happy to review and merge it.
Thanks.


fields: add label popup to custom fields

closes https://github.com/zenodo/who-rdm/issues/71

Decided to add a custom cmp https://github.com/zenodo/who-rdm/pull/72


Dependencies update

Coverage decreased (-0.002%) to 97.65% when pulling 3fa751f0549c8a5671c3bcf3732886ce83690ebc on dev into 9a7f821d001bf7ad4404754778fd492681ed7f75 on master.


EnsembleMPGOS

@FerencHegedus @nnagyd have been putting a lot of work into tuning ensemble GPU parallel CUDA kernel explicit solvers for non-stiff ODEs so we should probably should make use of that. Our GPU stack can be used to generate device code from Julia functions, so that fixes what would probably be the most major hurdle.  @FerencHegedus, could you help suggest a way to wrap this, like what level should we interact with the library that would be most beneficial? I think we can have a fixed driver code a la
https://github.com/FerencHegedus/Massively-Parallel-GPU-ODE-Solver/blob/master/Tutorials_SingleSystem_PerThread/T2_Lorenz/Lorenz.cu
that we pass things into, and then generate and compile the system code
https://github.com/FerencHegedus/Massively-Parallel-GPU-ODE-Solver/blob/master/Tutorials_SingleSystem_PerThread/T2_Lorenz/Lorenz_SystemDefinition.cuh
@vchuravy might need to help out here, and we might need to get MPGOS onto binary builder for this to work.

Hi @ChrisRackauckas, I am not 100% sure what you mean to interact with "the library": to fuse Julia with MPGOS or you want to build-up your own solver/library with Julia?
In any case, one should generate a single monolithic kernel, otherwise, the code has to load everything from the global memory in every kernel launch. If you do it every time step, it is a huge amount of overhead.
One of the things that were really beneficial for me is to load EVERYTHING into registers (even if I have complex ODE like the Keller-Miksis equation and have some amount of register spills). The register spill effectively means load from global memory, but I have to load the variables from global memory anyway. In case of the simple Lorenz system, every used data fit into the registers; however, if I do not pre-load these values explicitly into registers, the compiler does not do it automatically and cannot generate highly optimised code. The benefit of this technique was a huge simplification of the memory address calculations, and I saved tons of integer calculations. The performance increase was also significant. However, I did not check that the previous code was saturated by the integer peak throughput or not. Also, the code became more clear. Later, I realised that the restrict qualifier to my pointers might have been solved this issue, but I have already rewritten my code.
To be able to do the above thing, and allocate arrays in the device code, the array size has to be known at compile time. That is why I used template parameters. But the consequence is that you cannot use a static driver code, and a problem depended on system code for ODE functions. They must be compiled together.
Some years ago, I tested a case where I had the driver code and the ODE functions in different C++ modules. Everybody says that now the compilers can perform "intermodul" optimisations, but still I lost 30% of my performance even in the simple Duffing oscillator. This is another reason for compiling everything together.
I think playing with block sizes plays a minor role in problems we are talking about. What it is more important to regulate the register spills, and with it, the occupancy. However, using a different number of max registers, the code has to be recompiled. Since you want to metaprogram the solver from Julia, you can easily vary this as a parameter and suggest some optimised setup.
You are already in this path, but I cannot stress enough that the asynchronous treatment of the systems is important, not only from the performance point of view. You simply cannot solve certain problems without it.
Okay, this was a bit lengthy, but I hope I could help a bit to choose a good direction for you.
What is the language of the code you generate in Julia?

In any case, one should generate a single monolithic kernel, otherwise, the code has to load everything from the global memory in every kernel launch. If you do it every time step, it is a huge amount of overhead.

KernelAbstractions.jl already does that.

What is the language of the code you generate in Julia?

It's direct to LLVM with its .ptx backend, though I could target C/C++ if necessary (with some more restrictions).

I am not 100% sure what you mean to interact with "the library": to fuse Julia with MPGOS or you want to build-up your own solver/library with Julia?

We're building an EnsembleAsyncGPU, but that's a different project. This issue is just for making EnsembleMPGOS be a dispatch that generates and solves an ensemble with MPGOS, which in theory should be possible.

Ok, I understand. Yes theoretically, it should be possible. If I can help you with MPGOS, just let me know.

Hi Chris, sorry for the sometimes big lag, the November was terrible in terms of administration and online teaching stuff for me.
Is it still an open topic for you to make MPGOS callable from Julia? If yes, it would be nice to make a kind of interface. However, unfortunately, we have no experience with metaprogramming C++ from other languages (so far). For instance, I cannot see how the full capabilities of MPGOS could be achieved (e.g. overlapping CPU-GPU computations). Therefore, if we decide to start this project, can we ask some help at least from Julia part? And discuss how we should modify the C++ code and start this whole project.
Thanks a lot in advance.

Yes, I don't think we plan to tackle it immediately, but it's something to leave open and find someone interested in for. We can metaprogram C++ from Julia with Cxx.jl which allows for handling templates and all of that. Or we can use ModelingToolkit.jl to get the symbolic form of equations (for a subset of problems) and use that to write the code for MPGOS. Either way, there's a way to make it work, it's not the easiest project, but I'll be looking for people(/students) interested in building such a bridge.

Okay, when you feel we should start this just let us know.

Not necessarily needed anymore as the new EnsembleGPUKernel matches the performance by doing a similar kernel construction. See https://github.com/utkarsh530/GPUODEBenchmarks for details. Thanks for your benchmarks, it set us on the right path.

Hi Chris, nice work, I dowloaded tha arxiv paper yesterday, not have to read yet though. If you send me the raw data points, I will change the figure at www.gpuode.com.

@utkarsh530

Hi,
The data can be found here: https://github.com/utkarsh530/GPUODEBenchmarks/tree/main/paper_artifacts/data. You can just either Tesla V100 or RTX 5000 data points. The times are in milliseconds.


Ensure all pages works

There are many tracks/swagpaths that are simply not implemented. Make sure they follow this form:
[course]
  [h5p-course-item id=""]
  [h5p-course-item id=""]
  ...
[/course]

This refers only to the site online, it does not refer to a local site. Because we are testing the content that lives on the live site. The content on the local  machine does not have this content.

Clicked around a bit to get a random sample, and it looks good to me, so closing!

Ok. Go ahead and close it, I can always edit it if need arises.
Best regards!
Alt email(s):
<EMAIL_ADDRESS>Cell: +254 720 130 432
Thanks!
On Wed, Dec 9, 2015 at 2:38 PM, Mikael Lindqvist<EMAIL_ADDRESS>wrote:

Closed #10 https://github.com/tunapanda/TI-wp-content-theme/issues/10.
—
Reply to this email directly or view it on GitHub
https://github.com/tunapanda/TI-wp-content-theme/issues/10#event-486772389
.


Distributed repository strategy

As an architect I want to have a rough plan of how Paranext can be subdivided into different repositories.
Develop a plan for how to subdivide Paranext into distributed but related github repos.  Setup any additional repos that we know we need now.

@GlennPruitt I believe we did this with #7 . Shall we close?

yes
…………………………………
[image: Paratext logo]
Glenn Pruitt
Paratext Product Manager
@.***
@.***>
On Mon, Mar 6, 2023 at 10:01 AM TJ Couch @.***> wrote:

@GlennPruitt https://github.com/GlennPruitt I believe we did this with
#7 https://github.com/paranext/paranext/issues/7 . Shall we close?
—
Reply to this email directly, view it on GitHub
https://github.com/paranext/paranext/issues/4#issuecomment-1456298089,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AGT7LYZ7ADFIM2JBTYXFDKTW2X34TANCNFSM6AAAAAATV2HSPE
.
You are receiving this because you were mentioned.Message ID:
@.***>


Module compiled with Swift 3.0.2 cannot be imported in Swift 3.1

Module compiled with Swift 3.0.2 cannot be imported in Swift 3.1

I'm very busy at the moment so I would appreciate it if someone can open up a PR for this. Usually build error due to Swift version upgrade can be easily resolved by following the error messages in Xcode.

You get this error because you build the previous Fuzi module in old versions of Xcode and you made an upgrade of Xcode and hence the version of swift is also updated. To solve this, try carthage update Fuzi to rebuild the project?

@Mr-GaoSai Please try as @SamuelZhaoY suggested


Undefined array key 1 in ServeCommand.php

Laravel Version
10.24.0
PHP Version
8.1.13
Database Driver & Version
No response
Description
Sometimes for no reasons the application stops when it is started using php artisan serve with the error
Undefined array key 1 in the ServeCommand.php line 312
https://github.com/laravel/framework/blob/10.x/src/Illuminate/Foundation/Console/ServeCommand.php#L312
It also was reported on stackoverflow
https://stackoverflow.com/questions/74283799/laravel-undefined-array-key-1-php-artisan-serve
Steps To Reproduce
php artisan serve
Undefined array key 1 in the ServeCommand.php line 312
Windows 10

Heya, thanks for reporting.
We'll need more info and/or code to debug this further. Can you please create a repository with the command below, commit the code that reproduces the issue as one separate commit on the main/master branch and share the repository here? Please make sure that you have the latest version of the Laravel installer in order to run this command. Please also make sure you have both Git & the GitHub CLI tool properly set up.
laravel new bug-report --github="--public"

Please do not amend and create a separate commit with your custom changes. After you've posted the repository, we'll try to reproduce the issue.
Thanks!

I guess the issue could be that I upgraded the application from version 6.x to 10.x, but I followed all the instructions

https://github.com/laravel/framework/issues/44282


stratify.path

This new d3.stratify feature imputes internal nodes using the specified path function, which returns slash-separated ids (as per a typical UNIX-y file system). Alternative to #184. Fixes #33.

Would it make sense to accept the path being returned as an array?

Maybe. At the least if we don’t, we should allow escaping of backslashes…
I can try to do the array-based approach. I think we’ll still need some normalization so that "/foo/bar".split("/") and "foo/bar".split("/") are considered equivalent.


Disable prow integration

prow is only used for "chatops" in this repo, so can be disabled:

https://github.com/redhat-cop/org/issues/459

This has been done, closing


Adding option to support extent parameter in imshow

Just wondering what the status of this pull request is? Having this merged in would help me out, but it seems like there hasn't been any activity on it for a long time...


Support Active Directory integration for user authentication

The idea is to integrate Artemis with Active Directory authentication system and potentially LDAP which is more open source. Thus, when a user inserts his credentials the GUI will authenticate the user against an Active Directory server and will assign him the appropriate role (admin or simple user).
If you have more questions on that we can discuss it.

Related to https://github.com/FORTH-ICS-INSPIRE/artemis/pull/105

Please include support for LDAPS as part of this, and/or LDAP+StartTLS.


fairseq 和 hydra-core 之间的依赖冲突问题

I卡依赖版本搭建时遇到报错
ERROR: Cannot install -r requirements-dml.txt (line 7) and fairseq because these package versions have conflicting dependencies.
The conflict is caused by:
fairseq 0.12.2 depends on omegaconf<2.1
hydra-core 1.0.7 depends on omegaconf<2.1 and >=2.0.5
To fix this you could try to:

loosen the range of package versions you've specified
remove package versions to allow pip to attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
fairseq 和 hydra-core版本互不兼容，需要怎么做，可否放开限制？

Try to use pip 24.0:
python -m pip install pip==24.0

Try to use pip 24.0: python -m pip install pip==24.0

Excellent solution


Patch experiment with continuous_train_and_eval

Fixes #103

I also got an evaluation problem on en-de data. but I have trained successfully. It seems it can train, but when it is time to do evaluation, it will be failed. Error info is shown below txt file:
eva.txt

Is it possible that this training schedule affects the performance of the system?
When I used the previous repo, before this fix, I managed to get a BLEU around 15 with a small model (200.000 training sentences, 2000 dev set, 2000 test set, batch size 32, 178.500 training steps). Whenever I have tried to replicate this experiment with the exact same configurations afterwards, I have only managed to get something around 5 BLEU. The only difference in configuration, is this new training schedule..


BigDftCalculation: return exit code if the calculation ran out of walltime

Currently, if the job is killed by the scheduler, the parser doesn't return an error and so the calcjob is marked with exit code 0 indicating a success. It would be good to add an exit code for this. I can recommend to use ERROR_OUT_OF_WALLTIME(400) which multiple plugins have adopted as a standard now. This will then also make it easy to add a process handler to the baseworkchain that will simply restart the calculation in this case. But even without this, it will be immediately obvious that the calculation actually failed and a user is not misled thinking everything was fine

How do other plugins detect timeout? Because it's really scheduler-dependent in my understanding, andshould probably be handled at this level.  We don't have any specific timeout message in BigDFT output (it does not know it's been killed, same as OOM), and the message in stdout depends on the used scheduler/machine, so I wouldn't rely on parsing it, or have each plugin do it.
I see that ERROR_SCHEDULER_OUT_OF_WALLTIME was introduced in aiida-core for slurm a few months ago, but it seems to be not triggered in this case and not used by other plugins? As we use slurm in quantum-mobile, I would expect to have it reported, so that we can eventually catch it and say if we want to restart or not, or just let it display a relevant message.

You basically have a few options:

Rely on the scheduler error parsing that comes with the scheduler plugin. As you already mentioned, since some while, the scheduler plugins can already parse their output and report some exit codes. This is implemented for the SlurmScheduler for the out-of-wallclock error, so that should have triggered in your case. However, we designed it such that a Parser can always decide to parse the information, and if applicable, return a more specific error code. That means, if the Parser returns an exit code, then that overwrites any exit code that the scheduler parser may have set. Since the BigDFTParser always explicitly returns ExitCode(0), this will always override the parser error. Have a look at the documentation on scheduler parser errors, to see how to inspect them in a parser and work with them. The only thing that surprises me is that I think in any case a log message should be recorded if the scheduler parser detects a problem, so there should be a trace of it in the logs even if it gets overwritten by the parser. I will perform some tests to see what the problem is here.
Alternatively, if the code supports it, you can set a soft limit that is lower than the requested walltime from the scheduler, which when exceeded will cause the code to shut down nicely, writing all necessary info to disk, such that it can be restarted from. This is what we do for Quantum ESPRESSO: it allows setting an input parameter max_seconds which we set to 95% of the scheduler walltime. This is done by the PwBaseWorkChain. This way, if the time is exceeded, the code will stop and print a line the output, which the parsers detects and returns the corresponding 400 exit code.
Despite the soft limit described above, it is still possible that the scheduler kills the job and the output file is not fully written. This is what is happening in your case, and this also can happen for Quantum ESPRESSO. What I do here is make sure that the output file was fully written. At least for QE this is possible as the code will write some line at the end of the run. If this isn't there, it was clear something bad happened. In this case it is possible we cannot detect the exact reason for the failure, but I still have the parser return an exit code in the 300's. These I reserve for critical errors of potentially unknown nature, but that almost certainly mean the desired result was not achieved and most likely it is impossible to restart from this calculation. Is this not possible for BigDFT? Is it impossible to look at the output yaml of a run that was interrupted and see that it was interrupted?

The only thing that surprises me is that I think in any case a log message should be recorded if the scheduler parser detects a problem, so there should be a trace of it in the logs even if it gets overwritten by the parser. I will perform some tests to see what the problem is here.

The reason this didn't get caught is because the scheduler parser output feature is only triggered when the detailed job info is available, which for SLURM is only possible when the accounting option is enabled. This is not the default for an installation as it requires more setup and databases, and so Quantum Mobile does not have this. The reason it needs that info and doesn't just parse it from the stderr, is because it is not sure that the format of the error messages in the stderr are consistent across versions and so the parsing may be very unstable.

Sometimes, log is still valid if only the cleanup part was cut, that was the reason why I didn't put a crash in the first place, to still provide outputs if possible.. But that was a bad idea, indeed.

If you can parse useful information and attach it as output nodes, I would always do this. This doesn't prevent you from returning a non-zero exit code afterwards. This is what I do for aiida-quantumespresso. Note that a non-zero code doesn't even necessarily mean that the calculation "failed". I have some codes that merely indicate some special edge case that occurred but the calculation and its results can be used in many cases just fine. The exit code is just there to easily distinguish these cases and allow anyone that needs or wants to react to it, do so easily.

But indeed this could be better done at the scheduler level, even if naive and imperfect (It's better to catch some and allow them to restart than to catch none and prevent restarting, even if not perfect, I think).

When I implemented the scheduler parsing in aiida-core I actually stayed away from parsing the text in the scheduler stdout and stderr. Experience teaches that parsing from normal text without formal structure is very fragile and error prone. For that reason, I decided that for now we only parse from the official status returned by sacct which uses error codes for the various exit modes, such as OOM and OOW. For Quantum ESPRESSO this was fine, since we had the soft wallclock mechanism we had very few "hard" OOW by the scheduler. But I see the point for other codes that might not have it. I will open an issue on aiida-core to raise your idea of adding stdout and stderr parsing and see if people agree that we add this.


Update/Add Wine

Tool information

Tool name: Wine
Add or update? Add
Desired version: 5.0 (latest stable)
Approximate size:
If this is an add request:

Brief description of tool: Run windows executables on linux
URL for tool's homepage: https://www.winehq.org/

Area for Triage:
Packages
Question, Bug, or Feature?:
Feature
Virtual environments affected

[x] macOS 10.15
[x] Ubuntu 16.04 LTS
[x] Ubuntu 18.04 LTS
[ ] Windows Server 2016 R2
[ ] Windows Server 2019

Can this tool be installed during the build?
Unsure.
Are you willing to submit a PR?
No.

In order to install Wine in runtime, need to use the following steps:
## MacOS
brew cask install wine-stable

## Ubuntu
sudo dpkg --add-architecture i386
wget -qO - https://dl.winehq.org/wine-builds/winehq.key | sudo apt-key add -
sudo add-apt-repository ppa:cybermax-dexter/sdl2-backport
sudo apt-add-repository "deb https://dl.winehq.org/wine-builds/ubuntu $(lsb_release -cs) main"
sudo apt install --install-recommends winehq-stable

Installation takes about 1,5-2 minutes.

https://github.com/actions/virtual-environments/issues/743#issuecomment-616349857
This works, thank you very much, you saved my day!

This stopped working within the last few days, have there been any changes to the virtual environments that could cause this?
I've narrowed down the install-chain to the following:
The following packages have unmet dependencies:
 wine-stable-i386:i386 : Depends: libgphoto2-6:i386 (>= 2.5.10) but it is not going to be installed
                         Depends: libxml2:i386 (>= 2.9.0) but it is not going to be installed
                         Recommends: libodbc1:i386 but it is not going to be installed
                         Recommends: libsane:i386 or
                                     libsane1:i386 but it is not going to be installed
                         Recommends: libxslt1.1:i386 but it is not going to be installed

cc @Darleev , this was on ubuntu 18.04, started may 2nd.

@kahuang @Sv443 @martmists I'm not completely sure, but It seems official wine repository is broken or something like this, since I tried to install on clean Ubuntu 18, but with no luck, I also tried to install all required libs etc, but it does not help as well.
I have found another way to install wine-5 on ubuntu, you can find it below:
sudo dpkg --add-architecture i386
sudo apt update
wget -qO- https://dl.winehq.org/wine-builds/winehq.key | sudo apt-key add -
sudo apt install software-properties-common
sudo apt-add-repository 'deb http://dl.winehq.org/wine-builds/ubuntu/ bionic main'
wget -qO- https://download.opensuse.org/repositories/Emulators:/Wine:/Debian/xUbuntu_18.04/Release.key | sudo apt-key add -
sudo sh -c 'echo "deb https://download.opensuse.org/repositories/Emulators:/Wine:/Debian/xUbuntu_18.04/ ./" > /etc/apt/sources.list.d/obs.list'
sudo apt update
sudo apt-get install --install-recommends winehq-stable

Please let us know in case of any questions, We will be glad to assist you further.

I'm using your solution https://github.com/actions/virtual-environments/issues/743#issuecomment-616349857 for a GitHub Actions runner on the latest Ubuntu and haven't had any problems

@Darleev Would it be possible to re-open this and have Wine installed by default?
It takes 6 minutes for us on average (about half the time of the total build time.)

@miketimofeev Thank you for the explanation. That's still a relief :)


🐛 [cloud_firestore] Hot reload breaks local emulator on web

Bug report
When configuring the emulator on web, a hot reload crashes Firestore because its settings cannot be changed after initialization.
I have tried numerous ways to try and detect if Firebase was already initialized, but I cannot seem to find a way to get it properly fixed. It also feels very hacky.
I'm simply using the configuration for the emulator as described in the docs.
Possibly the docs just need to be a bit more precise on how to implement it? Furthermore, the usage of the emulator for firestore is quite different from the simple useEmulator() functions for the other packages.
Steps to reproduce
Steps to reproduce the behavior:

Create a new flutter project with web support
Install cloud_firestore
Set up the emulators
Configure the flutter app to use the emulator
Make some hot reload changes, notice the "Firestore has already been started and its settings can no longer be changed. You can only modify settings before calling any other methods on a Firestore object."

Expected behavior
A hot reload should simply accept my settings again
Sample project
https://github.com/Vluf/cloud_firestore_web_emulator_bug

Additional context

Flutter doctor
Run flutter doctor and paste the output below:
Click To Expand
Doctor summary (to see all details, run flutter doctor -v):
[√] Flutter (Channel stable, 2.0.4, on Microsoft Windows [Version 10.0.19042.985], locale en-US)
[!] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    X Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/windows#android-setup for more details.
[√] Chrome - develop for the web
[√] Android Studio (version 4.1.0)
[√] IntelliJ IDEA Ultimate Edition (version 2021.1)
[√] VS Code (version 1.56.0)
[√] VS Code, 64-bit edition (version 1.26.1)
[√] Connected device (2 available)

! Doctor found issues in 1 category.

Flutter dependencies
Run flutter pub deps -- --style=compact and paste the output below:
Click To Expand
Dart SDK 2.12.2
Flutter SDK 2.0.4
cloud_firestore_web_emulator_bug 1.0.0+1

dependencies:
- cloud_firestore 2.2.0 [cloud_firestore_platform_interface cloud_firestore_web firebase_core firebase_core_platform_interface flutter meta]
- cupertino_icons 1.0.3
- firebase_core 1.2.0 [firebase_core_platform_interface firebase_core_web flutter meta]
- flutter 0.0.0 [characters collection meta typed_data vector_math sky_engine]

dev dependencies:
- flutter_test 0.0.0 [flutter test_api path fake_async clock stack_trace vector_math async boolean_selector characters charcode collection matcher meta source_span stream_channel string_scanner term_glyph typed_data]

transitive dependencies:
- async 2.5.0 [collection]
- boolean_selector 2.1.0 [source_span string_scanner]
- characters 1.1.0
- charcode 1.2.0
- clock 1.1.0
- cloud_firestore_platform_interface 5.1.0 [collection firebase_core flutter meta plugin_platform_interface]
- cloud_firestore_web 2.1.0 [cloud_firestore_platform_interface firebase_core firebase_core_web flutter flutter_web_plugins js]
- collection 1.15.0
- fake_async 1.2.0 [clock collection]
- firebase_core_platform_interface 4.0.1 [collection flutter meta plugin_platform_interface]
- firebase_core_web 1.1.0 [firebase_core_platform_interface flutter flutter_web_plugins js meta]
- flutter_web_plugins 0.0.0 [flutter js characters collection meta typed_data vector_math]
- js 0.6.3
- matcher 0.12.10 [stack_trace]
- meta 1.3.0
- path 1.8.0
- plugin_platform_interface 2.0.0 [meta]
- sky_engine 0.0.99
- source_span 1.8.0 [charcode collection path term_glyph]
- stack_trace 1.10.0 [path]
- stream_channel 2.1.0 [async]
- string_scanner 1.1.0 [charcode source_span]
- term_glyph 1.2.0
- test_api 0.2.19 [async boolean_selector collection meta path source_span stack_trace stream_channel string_scanner term_glyph matcher]
- typed_data 1.3.0 [collection]
- vector_math 2.1.0

Thanks for the detailed report and code sample. Replicable using latest stable and latest plugin version.
Firestore has already been started and its settings can no longer be changed. You can only modify settings before calling any other methods on a Firestore object.

 flutter doctor -v
dhs@Dhss-MacBook-Pro ncflutter % flutter doctor -v
[✓] Flutter (Channel stable, 2.2.0, on Mac OS X 10.15.4 19E2269 darwin-x64,
    locale en-GB)
    • Flutter version 2.2.0 at /Users/dhs/documents/fluttersdk/flutter
    • Framework revision b22742018b (9 days ago), 2021-05-14 19:12:57 -0700
    • Engine revision a9d88a4d18
    • Dart version 2.13.0

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.3, Build version 12C33
    • CocoaPods version 1.10.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.55.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (4 available)
    • AOSP on IA Emulator (mobile) • emulator-5554
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 90.0.4430.212

• No issues found!

@Vluf

Furthermore, the usage of the emulator for firestore is quite different from the simple useEmulator() functions for the other packages.

An issue is opened to track this, https://github.com/FirebaseExtended/flutterfire/issues/5224

The issue may be related to https://github.com/FirebaseExtended/flutterfire/issues/3935

I can confirm that this issue still exists using the latest version of cloud_firestore (2.3.0) and firebasejs (8.6.1 & 8.7.0).

This is still an issue using the cloud_firestore: 2.5.0 and the 8.6.1 of the firebase-firestore.js.
Here is the error with its stacktrace:

console error
FirebaseError: Firestore has already been started and its settings can no longer be changed. You can only modify settings before calling any other methods on a Firestore object.
    at new xr (https://www.gstatic.com/firebasejs/8.6.1/firebase-firestore.js:1:48453)
    at yp.rp._setSettings (https://www.gstatic.com/firebasejs/8.6.1/firebase-firestore.js:1:265710)
    at https://www.gstatic.com/firebasejs/8.6.1/firebase-firestore.js:1:308734
    at Sg.useEmulator (https://www.gstatic.com/firebasejs/8.6.1/firebase-firestore.js:1:309580)
    at firestore.Firestore._fromJsObject.useEmulator (http://localhost:55876/packages/cloud_firestore_web/src/interop/utils/utils.dart.lib.js:221:28)
    at cloud_firestore_web.FirebaseFirestoreWeb.new.useEmulator (http://localhost:55876/packages/cloud_firestore_web/src/write_batch_web.dart.lib.js:747:34)
    at cloud_firestore.FirebaseFirestore.__.useFirestoreEmulator (http://localhost:55876/packages/cloud_firestore/cloud_firestore.dart.lib.js:1378:26)
    at main (http://localhost:55876/packages/bottleshopdeliveryapp/main_web_dev.dart.lib.js:99:50)
    at main.next (<anonymous>)
    at http://localhost:55876/dart_sdk.js:37403:33
    at _RootZone.runUnary (http://localhost:55876/dart_sdk.js:37274:59)
    at _FutureListener.thenAwait.handleValue (http://localhost:55876/dart_sdk.js:32530:29)
    at handleValueCallback (http://localhost:55876/dart_sdk.js:33057:49)
    at Function._propagateToListeners (http://localhost:55876/dart_sdk.js:33095:17)
    at _Future.new.[_completeWithValue] (http://localhost:55876/dart_sdk.js:32943:23)
    at async._AsyncCallbackEntry.new.callback (http://localhost:55876/dart_sdk.js:32964:35)
    at Object._microtaskLoop (http://localhost:55876/dart_sdk.js:37526:13)
    at _startMicrotaskLoop (http://localhost:55876/dart_sdk.js:37532:13)
    at http://localhost:55876/dart_sdk.js:33303:9

Following are the package.yaml firebase packages used in the project:

firebase packages versions
  firebase_core: ^1.5.0
  firebase_messaging: ^10.0.5
  firebase_auth: ^3.0.2
  cloud_firestore: ^2.5.0
  firebase_crashlytics: ^2.2.0
  firebase_analytics: ^8.3.0
  firebase_storage: ^10.0.2
  cloud_functions: ^3.0.1

Additionally, here is the firebase related code in the main function of the project:

code snippet
.
.
.
  await Firebase.initializeApp();

  await FirebaseAuth.instance
      .useAuthEmulator('localhost', AppEnvironment.devFirebaseAuthPort);
  final host = defaultTargetPlatform == TargetPlatform.android
      ? '<IP_ADDRESS>'
      : 'localhost';
  FirebaseFirestore.instance
      .useFirestoreEmulator(host, AppEnvironment.devFirestorePort);
  FirebaseFunctions.instance.useFunctionsEmulator(
    'localhost',
    AppEnvironment.devFunctionsPort,
  );
.
.
.

I have the same problem

If this is helpful in any way, in my project, I've been restarting for a couple days with no issues, and then suddenly it started happening out of the blue.  In my case:
Main:
void main() {   WidgetsFlutterBinding.ensureInitialized();   SystemChrome.setPreferredOrientations([DeviceOrientation.portraitUp, DeviceOrientation.portraitDown]).then((_) {     var configuredApp = AppConfig(       appTitle: '...',       photoLocation: '...',       child: App(),     );     setupLocator();     Stripe.init(...);     return runApp(configuredApp);   }); }
App:
`class App extends StatelessWidget {
final Future _initialization = Firebase.initializeApp();
@override
Widget build(BuildContext context) {
return FutureBuilder(
future: _initialization,
builder: (context, snapshot) {
if (snapshot.hasError) {
return LoadingStart();
}
if (snapshot.connectionState == ConnectionState.done) {
FirebaseFirestore.instance.settings = const Settings(persistenceEnabled: false);
return MultiProvider(
providers: [...],
builder: (context, snapshot) {
return MyApp();
});
}
return LoadingStart();
},
);
}
}`
         FirebaseFirestore.instance.settings = const Settings(persistenceEnabled: false);
My log points me to the FutureBuilder in App re-initializing:  Firebase.initializeApp();
All this is probably obvious to you, but I figure anything that might help or trigger something.

@jbryanh My app initializes Firebase in main(). The issue still occurs.

@jbryanh My app initializes Firebase in main(). The issue still occurs.

I figured I'd share my configuration since I didn't start getting the error on restarts until I got a User.  In my migration from mobile to web, I had to work through authentication issues for a couple days, I was never restarting with a User...so I could restart with no issues.  As soon as I was reloading with a User, I started to get the issues.  Maybe the User isn't relevant.  Just figured I'd share the experience.
Additionally, I get the error on hot reload (cmd-s) and restart (Android Studio Play Button)...which feel like the same behavior in Web anyway, I'm not sure.  I have to stop the chrome emulator and restart entirely every time.

Yeah the web does not have Hot Reload. It's hot restart only.

I removed my answer here, as my problem isn't the same as this thread, though the error was the same.  In my case, I set
FirebaseFirestore.instance.settings = const Settings(persistenceEnabled: false);
For mobile this is fine, but for web, this creates the error.  Making this conditional only for mobile fixed my issue.

Can confirm the issue still occurs with the latest versions: cloud_firestore: ^3.1.6 and firebasejs/8.10.0. @darshankawar, any updates? From what I can tell:

Emulator initialization works the first time, but after using Firestore to get data, subsequent restarts break.
But this is happening on a hot restart (not reload), which should start back at main, reproducing the circumstances of the first flutter run.
With the new Dart-only initialization, I was able to clear all Firebase-related code from web/index.html, but this error still happens.
FirebaseAuth.instance.useAuthEmulator does work
Wrapping FirebaseFirestore.instance.useFirestoreEmulator in a try/catch seems to be a valid workaround since the first call does succeed, and future calls are not necessary.

@Levi-Lesches How does the try catch work? The error is thrown in the browser. Dart does not register it as an error.

Does the above code snippet not work? Are you able to print code? Assuming you follow the instructions for Dart-only initialization on web over at firebase.flutter.dev, and you're on the latest versions of flutter and all the firebase plugins, the error should be passed through to Dart. As I noted, it's not a FirebaseException like it should be, but rather a (Dart) class called JavaScriptObject that has the same properties (at least on Chrome). If you try catching it with on, it won't work, you have to use a blanket-catch.
Also, flutter isn't the best at catching asynchronous errors, so make sure you're on the latest version of flutter and that your function is being awaited properly if you made it async.

@Levi-Lesches
It works like a charm. Thank you, it would save me hours.


feat: rate modifiers and rules

For implementing this, we require:

[x] windingtree/win-stays#99
[x] #45
[x] #46

This is complete, with the exception of the tax information. As we take that all the prices quoted are going to be inclusive of all applicable taxes (excluding those that are levied at the point of arrival) - then any tax information here would be for 'guidance' / transparency purposes, which is more of a question for the stays-models (ie. Facility metadata).


Importer should not be case sensitive for column names

Giovanny tried to import a file with a column header named 'Title', but it only worked when it changed to 'title'.

Closing because of inactivity, reopen if needed.


clamd 2.0.0-rc1 crashes on initial block download

I've been switching between versions of clamd and somehow corrupted my blockchain.
I ended up deleting all the block indexes and trying to start again, using the existing blk000n.dat files as a bootstrap.dat. I made a bootstrap.dat file like this:
$ cd ~/.clam/blocks
$ l
total 1188452
-rw------- 1 chris chris 814304490 Jan 13  2016 blk00000.dat
-rw------- 1 chris chris 134215726 May  9  2016 blk00001.dat
-rw------- 1 chris chris 134217444 Oct  3 10:18 blk00002.dat
-rw------- 1 chris chris 134217728 Jan 14 14:24 blk00003.dat
$ cat blk* > ../bootstrap.dat
$

then left clamd processing the file overnight.
In the morning clamd was dead. I tried restarting it, bu after skipping all the blocks it had already processed:
2017-01-18 16:18:22 ERROR: ProcessBlock() : already have block<PHONE_NUMBER>fc1365a4c917f3d4117b192e5ee2c403e0948349f25bc9360985bb685d3f
2017-01-18 16:18:22 ERROR: ProcessBlock() : already have block 877066 d5b31f0bb0b5b5528a68da9131fb899fc33734ee87393403e7f5c7f5e0174fc1
2017-01-18 16:18:22 ERROR: ProcessBlock() : already have block 877067 f0ce35c826a086f78ab9514dd58e8d2c807d90d04b63b34a47ffb52279ec5a63
2017-01-18 16:18:22 ERROR: ProcessBlock() : duplicate proof-of-stake (COutPoint(6155188d80, 1),<PHONE_NUMBER>) for block 12d580f986f86ed81726396d1bf099a1926837ec2ee387a091c9be861401acd8
2017-01-18 16:18:22 ProcessBlock: ORPHAN BLOCK 0, 0 MB out of 41943040, prev=12d580f986f86ed81726396d1bf099a1926837ec2ee387a091c9be861401acd8
2017-01-18 16:18:22 ProcessBlock: ORPHAN BLOCK 0, 0 MB out of 41943040, prev=960701219e976360f80100220b50c4031b789842cf1f7a324ae1ea247e2685e7
2017-01-18 16:18:22 ProcessBlock: ORPHAN BLOCK 0, 0 MB out of 41943040, prev=1cc1778ea8791bc35c2bdad1a465c73206900bebdb050d126e1a2fc186f0fd05
2017-01-18 16:18:22 ProcessBlock: ORPHAN BLOCK 0, 0 MB out of 41943040, prev=1a43138e255e62c6d12ada7fa4eadf1f9fab10fda3083b0eabfefa47329773cc

I don't know what was going on there. All the following blocks were seen as 'orphan block 0'.
I stopped it, moved the bootstrap.dat file out of the way, restart it, let it connect to a peer, and that got it past the sticking point. Stopped it again, put the bootstrap.dat file back in place, and it was able to continue loading from it, until:
ProcessBlock() : parent block was previously rejected because of stake duplication. Reaccepting parent

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffe75c3700 (LWP 5692)]
CheckStakeKernelHash (pindexPrev=0x0, nBits=453022402, blockFrom=..., nTxPrevOffset=150, txPrev=..., prevout=..., nTimeTx=1457590624, hashProofOfStake=...,
    targetProofOfStake=..., fPrintProofOfStake=false) at kernel.cpp:401
401	    if (IsProtocolV2(pindexPrev->nHeight+1))
(gdb) where
#0  CheckStakeKernelHash (pindexPrev=0x0, nBits=453022402, blockFrom=..., nTxPrevOffset=150, txPrev=..., prevout=..., nTimeTx=1457590624, hashProofOfStake=...,
    targetProofOfStake=..., fPrintProofOfStake=false) at kernel.cpp:401
#1  0x0000555555695675 in CheckProofOfStake (pindexPrev=pindexPrev@entry=0x0, tx=..., nBits=453022402, hashProofOfStake=..., targetProofOfStake=...)
    at kernel.cpp:436
#2  0x00005555555d3376 in CheckProofOfStake (pindexPrev=pindexPrev@entry=0x0, pblock=pblock@entry=0x7fffe0f61590, hash=..., state=...) at main.cpp:3040
#3  0x00005555555ecf1b in ProcessBlock (state=..., pfrom=pfrom@entry=0x0, pblock=pblock@entry=0x7fffe75c2c20, dbp=dbp@entry=0x0) at main.cpp:3176
#4  0x00005555555edfef in LoadExternalBlockFile (fileIn=fileIn@entry=0x7fffe0000980, dbp=dbp@entry=0x0) at main.cpp:3925
#5  0x00005555555a8b43 in ThreadImport (vImportFiles=std::vector of length 0, capacity 0) at init.cpp:372
#6  0x00005555555c2bb4 in operator()<void (*)(std::vector<boost::filesystem::path>), boost::_bi::list0> (f=<optimized out>, a=<synthetic pointer>,
    this=<optimized out>) at /usr/include/boost/bind/bind.hpp:253
#7  operator() (this=<optimized out>) at /usr/include/boost/bind/bind_template.hpp:20
#8  boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::vector<boost::filesystem::path, std::allocator<boost::filesystem::path> >), boost::_bi::list1<boost::_bi::value<std::vector<boost::filesystem::path, std::allocator<boost::filesystem::path> > > > > >::run (this=<optimized out>)
    at /usr/include/boost/thread/detail/thread.hpp:117
#9  0x00007ffff753aaea in ?? () from /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.55.0
#10 0x00007ffff5ca40a4 in start_thread (arg=0x7fffe75c3700) at pthread_create.c:309
#11 0x00007ffff59d962d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
(gdb) p pindexPrev
$1 = (CBlockIndex *) 0x0
(gdb)

dooglus stop orphaning my block smh

I was hearing that solo-staking was very hard due to Just-Dice's staking wallet orphaning people's blocks, so I set up an experiment.
I made a new wallet and sent 10,000 CLAMs to it, split into 100 CLAM pieces.
I wanted to see how well it staked, and how much it would be orphaned by the Just-Dice staking wallet.
Here's the wallet so you can check for yourself how it's getting on.
So far it has been staking for 5.73 days. In that time it has staked 57 times, and of those 57 blocks 6 have been orphaned. That's 10.5% of the staked blocks which were orphaned.
The remaining 51 successful stakes in 5.73 days mean that you need 1123.46 CLAM staking in order to stake once per day.
Over the last 5 days Just-Dice has staked 1313.6621 + 1288.7924 + 1323.2905 + 1301.2370 + 1283.4634 = 6510.4454 CLAMs. JD is staking roughly 1575000 CLAMs.
Those numbers suggest that on JD you need 1575000 / 6510.4454 * 5 = 1209.59 CLAMs to stake once per day, and so over the last 5 days it has actually been better to solo stake than to stake at JD. That's not even taking JD's 10% staking commission into account.
This is likely the result of short-term variance. I can't believe that it is more efficient to solo-stake than to use the JD wallet. I'll report back in a week or so.

Thank you @dooglus for always "proving" being trust worthy (even though I was joking before, I still really appreciate your messages)

I posted mostly because I wanted somewhere to log the results of the experiment I'm running. It's out of place in this issue, but whatever. :)

I posted mostly because I wanted somewhere to log the results of the experiment I'm running. It's out of place in this issue, but whatever. :)

I'd suggest adding such information to the wiki here on github, where it will stay available even after this issue is closed.

I've been using this branch for a month on a wallet with no coin in the wallet, and it's not corrupt. The diff compared to master is pretty long, so it's hard to guess where the problem could be.

I'm not entirely sure where the logic error is yet, but the block of code I believe we will find it is @
https://github.com/nochowderforyou/clams/blob/2.0.0-rc1/src/main.cpp#L3075-L3114

I was switching between different wallet.dat files when I saw the corruption. It often seemed to want to reindex (or rescan, I forget) around half the chain when I switched, even though the wallet had recently been used. I don't know if that's related. The time my wallet got corrupted I had done a hard "kill -9" on the clam process because it was going to take hours to do what seemed to be an unnecessary rescan.

It looks to me like the issue could maybe be happening due to this commit: c940461
If you look at the same function in the rat4's master branch here there are some comments which suggest not changing chain data in ram at the loading stage:
    // The memory index structure will be changed after the db commits.```

My hunch is that it's not happening for me because there's no stake transactions in my wallet.

Closing issue, unrelated to client v2.1.0


fix: typehint channels correctly in event classes

…in Events classes
Pull Request Type

[ ] Feature addition
[x] Bugfix
[ ] Documentation update
[ ] Code refactor
[ ] Tests improvement
[ ] CI/CD pipeline enhancement
[ ] Other: [Replace with a description]

Description
This PR is to avoid these types of warnings from IDEs:

(extracted from support server)
Changes

Related Issues

Test Scenarios

Python Compatibility

[x] I've ensured my code works on Python 3.10.x
[x] I've ensured my code works on Python 3.11.x

Checklist

[x] I've run the pre-commit code linter over all edited files
[x] I've tested my changes on supported Python versions
[x] I've added tests for my code, if applicable
[x] I've updated / added  documentation, where applicable

That's some funky pre-commit.

I thought that the pre-commit had an isort in it, does it not? Everytime I PR it changes all imports places lol


Why would tinyxml2 create null entries by mistake?

`

uppercase

Date
Hall Attendant
Car Park
Sound
Platform
Mic Left
Mic Right
Chairman
Watchtower Reader

Assignments - October 2018

Assignments - October 2018
Personal Copy: Damian Gailes

Assignments - October 2018
Personal Copy: Felix Thompson

`

Please delete


Satisfy mypy complaints due to nexus merge

Fix the errors mypy throws due to the recent merger of the nexus/water module.

These errors seem to fall into two broad categories: the classes Annotation and Code from sdmx1 require InternationalStrings to be passed as text arguments. I have not yet worked with them, so I don't know for sure that I kept the intended behaviour. Secondly, several functions from the dataclasses module require arguments to be a DataclassInstance or of Type[DataclassInstance] or both. Curiously, the dataclasses-provided functionis_dataclass doesn't just return True when dealing with a DataclassInstance, but also when dealing with a Dataclass itself. So passing this function is not enough to ensure we're dealing with a DataclassInstance. Again, I've fixed the conditions so that the correct data types are provided (in util/config.py and util/common.py), but I don't know if I kept the intended behaviour. E.g. in config.py, the replace function is the first one to actually require existing to be a DataclassInstance, but maybe that check would also make sense before, i.e. when is_dataclass is called.
How to review
Please check that I kept the intended behaviour.

PR checklist

[ ] Continuous integration checks all ✅

~[ ] Add or expand tests; coverage checks both~ Only type-hint related changes intended

~[ ] Add, expand, or update documentation.~ Only type-hint related changes intended

~[ ] Update doc/whatsnew.~ Only type-hint related changes intended

Per the InternationalString errors, these are upstream. Can you please check with the branch from khaeru/sdmx#117? If that causes them to disappear, we can avoid the respective changes here.


Thread-safe ASE calculators

@tomdemeyere: This now doesn't cause a ph.out not found error, but there's a new Espresso error. The CRASH file reads:

 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
     task #         0
     from davcio : error #        30
     error reading file "./_ph0/pwscf.ebar1"
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

And when I open "./_ph0/pwscf.ebar1", it is blank. Any ideas?

@tomdemeyere: This now doesn't cause a ph.out not found error, but there's a new Espresso error. The CRASH file reads:

 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

     task #         0

     from davcio : error #        30

     error reading file "./_ph0/pwscf.ebar1"

 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

And when I open "./_ph0/pwscf.ebar1", it is blank. Any ideas?

ebar files are not conventional files; they are generated during a calculation and quickly deleted. Which hints me that the previous calculation was copied while running although it shouldn't happen, I can probably try it out and come back to you

Thanks for the info. That is helpful. My guess is that the os.chdir calls are still causing issues because of the multi-threading --- the calculator directory is now unambiguously correct, but the current directory can still fluctuate which could influence things. I'll keep thinking on this. It's possible we might not be able to easily guarantee thread-safety.

Miraculously fixed in #1579 in combination with #1581.


Add dfs maze search

(Put an X inside the [ ] to denote check mark [X].)

[X] If creating a new file :

[ ] added links to it in the README files ?
[X] included tests with it ?
[X] added description (overview of algorithm, time and space complexity, and possible edge case) in docstrings ?

[X] if done some changes :

[X] wrote short description in the PR explaining what the changes do ?
[ ] Fixes #[issue number] if related to any issue

[ ] other

unittest is added
also all global variables are changed to local variables