Refactor time domain PLAsTiCC photometry class

The PR contains refactoring of PLAsTiCCPhotometry class (time_domain_PLAsTiCC.py) and it depends on #52
PLAsTiCCPhotometry and SNPCCPhotometry classes share similar methods. In the future we can have a main class with common methods and inherit PLAsTiCCPhotometry and SNPCCPhotometry from it.

hi @utthishtastro sorry for the late reply on this,
I am getting this error when trying to fit the object_id = 13 from the first file of plasticc_test_lightcurves_01.csv.gz.
can you confirm if this happens to you or not?
[330 rows x 5 columns]
Traceback (most recent call last):
File "fit_PLAsTiCC_pool_time_domain.py", line 48, in
tel_names=['4m', '8m'], feature_method='Bazin', spec_SNR=10)
File "/media/emille/git/COIN/RESSPECT_repo/rupesh/rupesh-env/lib/python3.7/site-packages/resspect-0.1-py3.7.egg/resspect/time_domain_PLAsTiCC.py", line 465, in fit_one_lc
TypeError: 'module' object is not callable

sorry I found out what this is.
It is neessary to install progressbar2 as well, and then it works.
I am closing this issue and will add progressbar2 to the requirements.
sorry for the noise

Hi @emilleishida, only progressbar2 is sufficient. It is already available in requirements.txt file

hi @utthishtastro one more thing,
when I do:
database = PLAsTiCCPhotometry()
database.create_all_daily_files(.....)
database.read_metadata(....)
database.fit_one_lc( ... )
when I create the daily files they are created with header, but once I use the fit_one_lc function the header disappears.

@emilleishida Updated. But please check if the header with cost is proper in master branch.  I think uA uB ut0 utfall utrise YA YB Yt0 Ytfall Ytrise are missing. Please let me know if that is the case, I will update the PR with the correct header for cost

hi @utthishtastro you are correct.
uA uB ut0 utfall utrise YA YB Yt0 Ytfall Ytrise are missing.
also, in the header without cost there is a space missing:
ztriseYA   should be ztrise YA

@emilleishida thanks, updated accordingly.

Last thing, the fit works for DDF and for WFD if I give the number of the file, but if I use vol = None   it does not seem to find it
`Traceback (most recent call last):
File "/media/emille/git/COIN/RESSPECT_repo/rupesh/rupesh-env/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 3080, in get_loc
return self._engine.get_loc(casted_key)
File "pandas/_libs/index.pyx", line 70, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 101, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 4554, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 4562, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'mjd'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "fit_PLAsTiCC_pool_time_domain.py", line 49, in
tel_names=['4m', '8m'], feature_method='Bazin', spec_SNR=10)
File "/media/emille/git/COIN/RESSPECT_repo/rupesh/rupesh-env/lib/python3.7/site-packages/resspect-0.1-py3.7.egg/resspect/time_domain_PLAsTiCC.py", line 464, in fit_one_lc
File "/media/emille/git/COIN/RESSPECT_repo/rupesh/rupesh-env/lib/python3.7/site-packages/resspect-0.1-py3.7.egg/resspect/time_domain_PLAsTiCC.py", line 234, in _get_fit_days
File "/media/emille/git/COIN/RESSPECT_repo/rupesh/rupesh-env/lib/python3.7/site-packages/pandas/core/frame.py", line 3024, in getitem
indexer = self.columns.get_loc(key)
File "/media/emille/git/COIN/RESSPECT_repo/rupesh/rupesh-env/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 3082, in get_loc
raise KeyError(key) from err
KeyError: 'mjd'
`

@emilleishida updated. Please check now.

Now I get the following error
N/A% (0 of 1) |                                                                                | Elapsed Time: 0:00:00 ETA:  --:--:--Traceback (most recent call last):   File "fit_PLAsTiCC_pool_time_domain.py", line 48, in <module>     tel_names=['4m', '8m'], feature_method='Bazin', spec_SNR=10)   File "/media/emille/git/COIN/RESSPECT_repo/rupesh/rupesh-env/lib/python3.7/site-packages/resspect-0.1-py3.7.egg/resspect/time_domain_PLAsTiCC.py", line 476, in fit_one_lc TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

Please remove print from line 172
https://github.com/utthishtastro/RESSPECT/blob/45eeea2cdaa36aa41e7ab875068aa0761e5dbc91/resspect/time_domain_PLAsTiCC.py#L172

hi @utthishtastro I found the problem,
the only issue was that the PLAsTiCC zenodo files start from 1, not 0.
so I just made the starting point of the counter here
https://github.com/utthishtastro/RESSPECT/blob/45eeea2cdaa36aa41e7ab875068aa0761e5dbc91/resspect/time_domain_PLAsTiCC.py#L211
to be 1.
I am accepting this and will make the small change in the main branch.
Thank you deeply for the help on this!


vartable 0.1.0

I am submitting

[ ] a new package
[X] an update for a package

wrong name of the package in readme

I have read and followed the submission guidelines and, in particular, I

[X] selected a name that isn't the most obvious or canonical name for what the package does
[X] added a typst.toml file with all required keys
[X] added a README.md with documentation for my package
[X] have chosen a license and added a LICENSE file or linked one in my README.md
[X] tested my package locally on my system and it worked
[X] excluded PDFs or README images, if any, but not the LICENSE

It is, thank you (and thanks @JamesxX for the review)

thank you both for the reviews and for merging it!


Code for running breakout

Hey Tom,
I saw in the paper that you also applied this method to the Atari Breakout game. I have a looked a bit through the code but in this repo I only find the implementation for the grid games.
Could you also publish the code for the Breakout game?
Thanks in advance!
Manuel

Hi Manual,
Thanks a lot for reaching out. I spent the last half hour trying to dig up the implementation for the Breakout game and unfortunately I haven't been able to find it. I believe the code was on an old laptop. I can't believe that we didn't commit it anywhere, and I'll keep looking for a while longer, but at the moment, it appears lost. This is terrible practice by me and I'm really sorry. If you are aiming to compare against it in a paper, you should of course feel free to say that the experiment was not reproducible. I'm very embarrassed and sorry again!
Sincerely,
Tom

Don't worry man! I was just thinking about using this method to try out an idea I have, and the environment is more similar to the breakout environment than the grid environment. I should be able to implement it myself and this has the advantage of giving me a better understanding of your method. Thanks again and don't feel bad about it. Can happen to anyone! :)


[FancyZones] Filtering popup windows - follow up PR

Summary of the Pull Request
Follow-up for https://github.com/microsoft/PowerToys/pull/28956.
Removing temporary solution for the hotfix, and updating the generally used filter of processable windows.

PR Checklist

[ ] Closes: #xxx
[ ] Communication: I've discussed this with core contributors already. If work hasn't been agreed, this work might be rejected
[ ] Tests: Added/updated and all pass
[ ] Localization: All end user facing strings can be localized
[ ] Dev docs: Added/updated
[ ] New binaries: Added on the required places

[ ] JSON for signing for new binaries
[ ] WXS for installer for new binaries and localization folder
[ ] YML for CI pipeline for new test projects
[ ] YML for signed pipeline

[ ] Documentation updated: If checked, please file a pull request on our docs repo and link it here: #xxx

Detailed Description of the Pull Request / Additional comments

Validation Steps Performed

/azp run


SVG rendering lacks scaling so they become can become blurry

ℹ Computer information

Windows build number: 2004 (19041.388)
PowerToys version: 0.20.0
PowerToy module: File explorer preview

📝 Provide detailed reproduction steps (if any)

Open a explorer on a folder containing SVGs
SVGs should small width & width specified in their source (less than 50)
Change explorer view to large icons

✔ Expected result
SVGs thumbnails are show correctly scaled to icon size.
As SVG can be scaled to any output display size, the SVG canvas should be rescaled prior to display
Note. Removing the width & height attributes but specifying a viewBox resolves the issue, so the module should ignore width & height if a viewBox is present
❌ Actual result
SVGs are extremely blurry
📷 Screenshots
This is a screenshot of the official Azure icon set, which nearly all have a width and height of 18x18

Think we need to think about how we provide scaling here.

I can take a look

Any news? This is still a problem and quite frustrating as it makes the SVG function more or less unusable

@chrdavis, any chance you had a chance to look at this?

Thanks for the reminder.  I will take a look soon.

Would be fantastic and take the SVG function to a new level. Appreciate your efforts.

We'll close the issue after releasing 0.31.

We'll close the issue after releasing 0.31.

Fantastic. Really appreciate your effort. Looking forward to the 0.31 release. Thanks.

Fantastic. Really appreciate your effort. Looking forward to the 0.31 release. Thanks.

Medium:

Large:

Medium:

Large:


Fix nitpick return value

Make that nitpick returns 1 if some errors where detected.
This is more POLA and will helps callers to spot if there was errors in nit files given in arguments.

+1

:shipit:


Use pytest-plugins array to declare fixtures since entry points have been removed

The pytest-jupyter package has removed its use of entry points to register its fixtures since this approach would require that jupyter_server be installed for any packages being tested in environments in which pytest-jupyter has been installed, even though those tests have nothing to do with Jupyter Server.  As a result, it is now necessary to use the pytest-plugins=['plugin-name'] approach to letting pytest know which fixtures are needed.
This PR adds that declaration and addresses a test failure introduced by another recent fixture change relative to the base URL fixture jp_base_url.
Note: All metadata-handler tests will fail until this PR is merged.
Developer's Certificate of Origin 1.1
   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.

Just FYI - it sounds like the pytest-jupyter package will be going away soon (the repo will remain as a place to document various jupyter pytest fixtures).  Instead, they'll be moving the jupyter_server and jupyter_core fixture definitions back to their respective repositories since having them separated was just too difficult to manage during release publications.  Until then, this PR will be necessary to complete CI testing for Elyra 2.0.  I will update our references to the new locations once the change is in place.


invoke-rc.d: could not determine current runlevel

Your Windows build number:  (Type ver at a Windows Command Prompt)
Microsoft Windows [Version 10.0.16299.64]

What you're doing and what's happening:  (Copy&paste specific commands and their output, or include screen shots)
Trying to remove docker.io so I can install Docker CE

Current status:
/mnt/c/Users/spenc
⟩ sudo apt remove docker.io
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
  containerd runc
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
  docker.io
0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
After this operation, 62.7 MB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 30768 files and directories currently installed.)
Removing docker.io (1.13.1-0ubuntu1~16.04.2) ...
invoke-rc.d: could not determine current runlevel
 * Stopping Docker: docker
No process in pidfile '/var/run/docker-ssd.pid' found running; none killed.
invoke-rc.d: initscript docker, action "stop" failed.
dpkg: error processing package docker.io (--remove):
 subprocess installed pre-removal script returned error exit status 1
dpkg: error while cleaning up:
 subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
 docker.io
E: Sub-process /usr/bin/dpkg returned an error code (1)

Tried all manner of solutions on SO, sorry don't have the links
I tried following along to this: https://github.com/Microsoft/WSL/issues/1761
Still get the same output, I tried to ignore and go to the next step and install the .deb file with:
⟩ sudo dpkg -i /mnt/c/Users/spenc/Desktop/docker-ce_17.09.0~ce-0~ubuntu_amd64.deb
dpkg: considering removing docker.io in favour of docker-ce ...
dpkg: docker.io is not properly installed; ignoring any dependencies on it
dpkg: yes, will remove docker.io in favour of docker-ce
(Reading database ... 30768 files and directories currently installed.)
Preparing to unpack .../docker-ce_17.09.0~ce-0~ubuntu_amd64.deb ...
invoke-rc.d: could not determine current runlevel
 * Stopping Docker: docker
No process in pidfile '/var/run/docker-ssd.pid' found running; none killed.
invoke-rc.d: initscript docker, action "stop" failed.
dpkg: error processing archive /mnt/c/Users/spenc/Desktop/docker-ce_17.09.0~ce-0~ubuntu_amd64.deb (--install):
 subprocess installed pre-removal script returned error exit status 1
dpkg: error while cleaning up:
 subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
 /mnt/c/Users/spenc/Desktop/docker-ce_17.09.0~ce-0~ubuntu_amd64.deb

But I'm not getting very far

What's wrong / what should be happening instead: remove the program?

Strace of the failing command, if applicable:  (If some_command is failing, then run strace -o some_command.strace -f some_command some_args, and link the contents of some_command.strace in a gist here)

I can't work out what this means  tried this strace -o apt.strace -f remove docker.io but nothing happenend
See our contributing instructions for assistance.

Ok so I started again with WSL this time avoiding using Linuxbrew, docker.io isn't installed so I skip ahead to installing docker-ce and still get a similar error. Here's the output:
⟩ sudo dpkg -i /mnt/c/Users/spenc/Desktop/docker-ce_17.09.0~ce-0~ubuntu_amd64.deb
(Reading database ... 27427 files and directories currently installed.)
Preparing to unpack .../docker-ce_17.09.0~ce-0~ubuntu_amd64.deb ...
invoke-rc.d: could not determine current runlevel
 * Docker already stopped - file /var/run/docker-ssd.pid not found.
Unpacking docker-ce (17.09.0~ce-0~ubuntu) over (17.09.0~ce-0~ubuntu) ...
Setting up docker-ce (17.09.0~ce-0~ubuntu) ...
invoke-rc.d: could not determine current runlevel
Processing triggers for systemd (229-4ubuntu21) ...
Processing triggers for ureadahead (0.100.0-19) ...
Processing triggers for man-db (2.7.5-1) ...

Anyone able to offer any insight?
Thanks

Ok, it doesn't seem to be supported by WSL:
https://forums.docker.com/t/cant-get-docker-to-start/32314
If one of the contributors could confirm then I'm happy to close this.
Many thanks.

#2288 #2291. Please track progress in those open tickets. #2288 in particular tracks the ask in your OP (though it takes a little remedial German to see).

I've found the solution in #2288, putting it here for wanders from Google.
Edit the file /var/lib/dpkg/info/docker.io.prerm and remove/comment the if check at the end with the invoke command.
if ([ -x "/etc/init.d/docker" ] || [ -e "/etc/init/docker.conf" ]) && \
   [ "$1" = remove ]; then
        invoke-rc.d docker stop || exit $?
fi

However do make sure the docker service is not running sudo service docker stop.
See also jstachowiak's comment in that issue: #2288 (comment)

This fix works


Add a flag to exception handling to prevent returning a broken connec…

…tion to the pool which could be retrieved before the pool is being reset

@zecke That should be the desired behavior, no?

LGTM

thanks. Merged and in release 1.25


implicit conversion from 'std::numeric_limits::type' (aka 'long long') to 'double' changes value from<PHONE_NUMBER>854775807 to<PHONE_NUMBER>854775808

FAILED: velox/functions/sparksql/CMakeFiles/velox_functions_spark.dir/RegisterArithmetic.cpp.o
ccache /usr/local/opt/llvm/bin/clang++ -DBOOST_ALL_NO_LIB -DBOOST_CONTEXT_DYN_LINK -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_PROGRAM_OPTIONS_DYN_LINK -DBOOST_REGEX_DYN_LINK -DBOOST_SYSTEM_DYN_LINK -DBOOST_THREAD_DYN_LINK -DCODEGEN_ENABLED=1 -DFMT_LOCALE -DGFLAGS_IS_A_DLL=0 -I/usr/local/opt/icu4c/include -I/Users/abei/Code/velox/. -isystem /usr/local/include -isystem /Users/abei/Code/velox/velox -isystem /Users/abei/Code/velox/velox/external -isystem /Users/abei/Code/velox/velox/external/duckdb -isystem<EMAIL_ADDRESS>-mavx2 -mfma -mavx -mf16c -mbmi2 -march=native -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -Werror -Wall -Wextra -Wno-unused        -Wno-unused-parameter        -Wno-sign-compare        -Wno-ignored-qualifiers        -Wno-range-loop-analysis -Wno-mismatched-tags -O3 -DNDEBUG -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -std=gnu++17 -MD -MT velox/functions/sparksql/CMakeFiles/velox_functions_spark.dir/RegisterArithmetic.cpp.o -MF velox/functions/sparksql/CMakeFiles/velox_functions_spark.dir/RegisterArithmetic.cpp.o.d -o velox/functions/sparksql/CMakeFiles/velox_functions_spark.dir/RegisterArithmetic.cpp.o -c /Users/abei/Code/velox/velox/functions/sparksql/RegisterArithmetic.cpp
In file included from /Users/abei/Code/velox/velox/functions/sparksql/RegisterArithmetic.cpp:20:
/Users/abei/Code/velox/./velox/functions/sparksql/Arithmetic.h:99:13: error: implicit conversion from 'std::numeric_limits::type' (aka 'long long') to 'double' changes value from<PHONE_NUMBER>854775807 to<PHONE_NUMBER>854775808 [-Werror,-Wimplicit-const-int-float-conversion]
if (arg > std::numeric_limits<int64_t>::max()) {
~ ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
[305/830] Building CXX object velox/functions/prestosql/aggregates/CMakeFiles/velox_aggregates.dir/MinMaxByAggregates.cpp.o
ninja: build stopped: subcommand failed.
make[1]: *** [build] Error 1
make: *** [release] Error 2

similar to https://github.com/facebookincubator/velox/issues/1089

@majetideepak

@xumingming This should be fixed as part of https://github.com/facebookincubator/velox/pull/1240 that got merged yesterday. Can you pull this change and verify?

@majetideepak similar issue, but in different file: functions/sparksql/Arithmetic.h


Stack trace for UndefinedBehaviorError

Scala-js throws UndefinedBehaviorError in some cases. No stack trace is provided, which makes the bug very difficult to solve.
For example:
.../target/scala-2.11/uijs-fastopt.js:176
throw new $c_sjsr_UndefinedBehaviorError().init___jl_Throwable(
scala.scalajs.runtime.UndefinedBehaviorError: An undefined behavior was detected: undefined is not an instance of java.lang.String
It would be useful to get a full stack trace. If it cannot be provided, what would be a good way to debug the issue?

It does have stack traces. But if it reaches the top-level of Node.js, for some reason Node.js does not display it.
If you explicitly try/catch it in some strategic places (e.g., in main()), you can get nice stack traces:
def main(): Unit = {
  try {
    theWholeApp()
  } catch {
    case th: Throwable =>
      th.printStackTrace()
      throw th
  }
}

This is actually a duplicate of #1810.


More efficient EXP sum

You shouldn't use a for loop to sum things when you can avoid it. This is one such case.

I'm not sure why, but my local git shows only a few changed lines. It looks like the line endings got changed somewhere between me and git.

 How is using sum more efficient? Both are O(N).

I assume that's what your did, but again, I can't see it at the moment.

Instead of
exp = 0
for i in range(level):
    exp += self._required_exp(i)

do
exp = self._level_exp(level)

where _level_exp uses the formula for the sum of an arithmetic sequence:
def _level_exp(self, level: int):
    return level*LEVEL_CONSTANT + LEVEL_COEFFICIENT*level*(level-1)//2


Reduce redundancy in enums

Lots of redundancy especially around Response. For the most part I want to be able to drop into the inner struct of the enum variant. Maybe look into enum-dispatch

Notebook.cells is a Vec<Cell> where Cell is an enum of Markdown / code / raw, etc. It's a pain to drill down into Cell.id / Cell.source there.


"IllegalStateException" while using Neo4j

I'm encountering an IllegalStateException when I run the following test app (BlueprintsSidGraphTest.java) with Neo4jGraph. I also tried running the app with OrientGraph and TinkerGraph. With OrientGraph I see a different exception, but TinkerGraph works fine:

import java.util.ArrayList;
import java.util.List;
import java.util.Random;

import com.tinkerpop.blueprints.pgm.TransactionalGraph.Conclusion;
//import com.tinkerpop.blueprints.pgm.TransactionalGraph.Mode;
import com.tinkerpop.blueprints.pgm.*;
import com.tinkerpop.blueprints.pgm.Vertex;
import com.tinkerpop.blueprints.pgm.impls.orientdb.OrientGraph;
import com.tinkerpop.blueprints.pgm.impls.neo4j.Neo4jGraph;
import com.tinkerpop.blueprints.pgm.impls.tg.TinkerGraph;

/** A stress test app that cycles through
 * - adding a certain number of vertices
 * - randomly adding a certain number of edges
 * - randomly removing some vertices
 */
public class BlueprintsSidGraphTest {

	private Graph graph;

	public static void main(String[] args) {
		BlueprintsSidGraphTest gc = new BlueprintsSidGraphTest();
		gc.createRandomGraph();
	}

	public void createRandomGraph() {
		//*** Select graph provider here ***//
		//graph = new OrientGraph("local:./dbs/mydb1");
		graph = new Neo4jGraph("./dbs/mydbn1");
		//graph = new TinkerGraph();

		graph.clear();

		ArrayList<Vertex> vertices = new ArrayList<Vertex>();

		if (graph instanceof TransactionalGraph) {
			((TransactionalGraph)graph).setMaxBufferSize(1000);
		}
		//graph.startTransaction();

		addNVertices(100);

		reloadVertices(vertices);

		removeRandomVertices((List<Vertex>) vertices);

		//graph.stopTransaction(Conclusion.SUCCESS);
		//graph.startTransaction();

		reloadVertices(vertices);

		addRandomEdges((List<Vertex>) vertices);

		//graph.stopTransaction(Conclusion.SUCCESS);
		//graph.startTransaction();

		reloadVertices(vertices);

		removeRandomVertices((List<Vertex>) vertices);

		reloadVertices(vertices);

		addRandomEdges((List<Vertex>) vertices);
		addNVertices(100);

		reloadVertices(vertices);

		addRandomEdges((List<Vertex>) vertices);
		removeRandomVertices((List<Vertex>) vertices);

		//graph.stopTransaction(Conclusion.SUCCESS);
		graph.shutdown();
	}

	public void addNVertices(int n) {
		System.out.print("Adding " + n + " vertices...");
		// add n vertices
		for (int i = 0; i < n; i++) {
			Vertex vv = graph.addVertex(null);
			// set some properties?
		}
		System.out.print("done.\n");
	}

	public void addRandomEdges(List<Vertex> vertices) {
		System.out.print("Adding random edges...");
		int numVertices = vertices.size();
		Random random = new Random();
		// randomly pick pairs of vertices - log n vertices for every vertex
		for (Vertex v1 : vertices) {
			int logN = (int) Math.round((Math.log(numVertices)) / (Math.log(2)));
			for (int i = 0; i < logN; i++) {
				// pick a random vertex
				int randNum = random.nextInt(numVertices);
				Vertex v2 = vertices.get(randNum);
				graph.addEdge(null, v1, v2, "knows");
			}
		}
		System.out.print("done.\n");
	}

	public void removeRandomVertices(List<Vertex> vertices) {
		System.out.print("Removing random vertices...");
		// remove log N vertices
		Random random = new Random();
		int numVertices = vertices.size();
		int logN = (int) Math.round((Math.log(numVertices)) / (Math.log(2)));
		for (int i = 0; i < logN; i++) {
			numVertices = vertices.size();
			if (numVertices == 0) {
				break;
			}
			int randNum = random.nextInt(numVertices);
			Vertex vv = vertices.get(randNum);
			vertices.remove(randNum);
			graph.removeVertex(vv);
		}
		System.out.print("done.\n");
	}

	private void reloadVertices(ArrayList<Vertex> vertices) {
		vertices.clear();
		for (Vertex v : graph.getVertices())
			vertices.add(v);
	}

}

This is the output:
$java BlueprintsSidGraphTest
Adding 100 vertices...done.
Removing random vertices...done.
Adding random edges...Exception in thread "main" java.lang.IllegalStateException: Second node[95] is deleted and cannot be used to create a relationship
	at org.neo4j.kernel.impl.nioneo.xa.WriteTransaction.relationshipCreate(WriteTransaction.java:1358)
	at org.neo4j.kernel.impl.persistence.PersistenceManager.relationshipCreate(PersistenceManager.java:161)
	at org.neo4j.kernel.impl.core.NodeManager.createRelationship(NodeManager.java:312)
	at org.neo4j.kernel.impl.core.NodeImpl.createRelationshipTo(NodeImpl.java:492)
	at org.neo4j.kernel.impl.core.NodeProxy.createRelationshipTo(NodeProxy.java:197)
	at com.tinkerpop.blueprints.pgm.impls.neo4j.Neo4jGraph.addEdge(Neo4jGraph.java:260)
	at BlueprintsSidGraphTest.addRandomEdges(BlueprintsSidGraphTest.java:98)
	at BlueprintsSidGraphTest.createRandomGraph(BlueprintsSidGraphTest.java:54)
	at BlueprintsSidGraphTest.main(BlueprintsSidGraphTest.java:25)

Try with a transaction buffer size of 1 or commit your transaction buffer after you delete random vertices.

That works. Is this expected behavior? If so I'll close the issue.

Yes. Its because you have an open transaction where you created a "deleted vertex." In the same transaction you try and use a vertex that is marked for "deletion."

Makes sense.


UUri without UResource should be supported by all API

Please see the output produced by the code below
void test() {
        UAuthority authority = UAuthority.newBuilder()
                .setName("cloud")
                .setId(ByteString.copyFrom(new byte[] {1 ,2 ,3}))
                .build();
        System.out.println("authority: " + authority);

        UEntity entity = UEntityFactory.fromProto(UTwinProto.getDescriptor().getServices().get(0));
        System.out.println("entity: " + entity);

        Descriptors.MethodDescriptor methodDescriptor = (serviceDescriptor.findMethodByName("GetLastMessages"));
        String methodName = methodDescriptor.getName();
        int methodId = methodDescriptor.getOptions().getExtension(UprotocolOptions.methodId);
        UResource resource = UResourceBuilder.forRpcRequest(methodName, methodId);
        System.out.println("resource: " + resource);

        UUri uri = UUri.newBuilder()
                .setAuthority(authority)
                .setEntity(entity)
                //.setResource(resource)
                .build();

        System.out.println("isLong: " + UriValidator.isLongForm(uri));
        System.out.println("isMicro: " + UriValidator.isMicroForm(uri));
        System.out.println("isResolved: " + UriValidator.isResolved(uri));

        String longUri = LongUriSerializer.instance().serialize(uri);
        byte[] microUri = MicroUriSerializer.instance().serialize(uri);
        System.out.println("long uri: " + longUri);
        System.out.println("micro uri: " + Arrays.toString(microUri));
}

authority: name: "cloud"
id: "\001\002\003"

entity: name: "core.utwin"
id: 26
version_major: 1

resource: name: "rpc"
instance: "GetLastMessages"
id: 1

isLong: false
isMicro: false
isResolved: false
long uri: //cloud/core.utwin/1
micro uri: []

@mishap4 , I thought about this and the code is right, as there is no uResource so we cannot encode it in micro and it is not resolved. isLong() might have to return true but the jury is out on that one...

I believe it should be considered as some default id for resource and isMicro also works. There are several places we use URI to point to entity only.

resource id of 0 means rpc.response topic which is the generic response
topic.
On Wed, Jan 31, 2024 at 3:28 PM mishap4 @.***> wrote:

I believe it should be considered as some default id for resource and
isMicro also works. There are several places we use URI to point to entity
only.
—
Reply to this email directly, view it on GitHub
https://github.com/eclipse-uprotocol/up-java/issues/81#issuecomment-1919901811,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACDANOWX4JIOVFYECLQRBALYRKSPRAVCNFSM6AAAAABCTN5BFCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJZHEYDCOBRGE
.
You are receiving this because you commented.Message ID:
@.***>

@mishap4 can we close this topic?

Now supported by changes in UUri per 1.5.8


[AutoDiff] NFC: separate non-upstreamed tests.

Move non-upstreamed tests on tensorflow branch to test/AutoDiff/downstream.
As tests are upstreamed to master branch in organized test/AutoDiff
subdirectories, corresponding tests in test/AutoDiff/downstream can be
deleted from tensorflow branch.
This patch only moves tests without changing any of them.
Resolves TF-1098.

@dan-zheng minor thing, what do you think of test/AutoDiff/downstream instead of test/AutoDiff/nonupstreamed?

@dan-zheng minor thing, what do you think of test/AutoDiff/downstream instead of test/AutoDiff/nonupstreamed?

I don't feel strongly, but I like the brevity! Renamed to test/AutoDiff/downstream.

@swift-ci Please test tensorflow

@swift-ci Please test tensorflow


MVKCmdBindDescriptorSets: Retain given objects.

Fixes a use-after-free bug when the pipeline layout is destroyed after
recording--e.g. in the
dEQP-VK.api.pipeline_layout.lifetime.destroy_after_end test.

Interesting (and a pain in the butt).
MVKCmdBindDescriptorSetsStatic destructor will need to similarly release the retained _pipelineLayout  and _descriptorSets.

Yep, found the leaks.

Also...would a similar issue of VkPipelineLayout lifetime also affect the following?
MVKCmdPushConstants (although we probably don't even need to be holding a reference to it).

No. I will remove that reference.

MVKCmdPushDescriptorSet
MVKCmdPushDescriptorSetWithTemplate

Yes.
I think I can also dispense with the retains of the MVKDescriptorSets, since VkDescriptorSet objects can't be destroyed until the command buffer is done executing; or otherwise, we'll also need to have them retain their MVKDescriptorPools.


rp.AuthURLHandler: stateFn could provide the current request as argument

Is your feature request related to a problem? Please describe.
The client/rp provides a way to save a state during the login flow. To do this, the rp.AuthURLHandler func accepts a stateFn func() string argument.
However I would like my state to be dependent on the request (to extract some request parameters, path or query).
Describe the solution you'd like
Change the argument to accept a func(*http.Request) string instead.
Describe alternatives you've considered
Wrap the rp.AuthURLHandler into a handler which provides a custom stateFn depending on the request.
Additional context
This would be a breaking change (trivial to fix for the caller, but still breaking, strictly speaking).
To make it non breaking, one could add a new func AuthURLRedirect (or some other name) and deprecate AuthURLHandler:

//AuthURLRedirect extends the `AuthURL` method with a http redirect handler
//including handling setting cookie for secure `state` transfer
func AuthURLRedirect(stateFn func(*http.Request) string, rp RelyingParty) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		opts := make([]AuthURLOpt, 0)
		state := stateFn(r)
		...
	}
}

//AuthURLHandler extends the `AuthURL` method with a http redirect handler
//including handling setting cookie for secure `state` transfer
//
//deprecated: use AuthURLRedirect
func AuthURLHandler(stateFn func() string, rp RelyingParty) http.HandlerFunc {
	return AuthURLRedirect(func(*http.Request) string { return stateFn() }, rp)
}

I would be willing to craft a PR I you think that this is a good idea.

I just realized that state should be used to prevent csrf attacks: https://cs.opensource.google/go/x/oauth2/+/d3ed0bb2:oauth2.go;l=150
In this case, it might make sense to somehow generate and handle it automatically.

Hey @oliverpool
In our case we already have a unique id created and stored in the database, which we then use for the state. The stateFn was therefore was a simple option to provide a slightly more open possibility.
But I see what you try to achieve and passing the http request to the function seems a sensible solution to get the current state of the users in you application (e.g. requested page) before they get redirected to the OP.
I am open to a PR with a concrete proposal 😃
As addition to your second comment from the OIDC spec:

state
RECOMMENDED. Opaque value used to maintain state between the request and the callback. Typically, Cross-Site Request Forgery (CSRF, XSRF) mitigation is done by cryptographically binding the value of this parameter with a browser cookie.

If you pass a cookieHandler, the library will create an encrypted cookie to mitigate such attacks:
https://github.com/caos/oidc/blob/219ba4e038457f9db7877074f6760e53d559ddfc/example/client/app/app.go#L33-L38

In the end, I went using golang.org/x/oauth2.Config.AuthCodeURL(state) directly.

Actually I switched to https://github.com/coreos/go-oidc, because of the IDToken.Claims which allows to unmarshal the user claims directly

Sorry that we could not help. But thanks for the request anyway.


[ARUserManager] Re-enable save creds in iCloud Keychain.

@mennenia I don’t understand the comment, re-enabling it doesn’t show me an alert or something. Was that the reason you had disabled it?

I just installed from scratch (on master, not this PR) and I saw the alert twice (even after accepting it the first time) - perhaps this call here is not needed as it's somewhere else?

When you sign in I don't believe there's any other way this could get triggered, maybe there is if you create a new account? What did you do?

I uninstalled the app, then hit run:

Oh I see, this isn't about showing the saved creds, it's about saving them after entering creds.

Doh 🍡

@alloy it was taken out because @katarinabatina and I couldn't find a way for the pop up to appear gracefully mid onboarding.
The problem:

We now create an account first (before personalisation)
As soon as you create the account in ARUserManager, you have the option to save creds. This results in the sudden popup between account creation and personalisation.
We can't delay account creation, because if it fails, we'd need to display the error to the user straight away (e.g. no internet connection, faulty password, actually tied to Facebook, etc).

So the popup in the beginning that Orta refers to, is for existing people to log in easily. We kept that one.
What I've also noticed is that it seems we save them anyway, because when I test multiple onboarding runs, they have been added to the keychain. Try creating an account, logging out, and seeing if your created account shows up.

Hmm, yeah I see that happening when creating a new account indeed. I was only signing-in with an existing account.
I would really like this re-enabled, though, can you have a think about how else to solve this? e.g. delay saving the credentials, like we do with the push notifications alert?

Yeah I'll have a think. I don't want to bombard the user with too many popups either... but placing keychain outside the context of log in / signup doesn't make sense, nor does moving push notification permission elsewhere, because we want it as soon as possible.
🤔

Bump, would really like this back, I keep having to fill out creds :)

OK, this is now back with https://github.com/artsy/eigen/pull/2709


when to support xpack sql api?

Some, but not all, of XPack API is already supported. See the xpack_*.go files in the repository.

Oh, I misread the title. It was about XPack SQL API... I have no plans to support that for now. But PRs are welcome.

Oh, I misread the title. It was about XPack SQL API... I have no plans to support that for now. But PRs are welcome.

please support XPack SQL API as soon as you can do，thanks a lot！


Bash shell integration -- exit code is not restored when PROMPT_COMMAND is an array

Type: Bug
VS Code version: Code - Insiders 1.71.0-insider (eebdf8174b087979ae6af103f6f2e2f6f9062056, 2022-08-12T05:17:00.323Z)
OS version: Windows_NT x64 10.0.19044
Modes:
Sandboxed: Yes
Remote OS version: Linux x64 4.15.0-169-generic
When the original PROMPT_COMMAND is an array:

exit code of last executed command is not restored before executing the commands stored inside the original PROMPT_COMMAND array
__vsc_status is assigned the exit code of the penultimate command stored inside the original PROMPT_COMMAND array

Those 2 bugs are located in this section of file vs/workbench/contrib/terminal/browser/media/shellIntegration-bash.sh :
__vsc_prompt_cmd_original() {
	__vsc_status="$?"
	# Evaluate the original PROMPT_COMMAND similarly to how bash would normally
	# See https://unix.stackexchange.com/a/672843 for technique
	if [[ ${#__vsc_original_prompt_command[@]} -gt 1 ]]; then
		for cmd in "${__vsc_original_prompt_command[@]}"; do
			__vsc_status="$?"
			eval "${cmd:-}"
		done
	else
		__vsc_restore_exit_code "${__vsc_status}"
		eval "${__vsc_original_prompt_command:-}"
	fi
	__vsc_precmd
}

System Info

Item
Value

CPUs
AMD Ryzen 5 PRO 3500U w/ Radeon Vega Mobile Gfx (8 x 2096)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: disabled_offdirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okskia_renderer: enabled_onvideo_decode: enabledvideo_encode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabledwebgpu: disabled_off

Load (avg)
undefined

Memory (System)
13.93GB (0.50GB free)

Process Argv
--crash-reporter-id 1c1378c2-301b-4d2f-9d58-b675e62c231f

Screen Reader
no

VM
0%

Item
Value

Remote
SSH: VM

OS
Linux x64 4.15.0-169-generic

CPUs
AMD Ryzen 5 PRO 3500U w/ Radeon Vega Mobile Gfx (4 x 2095)

Memory (System)
5.82GB (4.79GB free)

VM
100%

Extensions (2)

Extension
Author (truncated)
Version

remote-ssh
ms-
0.85.2022071315

remote-ssh-edit
ms-
0.80.0

to reproduce:
in your .bashrc or .bash_profile, set your PROMPT_COMMAND to be an array with one of the functions echoing the last command's exit code IE $?:
PROMPT_COMMAND="hi;bye"

hi() {
  echo "last command exit code"
  echo "$?"
  echo 'running first part of PROMPT_COMMAND array'
}
bye() {
  echo 'running second part of PROMPT_COMMAND array'
}

🐛 no matter what you run, for example a non-existent command fjsdkfj, the last command's exit code is always 0

I can reproduce issue 2 though

Friendly ping! Looks like this issue requires some further steps to be verified. Please provide us with the steps necessary to verify this issue.


Migrate to GitHub actions

:tada: This PR is included in version 2.3.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Spurious "SEND OK" in communication

Since we managed to figure out https://github.com/espressif/esp32-at/issues/11 - things work much better, but there is still an issue with the text "SEND OK" appearing sometimes inside the payload, like this:
+IPD,0,40:SEND OK ....
It happens seldomly though, so may be some kind of timing thing. We are fully aware that this is the status response from CIPSEND (page 26 in docs) but we handle those properly and they shouldn't appear inside the payload from IPD. It seems like there are two different threads somehow getting interleaved here.

Thanks for your reporting, we will revise it in the future release.

This makes our current code unstable - since we have no idea when/where this pops up inside the payload. So for us it is fairly important and it is a serious bug. Are you not going to look into it?

Hi gokr, we are trying to duplicate it and debug it now. I'll let you know if there is any update.

Great! I can see if I can get more info from the guy on our side that has been debugging it.

Just verified - when it shows up it seems to always be first in the payload. But it does not affect the size parameter (as expected) so the payload then ends up "too long" so to speak.

I have the same or a very similar problem.
Sending two request directly after another, I get:
0,CONNECT

+IPD,0,306:1,CONNECT

+IPD,1,285:GET /value HTTP/1.1
Host: <IP_ADDRESS>:333
User-Agent: Mozilla/5.0 (Android 5.1.1; Tablet; rv:53.0) Gecko/53.0 Firefox/53.0
Accept: */*
Accept-Language: de,de-DE;q=0.8,en-US;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Referer: http://<IP_ADDRESS>:333/
Connection: keep-alive

PUT /unblock HTTP/1.1
Host: <IP_ADDRESS>:333
User-Agent: Mozilla/5.0 (Android 5.1.1; Tablet; rv:53.0) Gecko/53.0 Firefox/53.0
Accept: */*
Accept-Language: de,de-DE;q=0.8,en-US;q=0.5,en;q=
Accept-Encoding: gzip, deflate
Referer: http://<IP_ADDRESS>:333/
Connection: keep-alive
Content-Length: 0

In the file esp-idf/components/driver/uart.c the function uart_tx_all uses a mutex, but probably this function is called twice: first for the IPD header, then for the IPD data, so other messages can be placed between header and data.
As the calling code probably is in components/at/lib/libat_core.a, I cannot analyze the problem further without access to the library source code.

Hi @hs1956,we have fixed it, but have not released it on github.

Ok, good! Hoping for a release then :)

@xcguang Any reason we can't get this fix in our hands? My customer is waiting for this and they have a deadline to meet.

Hi @gokr , we have updated it, please have a try.


Lab does not exist (60000)

using unetlab-1.0.0-8
1 create new username (user-a) with role editor
2 login using username 'user-a' and create iol lab "/test-lab/test-1.unl"
3 logout
4 login using username 'admin'
5 rename "/test-lab/test-1.unl" with /rename-lab/test-2.unl"
6 logout
7 login again using username 'user-a'
8 blank screen with error notification "Lab does not exist (60000)"
9 'user-a' cannot do anything (there was no menu / no logout button)

fixed in the next release.
For now fix from cli by creating lab and exiting it properly


Sherpa barebones + route additions [WIP]

What changed

Basic structure of Sherpa/PPE app
Added initial batch of routes

Issues

Fixes #12

What's needed

[x] Complete set of routes
[x] Attention to weird unicode errors when bringing in html (ex. "what are you sourcing?" page)
[x] Style buttons
[x] De-uglifying

Screencap

:+1: feel free to merge


Initial checkin of custom docker configuration option. Addresses #205.

The templating of this code is incomplete pending a discussion of #210.

retest this please

This will not pass pr checks until https://github.com/samsung-cnct/kraken-ci-jobs/blob/master/k2-configs/default-e2e.yaml is updated. I am waiting to get sign off on the config file change here before making the require ci-jobs changes.

Note that this change must be checked in at the same time as kraken-ci-jobs/#165

Given your comment about air-gapped installations, should we change the k2 config from      url: "https://s3-us-west-2.amazonaws.com/samsung-cnct/artifacts/docker/v1.12.x/docker-1.12.6%2B7ab89465.tgz" to       scriptUrl: "https://s3-us-west-2.amazonaws.com/samsung-cnct/artifacts/install-artifact.sh"      binaryUrl: "https://s3-us-west-2.amazonaws.com/samsung-cnct/artifacts/docker/v1.12.x/docker-1.12.6%2B7ab89465.tgz" binaryUrl would still be required when type=tgz but scriptUrl could be optional, defaulting to us if not specified. One thing I don't like about this is that it means the contents of scriptUrl would have to conform to the arguments expected by install-artifact.sh. We improve could the situation by defining installCmd instead of url or *url. Thoughts?

I'd still prefer we have some signature checking but the ssl cert guarantee is good enough for now.

If were going to allow a custom command to specified (via a installCmd option) then I think the containerConfig type should be "custom" and not "tgz".

After dancing around the multiple levels of escaping necessary to include a custom command in the k2 config (YAML -> terraform -> cloud-init -> systemd -> bash) I think we should punt on supporting custom commands for now. Instead we should keep the tgz type and introduce a custom type in the future for airgapped solutions.

One nit:  otherwise, LGTM!


List of government words (and definitions) including DTA ones

What do you need?
https://github.com/AusDTO/gov-au-content-guide/issues/153
Which entry/section is this related to?
http://content-guide.apps.staging.digital.gov.au/terms-phrases/

https://github.com/govau/content-guide/issues/45

Bumping this forward - we have user need to get some specific terms in guidance soon, if not all of them.

Relevant stuff here @libbyv https://sites.google.com/a/digital.gov.au/dtointranet/home/how-do-i-1/understand-dto-words

Here is a round-up of all glossaries that need to be combined into 1: https://docs.google.com/document/d/18p2_kGSnwdpOqax8WDz2uUa7Fl3dPwR--wKFjXyFrNU/edit

Now being tracked in Trello


docs: Add link to youtube channel to website

https://www.youtube.com/@owaspmodsecuritycoreruleset

Closing.


Pros/Cons of using shared Redis client as pub/sub client?

Hello.
Can someone explain pros & cons using multiple Redis clients instead of one shared?
I have an application using Redis storage additionally to socket.io-redis package.
It means that every instance has 3 connections to the Redis DB.
There is an example in the repo README.md:
const redis = require('redis');
const redisAdapter = require('socket.io-redis');
const pub = redis.createClient(port, host, { auth_pass: "pwd" });
const sub = redis.createClient(port, host, { auth_pass: "pwd" });
io.adapter(redisAdapter({ pubClient: pub, subClient: sub }));

can I use the following solution:
const redis = require('redis');
const redisAdapter = require('socket.io-redis');
const redisClient = redis.createClient(port, host, { auth_pass: "pwd" }); // or take existing connection
io.adapter(redisAdapter({ pubClient: redisClient, subClient: redisClient }));

what is the (dis)advantage of doing so?

I think using a shared client will not work, because the subscriber (subClient) is put in a special state:
From https://redis.io/commands/subscribe

Once the client enters the subscribed state it is not supposed to issue any other commands, except for additional SUBSCRIBE, PSUBSCRIBE, UNSUBSCRIBE, PUNSUBSCRIBE, PING and QUIT commands.

From https://github.com/NodeRedis/node-redis#pubsub:

When a client issues a SUBSCRIBE or PSUBSCRIBE, that connection is put into a "subscriber" mode. At that point, the only valid commands are those that modify the subscription set, and quit (also ping on some redis versions). When the subscription set is empty, the connection is put back into regular mode.

Could find that info. Thanks for quick answer.
Closing the issue now.


feat: partial delete form

listen to the delete event queue to handle partial delete in DB
add deleteType and fixedFields in S3 Metadata when new forms are saved
Mongock migration to bulk add deleteType and fixedFields to Metadata of all the existing forms

TIS21-4009
TIS21-4019
TIS21-4077
TIS21-4023

🙈 Oh.. I found that I have misunderstood Sonarcloud, and thought it suggested me to use var after Java10. (lol)
https://rules.sonarsource.com/java/RSPEC-6212
Changed it back now.


Add to Calendar feature

Add to Calendar feature for

Class timetable
Final exam

https://pub.dev/packages/add_2_calendar


[New Resource] NIC Teaming (NetLBFO)

Add Nic Teaming (NetLBFO) to the list of resources.

Here it is: https://github.com/rchaganti/DSCResources/tree/master/cWindowsOS/DSCResources/cNetworkTeam
I just need to update it as per contribution guidelines here and add tests! :D

When I copy this new resource to my fork, what should I name it? Leave it as cNetworkTeam or name it MSFT_xNetworkTeam?

If the intention is to submit a PR to us then it should be renamed MSFT_xNetworkTeam. Though you bring up a good question that I've proposed elsewhere that we should drop the x/c nomenclature. So it would just be MSFT_NetworkTeam instead.  What are your thoughts?

Yes, I will submit a PR here. Given an option, I will remove x or c from it completely.

The x/c isn't really that useful TBH - I thought x was for experimental and c was for community - but it seems like useless information to me. But to remove it now from all resources would mean all configs using them would need to be updated - which might not be ideal.
All my resources use c, but as I'm transferring them across to MSFT I'm changing them to x.
As long as the naming is consistent I'm happy :)

I didn't mean that we change existing resources. We can just drop the x/c prefix for all new resources.

I just noticed that there has been some discussion already around dropping the x/c nomenclature in this issue:
https://github.com/PowerShell/DscResources/issues/27
It looks like the policy was to drop the x/c off the resources once ALL 'c' resources are integrated and all 'x' resources are adhering to the best practices policy.
I'm not certain of what the final plan is currently though as this discussion was during June last year.
I'd suggest we stick with the 'x' for now and when a final instruction is issued we could submit a PR to change all resources. This actually shouldn't be difficult as I've changed some of my resources from 'c' to 'x' without too much difficulty.
@tysonjhayes ?

just a progress update on MSFT_NetworkTeam. I completed the style and code guideline updates to the resource module and added examples. I am adding unit / integration tests and should be able to push the PR early next week.


Feature Request: Stellar exchange usage in the sample demo wallet

What problem does your feature solve?
Asset exchange feature
What would you like to see?
I want to transfer TypeA asset from my side and destination account get it in the form of TypeB asset.
What alternatives are there?

Thats called a path payment.


Fix contact-page render issue#19

Thanks for reading my pull request

Can you include a screenshot of the render page. Thanks.

I wanted to know if this PR will be included in my progress in hacktoberfest?

Yes it will be included for Hacktoberfest. You can find about this PR by authenticating your account on hacktoberfest official website. There you can see progress of this PR.


[usd] Fixes for recent TBB versions and osx-arm64

[x] Changes comply with the maintainer guide.
[ ] SHA512s are updated for each updated download.
[x] The "supports" clause reflects platforms that may be fixed by this new version.
[x] Any fixed CI baseline entries are removed from that file.
[x] Any patches that are no longer applied are deleted from the port's directory.
[x] The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
[x] Only one version is added to each modified port's versions file.

Has this patch been submitted upstream? This is much larger than we normally would want to maintain ourselves.

Has this patch been submitted upstream? This is much larger than we normally would want to maintain ourselves.

Yes. Most of these changes are already merged in the dev branch but for some reason it takes time to make its way to the release.

Is the failing x64_osx job related to my change? It's not clear to me.

Is the failing x64_osx job related to my change? It's not clear to me.

No logs were generated so I've re-ran the test.

Tagging vcpkg-team-review to ask if a patch this big is OK. If the patches do indeed exist in upstream you might be able to avoid this concern by downloading the patches directly from upstream, for example:
https://github.com/microsoft/vcpkg/blob/c4467cb686f92671f0172aa8299a77d908175b4e/ports/nifly/portfile.cmake#L3-L9

Tagging vcpkg-team-review to ask if a patch this big is OK. If the patches do indeed exist in upstream you might be able to avoid this concern by downloading the patches directly from upstream, for example:
https://github.com/microsoft/vcpkg/blob/c4467cb686f92671f0172aa8299a77d908175b4e/ports/nifly/portfile.cmake#L3-L9

See https://github.com/microsoft/vcpkg/pull/39292#discussion_r1642496781
I tried that but the commits are conflicting with the release branch

@AugP
@ras0219-msft
@data-queue
and I discussed this today.
@BillyONeal I am weakly against taking this because I'm concerned about the licensing implications
@ras0219-msft Option 1: We close this and say 'anyone who wants these changes can use what's in this PR until upstream does a release'
Option 2: We just ship the development snapshot with these changes already inside instead.
@ras0219-msft has opinions about how to do versioning if you choose option 2.
Please feel free to choose one or the other and either close it (option 1) or just use a version with it fixed and press 'ready for review' when that's done. Thanks for fixing the port!

I'd rather wait for the next release in this case.
There's another fix in this PR related to macos ARM support, should I create another PR or remove TBB related patches in this one and force push? It's basically just changing !arm to !(windows & arm)

Closing this since it's outdated and the new version is supporting oneTBB now according to the release note.
See https://github.com/microsoft/vcpkg/pull/40225 and https://github.com/microsoft/vcpkg/pull/41864


Support buildMultiTargeting

Example nuget package: xunit.core/2.4.1

Not necessary with the current nuget implementation.


Add .gitignore & clean paths to use relative paths with 'path' utility

I was not able to run the app (without Docker) because of absolute paths in source. There is no need to use absolute paths, relative paths work well with Webpack and Node (the safest way with Node being using path utility to join path parts on any OS). I hope this doesn't break Docker build; I didn't succeed to run Docker bu I'm really not an expert with it, that's why I used plain npm-style 😉

thanks for posting this.  it's important to get the build working for non-docker users.  I'm sure we can get it working for both development environments with relative paths.
I'll test this out as soon as I have some time later today.

this is working great with docker too. :ship:


iOS Build failed

Before submitting a new issue

[X] I tested using the latest release of the library, as maybe the bug has been already fixed.
[X] I checked for possible duplicate issues, with possible answers.

Bug summary
After updating my react-native project to 0.73.4 and installing this package with latest version ("react-native-permissions": "^4.1.2",) my ios build is failed and showing  Error --->Duplicate case value 'EKAuthorizationStatusDenied'. and.  Use of undeclared identifier 'EKAuthorizationStatusFullAccess'

Library version
4.1.2
Environment info
System:
  OS: macOS 12.6.1
  CPU: (8) arm64 Apple M1
  Memory: 365.98 MB / 16.00 GB
  Shell:
    version: 5.8.1
    path: /bin/zsh
Binaries:
  Node:
    version: 18.19.0
    path: /opt/homebrew/bin/node
  Yarn:
    version: 1.22.17
    path: /opt/homebrew/bin/yarn
  npm:
    version: 10.2.3
    path: /opt/homebrew/bin/npm
  Watchman: Not Found
Managers:
  CocoaPods:
    version: 1.14.3
    path: /opt/homebrew/bin/pod
SDKs:
  iOS SDK:
    Platforms:
      - DriverKit 22.2
      - iOS 16.2
      - macOS 13.1
      - tvOS 16.1
      - watchOS 9.1
  Android SDK: Not Found
IDEs:
  Android Studio: 2023.1 AI-231.9392.1.2311.11330709
  Xcode:
    version: 14.2/14C18
    path: /usr/bin/xcodebuild
Languages:
  Java:
    version: 17.0.10
    path: /usr/bin/javac
  Ruby:
    version: 2.6.10
    path: /usr/bin/ruby
npmPackages:
  "@react-native-community/cli": Not Found
  react:
    installed: 18.2.0
    wanted: 18.2.0
  react-native:
    installed: 0.73.4
    wanted: 0.73.4
  react-native-macos: Not Found
npmGlobalPackages:
  "*react-native*": Not Found
Android:
  hermesEnabled: true
  newArchEnabled: false
iOS:
  hermesEnabled: true
  newArchEnabled: false

Steps to reproduce
After updating my react-native project to 0.73.4 and installing this package with latest version ("react-native-permissions": "^4.1.2",) my ios build is failed and showing  Error --->Duplicate case value 'EKAuthorizationStatusDenied'. and.  Use of undeclared identifier 'EKAuthorizationStatusFullAccess'
Reproducible sample code
**Pod file setup**

  def node_require(script)
     # Resolve script with node to allow for hoisting
     require Pod::Executable.execute_command('node', ['-p',
      "require.resolve(
        '#{script}',
         {paths: [process.argv[1]]},
      )", __dir__]).strip
   end

   node_require('react-native/scripts/react_native_pods.rb')
   node_require('react-native-permissions/scripts/setup.rb')

platform :ios, min_ios_version_supported
prepare_react_native_project!

setup_permissions([
  'AppTrackingTransparency',
  'Bluetooth',
  'Calendars',
  'CalendarsWriteOnly',
  'Camera',
  'Contacts',
  'FaceID',
  'LocationAccuracy',
  'LocationAlways',
  'LocationWhenInUse',
  'MediaLibrary',
  'Microphone',
  'Motion',
  'Notifications',
  'PhotoLibrary',
  'PhotoLibraryAddOnly',
  'Reminders',
  'Siri',
  'SpeechRecognition',
  'StoreKit',
])

flipper_config = ENV['NO_FLIPPER'] == "1" ? FlipperConfiguration.disabled : FlipperConfiguration.enabled

linkage = ENV['USE_FRAMEWORKS']
if linkage != nil
  Pod::UI.puts "Configuring Pod with #{linkage}ally linked Frameworks".green
  use_frameworks! :linkage => linkage.to_sym
end

target 'faveo_support_app_master' do
  config = use_native_modules!

  use_react_native!(
    :path => config[:reactNativePath],
    # Enables Flipper.
    #
    # Note that if you have use_frameworks! enabled, Flipper will not work and
    # you should disable the next line.
    :flipper_configuration => flipper_config,
    # An absolute path to your application root.
    :app_path => "#{Pod::Config.instance.installation_root}/.."
  )

  # pod 'RNVectorIcons', :path => '../node_modules/react-native-vector-icons'

  target 'faveo_support_app_masterTests' do
    inherit! :complete
    # Pods for testing
  end

  # # Enables Flipper.
  # #
  # # Note that if you have use_frameworks! enabled, Flipper will not work and
  # # you should disable the next line.
  # use_flipper!()

  # post_install do |installer|
  #   react_native_post_install(installer)
  #   __apply_Xcode_12_5_M1_post_install_workaround(installer)
  # end

  post_install do |installer|
    # https://github.com/facebook/react-native/blob/main/packages/react-native/scripts/react_native_pods.rb#L197-L202
    react_native_post_install(
      installer,

      config[:reactNativePath],
      :mac_catalyst_enabled => false
    )
  end

end

Hi @DeepakSharma04, are you using Xcode >=15, like mentioned in the 4.0.0 release notes?
If yes, just clean your build cache and rebuild, that's it.

Hi @DeepakSharma04, are you using Xcode >=15, like mentioned in the 4.0.0 release notes?
EDIT: You don't

Xcode:
version: 14.2/14C18

Thanks


Use Azure.Data.Tables

Converts the main package and live tests to Azure.Data.Tables.
Some unit tests are excluded and are going to be ported later.
Some code is left commented out. These are features that I haven't ported fully yet.

/azp run net - tables - tests


Support parsing of string with multiple identifiers

I've been doing some work with the RIS format and have had all sorts of weird and worderful interpretations of how to record identifiers for documents.  To support this (as it's not going away in a hurry) I've added a method (StdNum::Helpers. extract_multiple_numbers) for extraction of multiple identifiers based on the StdNum::Helpers.extractNumber.
So what we get is the following:
string = '9780987115423 (print ed)<PHONE_NUMBER>430 (web ed)'
array_with_matches = ['9780987115423', '9780987115430']

Let me know what you think

Hi @billdueber just checking in to see if you've had a chance to look over this PR.  I'm looking to do another one shortly to do support for ISMN (see http://www.ismn-international.org/whatis.html).

Should I be releasing this as 1.5.0, or should I wait for the ISMN patch?

Thanks for putting me on the repo.  No problems with the delay - we all have lots of other things to be doing :)
WRT ISMN I'll be doing some more review today and will let you know.


[TASK] Integrate codecov to manage unit testing coverage

What's the task? Please describe
Let's leverage https://about.codecov.io/ for the unit tests coverage for major longhorn components including manager, instance-manager, engine, share-manager, and backing image manager.
Describe the items of the task (DoD, definition of done) you'd like

[ ] Integrate coverage report to codecov for each merge.

Additional context
Add any other context or screenshots about the task request here.

@innobead wait for https://github.com/longhorn/backing-image-manager/pull/66 being merged

Merged. Just close it if all work. Thanks @yangchiu .

longhorn-manager, longhorn-engine, longhorn-share-manager, longhorn-instance-manager and backing-image-manger are all integrated with codecov now.
The codecov overview page of longhorn-share-manager is empty. It's because there is no any test file in longhorn-share-manager repo, and the go test ended up output an empty test report file.

cc @longhorn/dev


feat: Implement rate limiting

Please describe the feature

As a Mimir maintainer, I want to make sure that users' have reasonable limits over how many tokens they can request from ChatGPT.
Detailed description

[ ] This limits could be enforced in different ways:

API key "K" can generate N tokens
User Account "U" can generate N tokens
IP address "A" can generate N tokens

The number of tokens "N" needs some research before we find a good number. AFAIK the maximum tokens per conversation is 4096. So if we want to allow a maximum 20 conversations per day, perhaps 20*4096 could be a good starting point?
To read more about rate limiting: https://platform.openai.com/docs/guides/rate-limits/overview
Acceptance criteria

[ ] Clients who have used more than N tokens should be stopped from starting conversations with Mimir
[ ] Integration tests should be implemented so that we make sure the limits are correctly in place

no fix


No notification when pylance is uninstalled

Testing #18964
Workspace:

User:

I also tried reloading the window. Am I doing something wrong here?

No notification happens for me in web scenario either.

@meganrogge the value of the python.languageServer setting in your workspace scope looks to be explicitly set to None, which will prevail over the user value. You can either clear this setting from your workspace settings, or set it explicitly to Pylance or Default.

Did the notification ever appear? If not, could you clarify the steps you followed?

I never see a notification - with my workspace setting set to Default or Pylance and same for the user settings, I uninstall Pylance and don't see an otification.

Oh, I see what you mean. Yeah we don't currently display anything right after uninstalling Pylance at the moment.

I think it's ok to not show a notification right after uninstalling Pylance, as I could see this would bother some users. So closing this.


error when analyzing single color images

I have been using this library for analizing stone textures (is the best library I have been able to found for this purpose) and some of them where fake textures generated by computer. Two of the textures I analized where completely smooth, just one gray color. When trying to extract features from this images I got a weird error and at the end the way I solve it was to add 1 to the gray value of the first pixel of the image.
Right now I can't reproduce the error but as soon as posible I will do it and give more information.
Thanks!

@edoelas have you made any progress investigating this issue?
A completely flat texture (i.e. just one gray value) does produce outlier values/errors as the variance is 0, and would lead to 0-division is certain features (mostly, in these cases an arbitrary value is returned)

I just wanted to test if this library suite my needs (and it is a great library, but my needs are really weird) so I just modified the slightly the value of one pixel.
I commented this because if the user is not aware of this problem it may lead to try to find an error that does not exists in its own code. Or even worse, if it is like my case, where I was analyzing three thousand images and two of them for some reason were complete flat, and, in this situation, maybe to my lack of experience, it was hard for me to find the problem.


kratos: Add webhook for updating lastLogin

Description
The user last_login from legacy wasn't migrated (see https://github.com/serlo/frontend/issues/2019): API still gets info from a column that doesn't get updated anymore (and this column should stay so).
We want to migrate it to kratos and for that it sounds simpler just to use a non blocking webhook each time someone logs in, updating metadata_public.lastLogin. Of course, it is here just a dry run, after that we should do the changes in cluster.
What should be changed

https://github.com/serlo/local-dev-env/blob/main/kratos/identity.schema.json: metadata_public receives field lastLogin (it is just for documentation) See https://www.ory.sh/docs/kratos/manage-identities/identity-schema
https://github.com/serlo/local-dev-env/blob/main/kratos/config.yml: add a non-blocking webhook after login. See https://www.ory.sh/docs/kratos/hooks/configure-hooks, which uses a jsonnet file (maybe the same as https://github.com/serlo/local-dev-env/blob/main/kratos/register_legacy.jsonnet)
Add a new endpoint in https://github.com/serlo/api.serlo.org/blob/main/packages/server/src/internals/server/kratos-middleware.ts where lastLogin something (new Date).toISOString().
In api's enpoint a secret check may not be necessary

For testing

Run yarn start:without:api
In api repo run yarn start and do your changes there
Clone frontend repo, follow the readme there, but change .env to use local environment. yarn dev
Head to localhost:3000 and login as one user.
Check db if lastLogin is set and with correct date.

Nice to have

Make a script for importing the last login of existing users to the metadata_public.lastLogin

closed via #8


Drop net 5 support

Do another release and then remove net5. Do it in a separate PR and do a major version bump.
.net 5 support is being dropped middle of next year and it's already causing us issues with packages.

Fixed by #831


questions about the api version

Will the older api versions be deprecated in future? For example, the stable version 2015-06-15 or the preview version 2016-04-30-preview.
To keep stable, we don't want to upgrade the api version in our software if there are no strong reasons. Now, we are using 2015-06-15 for several services (e.g. SRP). What's the benefit of upgrading api version? (stability, performance?) Any specific suggestions to upgrade the api versions? If there are some best practice, maybe it's better to document them.

@sergey-shandar any ideas?

normally we'll suggest to upgrade to lastest version, because latest version contains latest features/functionalities. about old api deprecation, it's a complex process and depending on each azure service plan, there's no general answer on whether some version will be deprecated. Hope this helps. close the issue for now. Feel free to create new one or reopen if any furthur questions.


Marine Life Data Network planning/execution

Repository name
marine_life_data_network
Repository purpose
Planning efforts relevant to IOOS Marine Life and IOOS DMAC.
Repository description
Planning and executing marine life efforts.
Repository type
Public
Repository license
BSD 3-Clause
Repository options

[X] Add a README.md file
[X] Add a .gitignore file

Specify a .gitignore file type
No response
Specify GitHub usernames or IOOS GitHub Team for repository access
mathewbiddle

add some capacity to gather requirements / user needs

Some questions I would like to answer before we proceed:

[ ] Should this simply be a repository to log issues for project management?
[ ] Should we use a project board?
[ ] What might be put in the repository?
[ ] I would like this to be a place where we can gather requirements. Can we use GitHub issue templates to do this?

@laurabrenskelle any thoughts on the above questions? I think we can start with a blank slate and work from there...
Mainly I'm curious about the name of the repo marine_life_data_network. Does that make sense.

@laurabrenskelle I've been thinking... What different purpose would this marine life repo serve, that https://github.com/ioos/mbon-docs is not serving now? I'd really like to get away from categorizing the DMAC work we are doing as "MBON". Instead it is IOOS Marine Life. Maybe we should just rename that repository?
If we do a rename to the repository, what happens to the ioos.github.io/mbon-docs website? What would need to change and where?
I think the project board could be added and referenced to from any repository?

True, I think IOOS Marine Life is bigger than just MBON, though the MBON projects are part of it. Are RAs doing marine life monitoring/observing that is not part of an MBON project? Are there other marine life observations we're missing by focusing on MBON? I see your point that any marine life observation work could just as easily use the same information as the mbon-docs. Maybe that should become marinelife-docs to serve as a one stop shop for how to share marine life observations with NOAA/IOOS/NCEI and OBIS?
I know MBON is trying to think about synthesis, but I think that is really key to IOOS Marine Life. The reason IOOS Marine Life is not just another data access portal for marine observations is that it is intended to offer actionable information about species distributions, trends, and indicators relating to essential variables to stakeholders in policymaking and conservation. I think having a dedicated repo/space to discuss what this looks like with folks from NMFS, ONMS, etc. might be helpful.
And yes, my understanding is that GitHub project boards can include issues from different repositories, not just the one it was made under.

created https://github.com/ioos/marine_life_data_network


Should we remove the Profile Functionality temporaily?

Manually creating the profile and viewing it only for oneself doesn't make much sense. Can we remove that feature altogether? Can we have it as a future scope as a mock ATS so that users can see how their resume performs and auto-parse the profile?
Any suggestions @jayrajmulani ?

Yes. I think it makes sense. We are already adding a couple of major features. Go ahead.

I have not completely removed the code, just commented the UI and backend pieces.

Hey @Yashasya, removing the profile compont is leading to ui tests failing that checks if profile component was rendered or not? Can you explore the UI tests and remove tests for profile and add tests for the components that we added?
https://github.com/jayrajmulani/se-group1-project2/blob/main/ui/__tests__/App.test.jsx#L185
Here is the test..

Also,  please check the style in the app.py. https://github.com/jayrajmulani/se-group1-project2/blob/main/backend/app.py#L85
Style checker has identified some issue, likely due to this comment here.

Issue fixed by #44


Cuckoo Sandbox module tracking issue (WIP)

Blocked on at least the following python 2.7 dependencies:

[ ] androguard

[x] builds
[ ] works
[ ] cleaned up

[ ] egghatch

[x] builds
[ ] works
[ ] cleaned up

[ ] httpreplay

[x] builds
[ ] works
[ ] cleaned up

[ ] oletools

[x] builds
[ ] works
[ ] cleaned up

[ ] peepdf

[x] builds
[ ] works
[ ] cleaned up

[ ] pyguacamole

[x] builds
[ ] works
[ ] cleaned up

[ ] pymisp

[x] builds
[ ] works
[ ] cleaned up

[ ] roach

[x] builds
[ ] works
[ ] cleaned up
capstone 3.0.5rc2 somethings real weird with the build system on this one

[ ] sflock

[x] builds
[ ] works
[ ] cleaned up

[ ] yara-python

[x] builds
[ ] works
[ ] cleaned up

I am now trying to fight through cuckoo's test suite.

Upstream: https://github.com/cuckoosandbox/cuckoo/issues/2676

As [malwr.com](http://malwr.com) is down with "we'll be back soon" for almost two years already, it would be nice to have an option for a private or local installation of Cuckoo Sandbox

I haven't gotten around to working on this for a while, I'll try to go back and publish some snippetsI had "soon"...

@deliciouslytyped @volth @FRidh @jonringer
I'm not sure how big of a draw Cuckoo Sandbox would be (see also #81418), but I stumbled on the project and this PR while looking for a cross-platform macOS+Linux sandbox solution.
It's too heavy for my use case, but I wanted to call it out because I've noticed work in other issues towards dropping the Python 2.7 package set and it looks like Cuckoo Sandbox still depends on Python 2.7.
FWIW, their blog post about the 2.0.7 release mentions ongoing work to update dependencies for 3.x compatibility.

Also last I heard it's roughly just one guy, so they probably have their hands full.


Calendar time issue

On the calendar after 11am it shows 12 am. Should be 12pm.

Fixed in v1.0.4. Thank you


Improve UI to make it as good as native apps

From https://reportedcab.slack.com/archives/C7Z0BHMMY/p1663092819678649:

hey team - so I met with @Joseph Frazier yesterday and we think we have a general plan for Reported. Basically due to insufficient resources for ios and android, we are going to direct our efforts to make the webapp a superior experience than the current mobile apps.  It’s not super easy but the first major step is to apply a framework like this https://onsen.io/v2/guide/ and beef up the mobile web UX. if anyone has web experience and wants to help, DM me and we can loop you into the webapp code. Step 2 would be to just encourage folks to use the webapp instead of ios/android and then we can begin making meaningful progress on the backend without having to apply changes to 3 different places.  eventually the plan would be to deprecate android/ios (but only if the webapp experience is superior. right now it isn’t). thoughts?

and https://reportedcab.slack.com/archives/C7Z0BHMMY/p1664676030509489?thread_ts=1663092819.678649&cid=C7Z0BHMMY:

Getting back to the original topic of the thread, improving the overall #webapp UI to make it as good as the native apps, I think I'm going to try using Onsen UI, as originally suggested by @jeff. I looked into Framework7 and Ionic, and while those seem to be a bit more popular, they seem to be a little more opinionated and geared more at starting new apps, rather than migrating existing apps over.
This decision isn't set in stone, so if anyone has advice/expertise here, please comment in the thread. Otherwise, I'll post back when I have an update!

Re. https://github.com/josephfrazier/Reported-Web/actions/runs/3167436493/jobs/5157931694 (Invalid state error), it looks like onsen doesn't work well with jest, see https://stackoverflow.com/questions/39843192/react-onsenui-mocha-test-error-error-invalid-state


Adding OneDrive to the list of default services for 1-year Advanced A…

…udit retention

Hi @sujitnaray PR merged. Looks like you opened this yesterday but I just saw it now. If you submit PRs in the future, be sure to @ comment me and then I'll get a notification right away and can respond faster. Thanks for making this change (particularly) updating the table at the end of the topic.


XML not highlighted in the expand shortcode.

Hello!
While using Hugo powered page for my personal project I use expand shortcode in conjunction with code highlighting function and if the code in the expand block is XML it’s not highlighted as it should be but parsed instead.
I figured out that if I remove htmlUnescape function from the expand shortcode, everything works fine.
My question is whether is this by design or is this indeed a bug since I don't really know the background why htmlUnescape is used here. If I choose any other programming language to be used for the highlighting inside expand shortcode everything works as regular.
Is there a different proposed way to use expand + highlight function while working with XML?
Thank you!
With unEscape function enabled:

With unEscape function removed (and as it should be):

Sample code:
https://gist.github.com/vhorvat/f5071d6e4b191d1829641772d94d959b
I use v115.2 of Hugo and latest Geekdoc version.

Hi, thanks for your report. This workaround was introduced in https://github.com/thegeeklab/hugo-geekdoc/pull/415, but I'm not able to reproduce the formatting issue anymore. I will remove htmlUnescape function from the short code for now.

Thank you for your fast reply!


Quality of Life changes

Here are some quality of life suggestions to make Lite a little bit easier to use. If someone else can make these in plugins too that would be nice

Right click menu on treeview
Give us a right click menu for the treeview, where right clicking the empty treeview or a file/folder gives us a menu to do things like deleting and creating files/folders, renaming, moving etc.
Open folder in editor
An easy way to open a new project folder would be in the editor itself like with a keybind opening a folder selector of the os.
Launching the executable opening the last opened workspace

bump. also, hot reloading the lua files for the app

There is a plug-in to open the last project opened. Also in Linux (I don't know how to do it in Windows or Mac) you can give the Laucher the argument "%F", that will open grag-dropped folders in the icon


return eml_validate check in eml_otherEntity_to_dataTable

I found that this function doesn't return an error if the eml_validate check fails.  Adding this in @jeanetteclark if you want to give it a quick review.  Curious to see if any of the unit tests fail now as well.

Hey Dominic I need to look closer at the second commit you made but probably won't have time this week. Did you test the case where you already have a bunch of dataTables, and want to convert a single otherEntity to a dataTable? I'm also not really sure what lines 718-721 are doing - seems like duplicate entities from the editor should be handled elsewhere if that is what it is doing

@jeanetteclark this what I mean by duplicate otherEntities created by the editor https://test.arcticdata.io/view/urn:uuid:d2710caa-9d77-4744-a76a-36a3e94705b7.  I didn't phrase it well and if you can improve on that definitely update the comment because that code isn't self-explanatory.
Here's an example of the error I was getting.  It wasn't showing up before because when validate_eml was to TRUE then eml_validate would run, but it didn't actually cause an error, it just silently failed.
cn_staging <- CNode('STAGING')
adc_test <- getMNode(cn_staging,'urn:node:mnTestARCTIC')
doc <- read_eml(getObject(adc_test, 'urn:uuid:d2710caa-9d77-4744-a76a-36a3e94705b7'))
# the first one has attributes and the second doesn't - remove it
doc$dataset$otherEntity[[2]] <- NULL
# Pass doc into `eml_otherEntity_to_dataTable`

# Here's the first line that causes problems
otherEntity <- doc$dataset$otherEntity
# If you print otherEntity to the console it's a list of length 1 (because it used to a list of length 2 but setting the 2nd otherEntity to NULL doesn't update it to the correct format)

# Ends up getting compounded by this line
otherEntity <- list(otherEntity)
# Now if you print it you'll see [[1]][[1]] notation


Enhancing Accessibility: Introducing Subtitle Support for Podcasts

Could we have subtitles for each episode? I think it would be useful for users who aren't proficient at listening to English podcasts. We could handle it with AI, and I'm so excited to implement it.

This is coming via the transcripts project. Follow along on #958


Call to undefined method

I have EssentialsPE installed and when someone warps the player is warped, but the following is displayed in the console.
[07:23:40] [Server thread/CRITICAL]: Unhandled exception executing command 'warp pe' in warp: Call to undefined method EssentialsPE\Loader::warpExists()
[07:23:40] [Server thread/CRITICAL]: Error: "Call to undefined method EssentialsPE\Loader::warpExists()" (EXCEPTION) in "/SimpleWarp-2.1.1.phar/src/falkirks/simplewarp/command/essentials/EssentialsWarpCommand" at line 24

Posted this 10 days ago and still not a peep? Anyone still working on this?

This issue was already fixed in https://github.com/Falkirks/SimpleWarp/commit/508e8f06e1f1e8910b06e7973ca20c3a916d4a1e. I have to build and release.

This is what I am trying to explain. I downloaded the untested build, built it via DevTools and is the version I was using. 2.1.1 Right?


Fix custom search modes not working

Fixes custom search modes no longer working. New update seems to use _computedWatchers.searchResults rather than sortedElements due to separating their behaviors.

Looks good, thanks!

released as 3.3.3!


Elasticsearch could not resolve the host [elasticsearch-master-headless]

Chart version:
latest
Kubernetes version:
1.15.1
Kubernetes provider: E.g. GKE (Google Kubernetes Engine)
Aliyun
Helm Version:
2.14.2
helm get release output
Error: release: "release" not found
[root@s5-aliyun-atyun elasticsearch]# helm get efk-es
REVISION: 1
RELEASED: Wed Sep  4 14:01:59 2019
CHART: elasticsearch-7.3.0
USER-SUPPLIED VALUES:
antiAffinity: soft
antiAffinityTopologyKey: kubernetes.io/hostname
clusterHealthCheckParams: wait_for_status=green&timeout=1s
clusterName: elasticsearch
esConfig: {}
esJavaOpts: -Xmx1g -Xms1g
esMajorVersion: "7"
extraEnvs: []
extraInitContainers: []
extraVolumeMounts: []
extraVolumes: []
fsGroup: ""
fullnameOverride: ""
httpPort: 9200
image: docker.elastic.co/elasticsearch/elasticsearch
imagePullPolicy: IfNotPresent
imagePullSecrets: []
imageTag: 7.3.0
ingress:
  annotations: {}
  enabled: false
  hosts:
  - chart-example.local
  path: /
  tls: []
initResources: {}
labels: {}
lifecycle: {}
masterService: ""
masterTerminationFix: false
maxUnavailable: 1
minimumMasterNodes: 1
nameOverride: ""
networkHost: <IP_ADDRESS>
nodeAffinity: {}
nodeGroup: master
nodeSelector:
  nodeType: efk
persistence:
  annotations: {}
  enabled: true
podAnnotations: {}
podManagementPolicy: Parallel
podSecurityContext:
  fsGroup: 1000
priorityClassName: ""
protocol: http
readinessProbe:
  failureThreshold: 3
  initialDelaySeconds: 10
  periodSeconds: 10
  successThreshold: 3
  timeoutSeconds: 5
replicas: 2
resources:
  limits:
    cpu: 1000m
    memory: 2Gi
  requests:
    cpu: 100m
    memory: 2Gi
roles:
  data: "true"
  ingest: "true"
  master: "true"
schedulerName: ""
secretMounts: []
securityContext:
  capabilities:
    drop:
    - ALL
  runAsNonRoot: true
  runAsUser: 1000
service:
  annotations: {}
  nodePort: null
  type: ClusterIP
sidecarResources: {}
sysctlInitContainer:
  enabled: true
sysctlVmMaxMapCount: 262144
terminationGracePeriod: 120
tolerations: []
transportPort: 9300
updateStrategy: RollingUpdate
volumeClaimTemplate:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 30Gi

COMPUTED VALUES:
antiAffinity: soft
antiAffinityTopologyKey: kubernetes.io/hostname
clusterHealthCheckParams: wait_for_status=green&timeout=1s
clusterName: elasticsearch
esConfig: {}
esJavaOpts: -Xmx1g -Xms1g
esMajorVersion: "7"
extraEnvs: []
extraInitContainers: []
extraVolumeMounts: []
extraVolumes: []
fsGroup: ""
fullnameOverride: ""
httpPort: 9200
image: docker.elastic.co/elasticsearch/elasticsearch
imagePullPolicy: IfNotPresent
imagePullSecrets: []
imageTag: 7.3.0
ingress:
  annotations: {}
  enabled: false
  hosts:
  - chart-example.local
  path: /
  tls: []
initResources: {}
labels: {}
lifecycle: {}
masterService: ""
masterTerminationFix: false
maxUnavailable: 1
minimumMasterNodes: 1
nameOverride: ""
networkHost: <IP_ADDRESS>
nodeAffinity: {}
nodeGroup: master
nodeSelector:
  nodeType: efk
persistence:
  annotations: {}
  enabled: true
podAnnotations: {}
podManagementPolicy: Parallel
podSecurityContext:
  fsGroup: 1000
priorityClassName: ""
protocol: http
readinessProbe:
  failureThreshold: 3
  initialDelaySeconds: 10
  periodSeconds: 10
  successThreshold: 3
  timeoutSeconds: 5
replicas: 2
resources:
  limits:
    cpu: 1000m
    memory: 2Gi
  requests:
    cpu: 100m
    memory: 2Gi
roles:
  data: "true"
  ingest: "true"
  master: "true"
schedulerName: ""
secretMounts: []
securityContext:
  capabilities:
    drop:
    - ALL
  runAsNonRoot: true
  runAsUser: 1000
service:
  annotations: {}
  type: ClusterIP
sidecarResources: {}
sysctlInitContainer:
  enabled: true
sysctlVmMaxMapCount: 262144
terminationGracePeriod: 120
tolerations: []
transportPort: 9300
updateStrategy: RollingUpdate
volumeClaimTemplate:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 30Gi

HOOKS:
---
# efk-es-kkyrp-test
apiVersion: v1
kind: Pod
metadata:
  name: "efk-es-kkyrp-test"
  annotations:
    "helm.sh/hook": test-success
spec:
  containers:
  - name: "efk-es-flgtb-test"
    image: "docker.elastic.co/elasticsearch/elasticsearch:7.3.0"
    command:
      - "sh"
      - "-c"
      - |
        #!/usr/bin/env bash -e
        curl -XGET --fail 'elasticsearch-master:9200/_cluster/health?wait_for_status=green&timeout=1s'
  restartPolicy: Never
MANIFEST:

---
# Source: elasticsearch/templates/poddisruptionbudget.yaml
apiVersion: policy/v1beta1
kind: PodDisruptionBudget
metadata:
  name: "elasticsearch-master-pdb"
spec:
  maxUnavailable: 1
  selector:
    matchLabels:
      app: "elasticsearch-master"
---
# Source: elasticsearch/templates/service.yaml
kind: Service
apiVersion: v1
metadata:
  name: elasticsearch-master
  labels:
    heritage: "Tiller"
    release: "efk-es"
    chart: "elasticsearch-7.3.0"
    app: "elasticsearch-master"
  annotations:
    {}

spec:
  type: ClusterIP
  selector:
    heritage: "Tiller"
    release: "efk-es"
    chart: "elasticsearch-7.3.0"
    app: "elasticsearch-master"
  ports:
  - name: http
    protocol: TCP
    port: 9200
  - name: transport
    protocol: TCP
    port: 9300
---
# Source: elasticsearch/templates/service.yaml
kind: Service
apiVersion: v1
metadata:
  name: elasticsearch-master-headless
  labels:
    heritage: "Tiller"
    release: "efk-es"
    chart: "elasticsearch-7.3.0"
    app: "elasticsearch-master"
  annotations:
    service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
spec:
  clusterIP: None # This is needed for statefulset hostnames like elasticsearch-0 to resolve
  # Create endpoints also if the related pod isn't ready
  publishNotReadyAddresses: true
  selector:
    app: "elasticsearch-master"
  ports:
  - name: http
    port: 9200
  - name: transport
    port: 9300
---
# Source: elasticsearch/templates/statefulset.yaml
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: elasticsearch-master
  labels:
    heritage: "Tiller"
    release: "efk-es"
    chart: "elasticsearch-7.3.0"
    app: "elasticsearch-master"
  annotations:
    esMajorVersion: "7"
spec:
  serviceName: elasticsearch-master-headless
  selector:
    matchLabels:
      app: "elasticsearch-master"
  replicas: 2
  podManagementPolicy: Parallel
  updateStrategy:
    type: RollingUpdate
  volumeClaimTemplates:
  - metadata:
      name: elasticsearch-master
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 30Gi

  template:
    metadata:
      name: "elasticsearch-master"
      labels:
        heritage: "Tiller"
        release: "efk-es"
        chart: "elasticsearch-7.3.0"
        app: "elasticsearch-master"
      annotations:

    spec:
      securityContext:
        fsGroup: 1000

      nodeSelector:
        nodeType: efk

      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 1
            podAffinityTerm:
              topologyKey: kubernetes.io/hostname
              labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - "elasticsearch-master"
      terminationGracePeriodSeconds: 120
      volumes:
      initContainers:
      - name: configure-sysctl
        securityContext:
          runAsUser: 0
          privileged: true
        image: "docker.elastic.co/elasticsearch/elasticsearch:7.3.0"
        command: ["sysctl", "-w", "vm.max_map_count=262144"]
        resources:
          {}

      containers:
      - name: "elasticsearch"
        securityContext:
          capabilities:
            drop:
            - ALL
          runAsNonRoot: true
          runAsUser: 1000

        image: "docker.elastic.co/elasticsearch/elasticsearch:7.3.0"
        imagePullPolicy: "IfNotPresent"
        readinessProbe:
          failureThreshold: 3
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 3
          timeoutSeconds: 5

          exec:
            command:
              - sh
              - -c
              - |
                #!/usr/bin/env bash -e
                # If the node is starting up wait for the cluster to be ready (request params: 'wait_for_status=green&timeout=1s' )
                # Once it has started only check that the node itself is responding
                START_FILE=/tmp/.es_start_file

                http () {
                    local path="${1}"
                    if [ -n "${ELASTIC_USERNAME}" ] && [ -n "${ELASTIC_PASSWORD}" ]; then
                      BASIC_AUTH="-u ${ELASTIC_USERNAME}:${ELASTIC_PASSWORD}"
                    else
                      BASIC_AUTH=''
                    fi
                    curl -XGET -s -k --fail ${BASIC_AUTH} http://<IP_ADDRESS>:9200${path}
                }

                if [ -f "${START_FILE}" ]; then
                    echo 'Elasticsearch is already running, lets check the node is healthy'
                    http "/"
                else
                    echo 'Waiting for elasticsearch cluster to become cluster to be ready (request params: "wait_for_status=green&timeout=1s" )'
                    if http "/_cluster/health?wait_for_status=green&timeout=1s" ; then
                        touch ${START_FILE}
                        exit 0
                    else
                        echo 'Cluster is not yet ready (request params: "wait_for_status=green&timeout=1s" )'
                        exit 1
                    fi
                fi
        ports:
        - name: http
          containerPort: 9200
        - name: transport
          containerPort: 9300
        resources:
          limits:
            cpu: 1000m
            memory: 2Gi
          requests:
            cpu: 100m
            memory: 2Gi

        env:
          - name: node.name
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: cluster.initial_master_nodes
            value: "elasticsearch-master-0,elasticsearch-master-1,"
          - name: discovery.seed_hosts
            value: "elasticsearch-master-headless"
          - name: cluster.name
            value: "elasticsearch"
          - name: network.host
            value: "<IP_ADDRESS>"
          - name: ES_JAVA_OPTS
            value: "-Xmx1g -Xms1g"
          - name: node.data
            value: "true"
          - name: node.ingest
            value: "true"
          - name: node.master
            value: "true"
        volumeMounts:
          - name: "elasticsearch-master"
            mountPath: /usr/share/elasticsearch/data

Describe the bug:
elasticsearch could not resolve the host [elasticsearch-master-headless]

{"type": "server", "timestamp": "2019-09-04T06:05:56,653+0000", "level": "DEBUG", "component": "o.e.a.a.c.h.TransportClusterHealthAction", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-1",  "message": "timed out while retrying [cluster:monitor/health] after failure (timeout [1s])"  }
{"type": "server", "timestamp": "2019-09-04T06:05:56,653+0000", "level": "WARN", "component": "r.suppressed", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-1",  "message": "path: /_cluster/health, params: {wait_for_status=green, timeout=1s}" ,
"stacktrace": ["org.elasticsearch.discovery.MasterNotDiscoveredException: null",
"at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$3.onTimeout(TransportMasterNodeAction.java:251) [elasticsearch-7.3.0.jar:7.3.0]",
"at org.elasticsearch.cluster.ClusterStateObserver$ContextPreservingListener.onTimeout(ClusterStateObserver.java:325) [elasticsearch-7.3.0.jar:7.3.0]",
"at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:252) [elasticsearch-7.3.0.jar:7.3.0]",
"at org.elasticsearch.cluster.service.ClusterApplierService$NotifyTimeout.run(ClusterApplierService.java:572) [elasticsearch-7.3.0.jar:7.3.0]",
"at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:688) [elasticsearch-7.3.0.jar:7.3.0]",
"at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]",
"at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]",
"at java.lang.Thread.run(Thread.java:835) [?:?]"] }
{"type": "server", "timestamp": "2019-09-04T06:05:57,567+0000", "level": "WARN", "component": "o.e.d.SeedHostsResolver", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-1",  "message": "failed to resolve host [elasticsearch-master-headless]" ,
"stacktrace": ["java.net.UnknownHostException: elasticsearch-master-headless",
"at java.net.InetAddress$CachedAddresses.get(InetAddress.java:797) ~[?:?]",
"at java.net.InetAddress.getAllByName0(InetAddress.java:1505) ~[?:?]",
"at java.net.InetAddress.getAllByName(InetAddress.java:1364) ~[?:?]",
"at java.net.InetAddress.getAllByName(InetAddress.java:1298) ~[?:?]",
"at org.elasticsearch.transport.TcpTransport.parse(TcpTransport.java:528) ~[elasticsearch-7.3.0.jar:7.3.0]",
"at org.elasticsearch.transport.TcpTransport.addressesFromString(TcpTransport.java:470) ~[elasticsearch-7.3.0.jar:7.3.0]",
"at org.elasticsearch.transport.TransportService.addressesFromString(TransportService.java:754) ~[elasticsearch-7.3.0.jar:7.3.0]",
"at org.elasticsearch.discovery.SeedHostsResolver.lambda$resolveHostsLists$0(SeedHostsResolver.java:144) ~[elasticsearch-7.3.0.jar:7.3.0]",
"at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]",
"at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:688) ~[elasticsearch-7.3.0.jar:7.3.0]",
"at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]",
"at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]",
"at java.lang.Thread.run(Thread.java:835) [?:?]"] }
{"type": "server", "timestamp": "2019-09-04T06:05:58,567+0000", "level": "WARN", "component": "o.e.d.SeedHostsResolver", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-1",  "message": "failed to resolve host [elasticsearch-master-headless]" ,
"stacktrace": ["java.net.UnknownHostException: elasticsearch-master-headless",
"at java.net.InetAddress$CachedAddresses.get(InetAddress.java:797) ~[?:?]",
"at java.net.InetAddress.getAllByName0(InetAddress.java:1505) ~[?:?]",
"at java.net.InetAddress.getAllByName(InetAddress.java:1364) ~[?:?]",
"at java.net.InetAddress.getAllByName(InetAddress.java:1298) ~[?:?]",
"at org.elasticsearch.transport.TcpTransport.parse(TcpTransport.java:528) ~[elasticsearch-7.3.0.jar:7.3.0]",
"at org.elasticsearch.transport.TcpTransport.addressesFromString(TcpTransport.java:470) ~[elasticsearch-7.3.0.jar:7.3.0]",

`

I'm having the same issue.
It's like it only works for already deployed elastic but cannot work as a fresh deployment.
So I'd suggest making sure you have at least one pod ready first (like if it's a fresh deploy you can drop readiness probe first) and then you can add it back.
My case is we're rolling terraform environments (from zero in case of testing envs) and so I need this to work from ground zero, there're ugly workarounds but would appreciate a more straightforward approach for fresh deployment case.

Aliyun

@100cm Everything you have posted looks pretty normal to me. The only thing stands out is that you listed "Aliyun" as your provider. Could you give me some more details about the environment you are running in? Is this some kind of managed service or have you deployed your own Kubenretes cluster on here? Is there anything special with it or the DNS setup? I'm going to run the values you provided against GKE to see if the configuration is valid.

I'm having the same issue.
It's like it only works for already deployed elastic but cannot work as a fresh deployment.
So I'd suggest making sure you have at least one pod ready first, like if it's a fresh deploy you can drop readiness probe first and then you can add it back.
My case is we're rolling terraform environments (from zero in case of testing envs) and so I need this to work from ground zero, there're ugly workarounds but would appreciate a more straightforward approach for fresh deployment case.

@LiuVII Could you open a separate issue too please with more details? It certainly works for fresh deployments. We have pretty extensive testing to make sure of this

Internally we are also using the charts with a terraform environment with great success. So if you are running into problems then it sounds like they should be fixable!

I'm going to run the values you provided against GKE to see if the configuration is valid.

With the exception of needing to remove:
nodeSelector:
  nodeType: efk

I was able to deploy a healthy cluster with the configuration you provided. It is also very possible that something might be broken with Kubernetes 1.15 as we only test with the available GKE versions (currently 1.14, 1.13 and 1.12). @LiuVII are you also running Kubernetes 1.15?