Handling large number of columns

Have you given any thought to handling cases (like time-series data) where the number of columns, rather than rows, is large?
We wrote a virtualizing grid (using Knockout) for that scenario, because as far as we could tell, none of the existing table libraries handled it well, if at all. The primary problems were:

Hanging a lot of configuration off an explicit Column entry
Assuming that scroll/overflow was vertical

The version my team wrote is internal, but a (very rough) open-source re-implementation is at https://github.com/jstclair/knockout-virtual-grid (hotkeys for editing the grid are documented at https://github.com/gnab/editableCell#list-of-keyboard-shortcuts )

We have thought about doing the same to columns that we do to rows with scrolling and only render ones that are visible + buffer to DOM, but we didn't have need to actually implement it. The solution we were thinking about was only for scrolling perf improvement and I think it would be able to handle hundreds of columns, but I don't think it would do i.e. million, because we still would have kept passing data by rows.
What number of columns do you have in mind when you say "large number"?

For our implementation, we were dealing with projected (power) generator outputs, where the customer typically needed to see 1-2 weeks at a time (@ 15 min resolution); so, say, 600 - 1200 columns. The number of generators was fixed, of course, so rows typically didn't need to scroll at all.
But as I mentioned, there's two parts - performance (which I believe isn't that hard, given your existing baseline), and the ability for the developer to actually configure the grid to use 100s-low 1000s of columns.

So right now, it the table will handle that many columns, but the performance of may be not perfect, because all cells of visible row are rendered into DOM.
For configuring large numbers of columns, this is already possible to do with the FixedDataTable, since you can pass children (column configs) to FixedDatatTable as array- you don't have to inline column definitions where you render array. You could do something like
columnNamesYouWantToSee.map(columnName => buildColumnConfigForName(columnName))
Where buildColumnConfigForName returns instance of FixedDataTableColumn based on the columnName- it can specify custom renderers, min widths etc. Such table could be just passed as children of FixedDataTable and it would render all those columns.
Is that something that would work in your case?

I'll have to try it out with something approximate to our data, but I guess thee key point is that you don't virtualized column cells if the row is visible. In our case, all the rows were almost always visible, so perfect-wise, that would probably be similar to our first attempt that simply used css overflow, if I'm understanding you correctly.
For configuration, that sounds pretty reasonable. One take could be to assume you'd want to configure fixed columns, but the rest could be auto-configured, in a similar way to infinite-scrolling rows. You might want to provide a template (formatting, etc) but they would be dynamically provisioned as you scrolled. Does that make sense?
And thanks for taking the time to address this issue. I understand it's outside of the typical use case.

+1 to this. This would be helpful for us, we have a few user with tables that have around ~750 columns :(.

I have performance problem when horizontal scrolling with fixed-data-table.
My grid has about 20+ columns, it scrolls smoothly with v2.0 but not with v3.0. When I scroll the grid the who broswer lags, I profile the lag in Chrome and find that the costest func is validate which maybe the React's internal method. I'm puzzled so I keeped with v2.0 now. :(

@airwin have you tried the new allowCellsRecycling config in v3.0? It was designed to solve the use cases described here by not rendering columns when they leave the viewport.
cc @wlis

@airwin same problem here (15 columns). @pieterv allowCellsRecycling=true is not helping, horizontal scrolling is super slow. Back to 0.2.0 :(

+1 to super slow horizontal scroll performance.
Vertical scrolling is fine, but horizontal scrolling lags like mad.  I've got about 30 columns.  allowCellsRecycling=true didn't do anything for me.

Hmm.. I think you got the answer. T.T  @pieterv

I have 25 columns and also get slow horizontal scrolling issue (seems like 2-5fps or something). But I noticed that this happens only when I open development tools in chrome. While they are closed - horizontal scrolling performance is quite good.
Adding allowCellsRecycling:true helps a little bit, but it's still slow.
Not sure why it happens but this might help to investigate this issue.
Note, that when I try to run same code with 0.2.0 - scrolling performance does not have any issues despite if development tools are opened or not

Just wanted to mention that I'm having this issue as well. Vertical scrolling is fast, but horizontal scrolling is very laggy. It happens even with relatively few columns (e.g. 10-15), and I don't notice a difference with allowCellsRecycling.

We've got 48 columns in 8 column groups and it is really laggy. You're all probably busy with 5.0 but this needs a fix as soon as you can.

I'm sad to see the problem remain unresolved. :(

I have the same problem, 16 column, vertical scrolling is buggy, but only when dev tools are opened in Chrome

Using the table for a reporting interface, where each week is mapped to a column with aggregated data. Scrolling horizontally is indeed super slow and laggy (allowCellsRecycling:true didn't make a difference neither - using 0.4.7 and react v0.13.3)

Any advance here? I'm having the same issue just with 30 columns..

I've just tried 0.5.0 on a separate project and the performance is still critically slow, regardless of the number of columns.
Downgrading to 0.2.0 still provides a noticeable improvement, although the render process takes a few seconds on even the smallest amounts of data to the point where I can't justify using fixed-data-table.
I'd be really interested to know if anyone receives optimal performance from fixed-data-table, and if they are, how do they have it configured.

Also sad to report that horizontal scrolling performance is pretty bad with anything that has over 15+ columns. Is there a plan to improve this? I would be happy to help.

@blackn1ght I still keep using the v0.2.0 in my project but I think I should search for other succedaneum , like https://github.com/zippyui/react-datagrid ?

I'm curious has any one tested to see if this is resolved with v6.0?

With 0.6.0 you have the option to use your own Cell (so you dont need the 3+ divs that come with the current default cell), i'm hoping this will help make over head of the table less. Let us know if this release helps at all.

I am using 0.6.0, I have allowCellsRecycling set to true and with only 10 columns the table scrolling is very slow. There are noticeable jumps and pauses. I tried using custom cells for all the data. That doesn't seem to have much effect either.

In 0.6.0, use shouldComponentUpdate in custom cell component will help a lot.

If you're need to handle a large number of columns and you're looking for an alternative, react-virtualized added support for grids (large number of both columns and rows) in the latest release.

Reporting the same slowness with 10+ columns (even with only a few dozen rows).
allowCellsRecycling and closing Chrome dev tools helped, but getting 2-5 fps with horizontal scrolling.


EdgeDB fails to handle a schema definition

EdgeDB Version: EdgeDB CLI 2.0.1+b407f6f
OS Version: Darwin Kernel Version 21.5.0: Tue Apr 26 21:08:22 PDT 2022; root:xnu-8020.121.3~4/RELEASE_X86_64

Steps to Reproduce:

execute edgedb migration create

Schema:
module default {
  type Person {
    required property name -> str {
      # empty block
    }
  }
}

Edgedb should either accept empty block as simple semi-colon or show a message that empty block are invalid syntax.
instead we have an InternalServerError with:
edgedb error: InternalServerError: 'NoneType' object is not iterable
  Hint: This is most likely a bug in EdgeDB. Please consider opening an issue ticket at https://github.com/edgedb/edgedb/issues/new?template=bug_report.md
  Server traceback:
      Traceback (most recent call last):
        File "/Users/a_random_username/Library/Application Support/edgedb/portable/2.0/lib/python3.10/site-packages/edb/server/compiler_pool/worker_proc.py", line 55, in worker
          res = meth(*args)
        File "/Users/a_random_username/Library/Application Support/edgedb/portable/2.0/lib/python3.10/site-packages/edb/server/compiler_pool/worker.py", line 160, in compile
          units, cstate = COMPILER.compile(
        File "/Users/a_random_username/Library/Application Support/edgedb/portable/2.0/lib/python3.10/site-packages/edb/server/compiler/compiler.py", line 2207, in compile
          unit_group = self._compile(ctx=ctx, source=source)
        File "/Users/a_random_username/Library/Application Support/edgedb/portable/2.0/lib/python3.10/site-packages/edb/server/compiler/compiler.py", line 1738, in _compile
          return self._try_compile(ctx=ctx, source=source)
        File "/Users/a_random_username/Library/Application Support/edgedb/portable/2.0/lib/python3.10/site-packages/edb/server/compiler/compiler.py", line 1802, in _try_compile
          comp, capabilities = self._compile_dispatch_ql(
        File "/Users/a_random_username/Library/Application Support/edgedb/portable/2.0/lib/python3.10/site-packages/edb/server/compiler/compiler.py", line 1668, in _compile_dispatch_ql
          query = self._compile_ql_migration(
        File "/Users/a_random_username/Library/Application Support/edgedb/portable/2.0/lib/python3.10/site-packages/edb/server/compiler/compiler.py", line 1049, in _compile_ql_migration
          target_schema = s_ddl.apply_sdl(
        File "/Users/a_random_username/Library/Application Support/edgedb/portable/2.0/lib/python3.10/site-packages/edb/schema/ddl.py", line 485, in apply_sdl
          ddl_stmts = s_decl.sdl_to_ddl(current_schema, documents)
        File "/Users/a_random_username/Library/Application Support/edgedb/portable/2.0/lib/python3.10/site-packages/edb/edgeql/declarative.py", line 419, in sdl_to_ddl
          trace_dependencies(decl_ast, ctx=tracectx)
        File "/Users/a_random_username/Library/Application Support/edgedb/portable/2.0/lib/python3.10/functools.py", line 889, in wrapper
          return dispatch(args[0].__class__)(*args, **kw)
        File "/Users/a_random_username/Library/Application Support/edgedb/portable/2.0/lib/python3.10/site-packages/edb/edgeql/declarative.py", line 926, in trace_default
          _register_item(node, ctx=ctx)
        File "/Users/a_random_username/Library/Application Support/edgedb/portable/2.0/lib/python3.10/site-packages/edb/edgeql/declarative.py", line 1057, in _register_item
          trace_dependencies(cmd, ctx=ctx)
        File "/Users/a_random_username/Library/Application Support/edgedb/portable/2.0/lib/python3.10/functools.py", line 889, in wrapper
          return dispatch(args[0].__class__)(*args, **kw)
        File "/Users/a_random_username/Library/Application Support/edgedb/portable/2.0/lib/python3.10/site-packages/edb/edgeql/declarative.py", line 838, in trace_ConcretePointer
          _register_item(
        File "/Users/a_random_username/Library/Application Support/edgedb/portable/2.0/lib/python3.10/site-packages/edb/edgeql/declarative.py", line 1072, in _register_item
          tdeps = qltracer.trace_refs(
        File "/Users/a_random_username/Library/Application Support/edgedb/portable/2.0/lib/python3.10/site-packages/edb/edgeql/tracer.py", line 285, in trace_refs
          trace(qltree, ctx=ctx)
        File "/Users/a_random_username/Library/Application Support/edgedb/portable/2.0/lib/python3.10/functools.py", line 889, in wrapper
          return dispatch(args[0].__class__)(*args, **kw)
        File "/Users/a_random_username/Library/Application Support/edgedb/portable/2.0/lib/python3.10/site-packages/edb/edgeql/tracer.py", line 881, in trace_Shape
          for element in node.elements:
      TypeError: 'NoneType' object is not iterable

Is this the full schema that produces this failure for you? It works when I try it.

I can produce this backtrace with this somewhat similar (but solidly broken) schema:
  type Person {
    required property name := str {
      # empty block
    }
  }

where we write := instead of -> and so str { } is interpreted as a shape.

Yes := as a shape, sorry I couldn't put the whole schema.
Thank you for the prompt reply

Still a bug that it produces an ISE


Investigate passing a parameter through a binding

In the current SDK binding implementation for in-proc, users can use a trigger binding to bind to a POCO, then that to pass a parameter to the next binding.
For example in the code snippet below, we are using a BlobTrigger and binding to a Book, this book has an ID property. In the input binding, we are able to use {id} to extract the ID value from the Book provided by the trigger. This is then parsed by the input binding so that we can get the content of the blob named "{id}.txt". So if the blob we upload to the book-trigger container has the ID value set to "1". Then the input binding will grab the contents of the "1.txt" file from the book-input container.
public static class BlobBookFunction
{
    [Function(nameof(BlobBookFunction))]
    public static async Task Run(
        [BlobTrigger("book-trigger", Connection = "AzureWebJobsStorage")] Book book,
        [BlobInput("book-input/{id}.txt", Connection = "AzureWebJobsStorage")] string myBlob,
        FunctionContext context)
    {
        var logger = context.GetLogger(nameof(BlobBookFunction));
        logger.LogInformation("Blob content: {content}", myBlob);
    }
}

public class Book
{
    public string Name { get; set; }
    public string Id { get; set; }
}

Actions

[ ] Investigate if it is possible to make this work with out-of-proc SDK bindings
[ ] Create an issue to track the work required to enable this feature

FYI. We do support using the value from one binding for another.
See this example,
[Function("GetItemsFromBlob")]
public HttpResponseData Run(
     [HttpTrigger(AuthorizationLevel.Function, "get", Route ="items/{itemId}")] HttpRequestData req,
     [BlobInput("samples-workitems/{ItemId}.txt", Connection = "MyStorageConnString")] string blob
    )
{
    var response = req.CreateResponse(HttpStatusCode.OK);
    response.Headers.Add("Content-Type", "text/plain; charset=utf-8");

    response.WriteString($"Blob content: {blob}");

    return response;
}

This sample is reading the itemId value from the http request URL and use that for getting the blob input entry. For example if you issue a request like /api/items/foo, "foo" will be bound to the ItemId segment of the route and the blob input binding will try to get the "foo.txt" file from the "sample-workitems" container.
I haven't checked the child property(Id from Book) case works.

That's good to know, let me try the example you provided. I tried the book example with the reference binding object prototype we have for SDK-type bindings and got this error:
No value for named parameter 'id'
So something is definitely disconnected somewhere and needs investigation

We need to investigate whether the current binding data pipeline (in WebJobs) will continue to work once we enable SDK Type bindings.

We found a bug with DefaultTypes that makes this specific scenario not work - this is a breaking change that may need to be addressed with 5.0
We opted to make ParameterBindingData modal, meaning if an input binding uses an expression, the trigger binding will not use ParameterBindingData (the input binding will continue to use ParameterBindingData)


Some parser bugs from fuzzing

I ran a fuzzer for a bit and found a few cases where the parser incorrectly rejects valid programs. (I'm running it from node, if that makes a difference.)
If you'd prefer I not file issues found with a fuzzer, let me know.

All of the following are incorrectly rejected by the parser:
(async,e)=>null

async function f(){ for await (const t of /a/); }

(class{}/null)

{ {} {} /a/ }

also
(function(p=function*(){yield}){})


g_key_file_load_from_data_dirs arg 2 should be *const

g_key_file_load_from_data_dirs arg 2 should be *const. It is currently generated as *mut
For refrence, the C prototype is
gboolean g_key_file_load_from_data_dirs (GKeyFile *key_file,
                                const gchar *file,
                                gchar **full_path,
                                GKeyFileFlags flags,
                                GError **error);

Problem in source GLib-2.0.gir which is not your.
<parameter name="file" transfer-ownership="none">
  <doc xml:space="preserve">a relative path to a filename to open and parse</doc>
  <type name="filename" c:type="gchar*"/>
</parameter>

But it seems don't effect on generated code (with  g_key_file_load_from_file as example)
    pub fn load_from_file<T: AsRef<std::path::Path>>(&self, file: T, flags: glib::KeyFileFlags) -> Result<(), Error> {
        unsafe {
            let mut error = ptr::null_mut();
            let _ = ffi::g_key_file_load_from_file(self.to_glib_none().0, file.as_ref().to_glib_none().0, flags.to_glib(), &mut error);
            if error.is_null() { Ok(()) } else { Err(from_glib_full(error)) }
        }
    }

I don't even see these wrappers in the repository. I'd just use that. See gtk-rs/glib#135

Its from my attempt to generate GKeyFile from your issue.

ah ok. I see. Thanks !

Anyway passing a string requires (copying it into a temporary CString) so generally this kind of discrepancy should not be a problem.


Add support for LaTex mode (TeX files)

The extension should be able to work with TeX files.

Yes, it activates properly, thanks!


Bind this repository (or a branch) to runtime-spec releases

This is somewhat related to #16 (where a comment sounded like “we'll get serious once we get to 1.0”), and #69 (where I argue that we should not float with runC).  But concretely, I am unable to test ccon with the current ocitools.  For one, my command-line API is different (I expect RUNTIME start --id $CONTAINER_ID, while runC seems to prefer RUNTIME start $CONTAINER_ID).  But once I've patched around that:
ocitools$ git diff -U0
diff --git a/test_runtime.sh b/test_runtime.sh
index da3f026..d498ae8 100755
--- a/test_runtime.sh
+++ b/test_runtime.sh
@@ -84 +84 @@ popd > /dev/null
-TESTCMD="${RUNTIME} start $(uuidgen)"
+TESTCMD="${RUNTIME} start --id $(uuidgen)"

I still cannot run the tests, because they're based on an unreleased 0.6.0-dev version of the spec:
ocitools$ ./test_runtime.sh -l debug -r ccon-oci
-----------------------------------------------------------------------------------
VALIDATING RUNTIME: ccon-oci
-----------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/wking/bin/ccon-oci", line 578, in <module>
    main()
  File "/home/wking/bin/ccon-oci", line 574, in main
    args.func(**kwargs)
  File "/home/wking/bin/ccon-oci", line 442, in start
    config_state = _load_config(bundle=bundle, id=id)
  File "/home/wking/bin/ccon-oci", line 421, in _load_config
    version = _get_version(config=config)
  File "/home/wking/bin/ccon-oci", line 64, in _get_version
    'unsupported OCI-spec version: {}'.format(version))
NotImplementedError: unsupported OCI-spec version: 0.6.0-dev
Runtime ccon-oci failed validation

Is there a branching plan for supporting per-version testing?  For example, we might want to start a v1.0 branch to track the v1.0 release series while master continues on with any changes destined for v1.1 / v2.0, etc.  Then the v1.0 branch could generate (at the moment) 1.0.0-rc1 specs that ccon-oci understands.

After more discussion in #140, we got a v1.0.0.rc1 branch last month.  So far most work has gone into master with frequent backport cherry-picks and the odd PR targeting v1.0.0.rc1.  This works well enough for me, so I'm closing this issue.


Documenting the targetPendingRequests field

Adding reference documentation for the targetPendingRequests field, which was added in #223
Checklist

[x] Commits are signed with Developer Certificate of Origin (DCO)
[x] Any necessary documentation is added, such as:

README.md
The docs/ directory
The docs repo

Fixes #230

cc/ @ajanth97

@tomkerkhove good point. I moved the technical stuff into the design doc as you suggested.

@tomkerkhove can you PTAL at this again?


"Piatra" is listed in the drop-down menu

Description:
"Piatra" is listed in the drop-down menu alongside its corresponding county and commune, according to the law
Precondition:
The website is up an running.

Step 1
Type "Piatra" in the search bar.

Expected results
The location is listed in the drop-down menu as "Piatra, Argeș (Brăduleț)".


Error with empty database and multiple processes working

This is a real not blocking bug, but if we use Mongodb sequence in database and two process try to insert simultaneously a commit, both of them generate next id with an upsert, record is not present, both go in upsert, one of them get an error and the commit was lost.
With empty database I was able to replicate in a project on 1 run on 5/6.
Here is the exception. It would be better to init counter to 0 when you init your persistence layer.
MongoDB.Driver.MongoCommandException: Command findAndModify failed: E11000 duplicate key error collection: jarvis-saga.Commits_sequence index: _id_ dup key: { : "Commits" }.
   at MongoDB.Driver.Core.WireProtocol.CommandUsingCommandMessageWireProtocol`1.ProcessResponse(ConnectionId connectionId, CommandMessage responseMessage)
   at MongoDB.Driver.Core.WireProtocol.CommandUsingCommandMessageWireProtocol`1.<ExecuteAsync>d__13.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MongoDB.Driver.Core.Servers.Server.ServerChannel.<ExecuteProtocolAsync>d__29`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MongoDB.Driver.Core.Operations.CommandOperationBase`1.<ExecuteProtocolAsync>d__28.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MongoDB.Driver.Core.Operations.WriteCommandOperation`1.<ExecuteAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MongoDB.Driver.Core.Operations.FindAndModifyOperationBase`1.<ExecuteAttemptAsync>d__27.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MongoDB.Driver.Core.Operations.RetryableWriteOperationExecutor.<ExecuteAsync>d__3`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MongoDB.Driver.Core.Operations.RetryableWriteOperationExecutor.<ExecuteAsync>d__2`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MongoDB.Driver.OperationExecutor.<ExecuteWriteOperationAsync>d__5`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MongoDB.Driver.MongoCollectionImpl`1.<ExecuteWriteOperationAsync>d__95`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MongoDB.Driver.MongoCollectionImpl`1.<UsingImplicitSessionAsync>d__101`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NStore.Persistence.Mongo.MongoPersistence`1.<GetNextId>d__28.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NStore.Persistence.Mongo.MongoPersistence`1.<AppendAsync>d__29.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Jarvis.Common.Shared.Utils.NStore.JarvisNStoreWrappedPersistence.<AppendAsync>d__8.MoveNext() in C:\develop\Proximo\Jarvis\src\Intranet\Jarvis.Common.Shared\Utils\NStore\JarvisNStoreWrappedPersistence.cs:line 57
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NStore.Core.Streams.OptimisticConcurrencyStream.<AppendAsync>d__15.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NStore.Domain.Repository.<SaveAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Jarvis.Framework.Bus.Rebus.Integration.Adapters.RebusSagaAdapter`3.<Handle>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Rebus.Pipeline.Receive.HandlerInvoker`1.<Invoke>d__14.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Rebus.Pipeline.Receive.DispatchIncomingMessageStep.<Process>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Rebus.Events.Steps.MessageReceivedStep.<Process>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Rebus.Sagas.LoadSagaDataStep.<Process>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Rebus.Pipeline.Receive.ActivateHandlersStep.<Process>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Rebus.Pipeline.Receive.HandleRoutingSlipsStep.<Process>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Rebus.Pipeline.Receive.DeserializeIncomingMessageStep.<Process>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Rebus.Pipeline.Receive.HandleDeferredMessagesStep.<Process>d__12.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Rebus.Retry.FailFast.FailFastStep.<Process>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Rebus.Retry.Simple.SimpleRetryStrategyStep.<DispatchWithTrackerIdentifier>d__10.MoveNext()

Closed in 0.10.1 hotfix.


There should be documentation for Powershell.

There should be similar documentation for Powershell.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 578368f2-ff6f-b692-c843-706daf73be16
Version Independent ID: f560235a-7cc0-9ac5-7694-40ce6382e49f
Content: Azure Quickstart - Run Batch job - CLI
Content Source: articles/batch/quick-create-cli.md
Service: batch
GitHub Login: @dlepow
Microsoft Alias: danlep

https://docs.microsoft.com/en-us/azure/batch/quick-run-dotnet
This should also be doable thru PowerShell.

@Ayanmullick - Ayan, thanks for the good suggestion. This is already on our backlog list of doc work, and I'll note your request in the item to help prioritize it. In the meantime, if you haven't already, please check out this article about getting started with Batch PowerShell. Cheers.
#please-close

the "getting started with Batch PowerShell" document documents Powershell steps for right up to before executing a job.  The requested document should be in the 'Next Steps' section of this page. Any tentative timescale for ETA? weeks, months, quarters?

The below code will apply an application package to auto-pool nodes, create a job and task and delete the pool once the job is complete.
#Batch account context
$context = Get-AzureRMBatchAccount -AccountName <>
$batchapp = Get-AzureRmBatchApplication -AccountName <> -ResourceGroupName <> -ApplicationId testapp -ErrorAction stop

$appObject = New-Object Microsoft.Azure.Commands.Batch.Models.PSApplicationPackageReference
$myarray = New-Object System.Collections.Generic.List[$appObject]
$appref = New-Object Microsoft.Azure.Commands.Batch.Models.PSApplicationPackageReference
$appref.ApplicationId = $batchapp.id

#Application package version
$appref.Version = "0.2"
$myarray.Add( $appref )

#Pool configuration
$PoolInformation = New-Object -TypeName "Microsoft.Azure.Commands.Batch.Models.PSPoolInformation"
$PoolInformation.AutoPoolSpecification = New-Object -TypeName “Microsoft.Azure.Commands.Batch.Models.PSAutoPoolSpecification”
$PoolInformation.AutoPoolSpecification.AutoPoolIdPrefix = “Foo”
$PoolInformation.AutoPoolSpecification.PoolSpecification = New-Object -TypeName “Microsoft.Azure.Commands.Batch.Models.PSPoolSpecification”
$PoolInformation.AutoPoolSpecification.PoolSpecification.TargetDedicatedComputeNodes = 2
$PoolInformation.AutoPoolSpecification.PoolSpecification.TargetLowPriorityComputeNodes= 2

$configuration = New-Object -TypeName "Microsoft.Azure.Commands.Batch.Models.PSCloudServiceConfiguration" -ArgumentList @(5,"*")
$PoolInformation.AutoPoolSpecification.PoolSpecification.CloudServiceConfiguration = $configuration
$PoolInformation.AutoPoolSpecification.PoolSpecification.VirtualMachineSize = "Large"
$PoolInformation.AutoPoolSpecification.PoolLifetimeOption = 1
$PoolInformation.AutoPoolSpecification.PoolSpecification.ApplicationPackageReferences = $myarray

New-AzureBatchJob -Id testjob8 -PoolInformation $PoolInformation -BatchContext $Context -OnAllTasksComplete TerminateJob

#Addig tasks to job
New-AzureBatchTask -JobId  testjob8 -CommandLine 'cmd /c "powershell -executionpolicy bypass -File %AZ_BATCH_APP_PACKAGE_testapp#0.2%\testbatch.ps1' -Id "task123" -BatchContext $context


Don't assume a fixed length for endowed_accounts

maybe it should be mentionned in the doc that the first half of the endowments account are choosen as members

It's not mentioned that we replace endowed_accounts with a default value either. I'm not sure and it's also more file local and nothing that should be used from outside of this file IMHO.


Create index.html

want to add a js file too once this is merged

const path = require('path');
const express = require('express');
const WebSocket = require('ws');
const app = express();
const WS_PORT  = 65080;
const HTTP_PORT = 80;
const wsServer = new WebSocket.Server({port: WS_PORT}, ()=> console.log(WS Server is listening at ${WS_PORT}));
/* Allows multiple clients to connect*/
let connectedClients = [];
wsServer.on('connection', (ws, req)=>{
console.log('Connected');
connectedClients.push(ws);
ws.on('message', data => {
    connectedClients.forEach((ws,i)=>{
        if(ws.readyState === ws.OPEN){
            ws.send(data);
        }else{
            connectedClients.splice(i ,1);
        }
    })
});

});
app.get('/client',(req,res)=>res.sendFile(path.resolve(__dirname, './client.html')));
app.listen(HTTP_PORT, ()=> console.log(HTTP server listening at ${HTTP_PORT}));
/* server.js */

I tried to push through cli but was getting

ok done

Please check the readme.md file properly,, full instruction there


Install-DbaInstance: Added parameters for collations

Type of Change

[ ] Bug fix (non-breaking change, fixes # )
[x] New feature (non-breaking change, adds functionality, fixes #8007 )
[ ] Breaking change (effects multiple commands or functionality, fixes # )
[ ] Ran manual Pester test and has passed (.\tests\manual.pester.ps1)
[ ] Adding code coverage to existing functionality
[ ] Pester test is included
[ ] If new file reference added for test, has is been added to github.com/sqlcollaborative/appveyor-lab ?
[ ] Unit test is included
[ ] Documentation
[ ] Build system

See linked issue for details.

Looks great to me! @nvarscar can you give it a look? 🙇🏼

Thank you both 😊


how to change only Labels in a saved dataset ?

I wonder if we can modify the Labels in a large saved dataset pickle file ?

Hallo, sorry for the long absence.
At the moment the only way I see it would be to load it from disk like:
ds =  MemoryGraphList() ds.load("filapth/datasetname.pickle")
and then simply change the labels you want. Just note that you should do all this before running e.g. ds.clean
ds[101].assign_property("graph_label", 10.2)
you can if you want manually add the ID of the graph:
for i in range(len(ds)):     ds[i].assign_property("manual_id", i)

works perfectly, exactly what i was looking for!


Generated instructions in automated test can't execute correctly on Windows

Description
Reproduction steps

Start the Ruby LSP using a certain editor
Open the ruby test file(for example test\requests\definition_expectations_test.rb)
click run in terminal attached to a test function
the instruction is like bundle exec ruby -Itest xxx/test/requests/definition_expectations_test.rb --name DefinitionExpectationsTest\#test_jumping_to_method_definitions_when_declaration_exists
output is not correct:

Code snippet or error message
# Running tests with run options --name DefinitionExpectationsTest\#test_jumping_to_method_definitions_when_declaration_exists --seed 868:

Finished tests in 0.001863s, 0.0000 tests/s, 0.0000 assertions/s.

0 tests, 0 assertions, 0 failures, 0 errors, 0 skips

but If I modifed the instruction into bundle exec ruby -Itest xxx/test/requests/definition_expectations_test.rb --name DefinitionExpectationsTest#test_jumping_to_method_definitions_when_declaration_exists (just delete a '\')
and then, the ouput is better:
# Running tests with run options --name DefinitionExpectationsTest#test_jumping_to_method_definitions_when_declaration_exists --seed 47762:

.

Finished tests in 0.050074s, 19.9706 tests/s, 19.9706 assertions/s.

1 tests, 1 assertions, 0 failures, 0 errors, 0 skips

The os is win11, I run it on VSCode 1.88.1. The extension edition is v0.5.20. But the starting info in the output window is Initializing Ruby LSP v0.16.7，I'm not very sure the role of edition in this phenomenon. I hope somebody can check it. Thanks!

@Super-Xray Can you try this branch please? https://github.com/Shopify/ruby-lsp/pull/2109

Sure!

I think this problem has been solved. Thanks! @andyw8


The end of GoodTwitter

https://twitter.com/ZusorOW/status/1258885451055800320

will it really be the end of goodtwitter ?

Thanks for making this work as long as it could!


Handle clicking outside the table

When loosing focus from the table by clicking somewhere else, it should remove tabindex. This can be done with a listener on the document, looking if target has a parent inside the table. Possibly also send something to nebuala via the useAccessibility hook.

Requirements:

When clicking outside a table, the tabindex should be removed from the previously focused cell.


Consider bumping up stack LTS to newer version

With lts-3.2 from 2015-08-24 I'm kinda scared to install this tool with stack in 2017. Bumping up LTS to something rather new, like current one — lts-8.19 from 2017-06-18 makes it easier to install. Because nobody wants to store lts-3.2 cache in his home directory... And new versions of libraries also can increase tool quality (probably).

haha, fair point. Could you send me a PR that does this, if possible? :)

@bollu Yes, I can :)


Rework StartExampleDownloads.ps1

Right now, this scripts downloads and processed every SPDownload.txt file separately. As we now often have the same download file (TPM, ME) for several models, we should try to download the files only once to save bandwidth and time.

And we should change the name of the script as well...

How would you account for different models using the same download? Would you separate the downloads from the models by having a separate softpaq folder, and having the model configurations point to the softpaqs?

@napzero: The basic idea was that StartExampleDownloads.ps1 would first scan for any SPDownlod.txt and create an internal list that includes the download URL and the target path. Then, all duplicate URL would be removed and the path would be "attached" to already existing download URL.
That way, a URL that is used by more than one model would only be downloaded once and then copied to all folders that used that URL. That would have been especially useful for the TPM update downloads as we have more than 10 models that all used the TPM update.
However, since we have the \Shared folder from v4 onwards, there are very few duplicate downlaods. For example, the TPM update is now stored only once inside the \Shared\TPM SLB 9670 folder so it's also downloaded only once. The same applies to BIOS updates for model families as EliteBook 830/840/850 and other files.
I will close this issue.


add support for array of array

This PR is for https://github.com/NVIDIA/spark-rapids/issues/2011

build

build

build

Can we please not rebase. It can really confuse github with outstanding commits. Just merge.

build

build

Please do not rebase your patch. Just merge in the new branch. It is much less likely to confuse github.


Not an Issue, possible enhancement

Original report by HiddenVision (Bitbucket: HiddenVision, GitHub: HiddenVision).

Just a quick thought after getting things to work on My Gen 3 Dot.
Many thanks to @Arpad for his thoughts and Json extracts, worked first time.
I edited the Library so that the name I pass is automatically prep-ended with the Device ID.
I found that programming multiple devices with the same code led to confusion (on my part) as all the names were the same. Obviously the usable name can be edited in Alexa app.
Just thinking this is a cute mod for someone to add into the core code.
I admit I am using EspAlexa Library but I think the logic stands for most Alexa libs.
Example line
EspalexaDevice* d = new EspalexaDevice(deviceName+encodeLightId(currentDeviceCount), callback, initialValue);

//The +encodeLightId(currentDeviceCount) is the bit I added

For example I set the name to “Socket_” or “Light_” and end up with a unique name for every device programmed.
I suspect there is a better method but this worked first time round so I left it be.
Sorry if this upsets anyone.
Hv..

@HiddenVision
I normally just handle this in my code, example:
for (int i = 0; i < 8; i++)
        {
                ESP_LOGI(TAG, "Adding fauxmo device %s %c (%d)", deviceBasename, i + 49, i + 49);
                snprintf(d, 15, "%s %c", deviceBasename, i + 49);
                // TODO - get these from flash
                strcpy( deviceName[i], d);
                fauxmo.addDevice(d);
}

It might be worth thinking about adding an option to do this automatically in the future, however.

So if you program multiple devices with the same binary will they all have unique names?
That was the point of the initial post.
I am fully sorted by the added code I suggested so no need for me to add anything new.

I suggest renaming the issue so we can help find a permanent solution for everyone about the same...

So if you program multiple devices with the same binary will they all have unique names?

Yes. As of a6c0300580a6d570c4d677d8252e990338c213b9 the uniqueid is generated based on a hash of the device's MAC and the name you give to the device.

So if you program multiple devices with the same binary will they all have unique names?

Yes. As of a6c0300580a6d570c4d677d8252e990338c213b9 the uniqueid is generated based on a hash of the device's MAC and the name you give to the device.


Add fanquake dev key

Add fanquake's key fingerprint to the contact page.

ACK

Tested ACK fcb9e23a99723e395b8bb5c11c60ad62e8e63c82 .  Long-established contributor, established maintainer, and this PR is signed by someone already on this list.  I've never verified fanquake's key in person, but it matches trusted sources I have going back several years.
This PR has been open without objection for almost a week, so going to merge now.


@conda-forge-admin, please add bot automerge

@conda-forge-admin, please add bot automerge

Hi! This is the friendly automated conda-forge-webservice.
I just wanted to let you know that I added bot automerge in conda-forge/types-pymysql-feedstock#6.


If I star your repo can you star mine?(CONNECT)

Yes.  Not a problem.

Thank you very much have a great day!

You to!


Newsboat does not render article links from FreshRSS

Newsboat version (copy the output of newsboat -v or the first line of git show):
Newsboat 2.24.0 - https://newsboat.org/
Copyright (C) 2006-2015 Andreas Krennmair
Copyright (C) 2015-2021 Alexander Batischev
Copyright (C) 2006-2017 Newsbeuter contributors
Copyright (C) 2017-2021 Newsboat contributors

Newsboat is free software licensed under the MIT License. (Type `newsboat -vv' to see the full text.)
It bundles:
- JSON for Modern C++ library, licensed under the MIT License: https://github.com/nlohmann/json
- optional-lite library, licensed under the Boost Software License: https://github.com/martinmoene/optional-lite
- expected-lite library, licensed under the Boost Software License: https://github.com/martinmoene/expected-lite

Newsboat 2.24.0
System: Darwin 18.7.0 (x86_64)
Compiler: g++ 4.2.1 Compatible Apple LLVM 11.0.0 (clang-1<IP_ADDRESS>)
ncurses: ncurses 5.7.20081102 (compiled with 5.7)
libcurl: libcurl/7.54.0 LibreSSL/2.6.5 zlib/1.2.11 nghttp2/1.24.1 (compiled with 7.54.0)
SQLite: 3.24.0 (compiled with 3.24.0)
libxml2: compiled with 2.9.4

Config file (copy from ~/.newsboat/config or ~/.config/newsboat/config):
browser "lynx %u"

#feed-sort-order title-desc

text-width 72

cache-file ~/.config/newsboat/cache.db

# -- navigation ----------------------------------------------------------------

goto-next-feed no

bind-key j down feedlist
bind-key k up feedlist
bind-key j next articlelist
bind-key k prev articlelist
bind-key J next-feed articlelist
bind-key K prev-feed articlelist
#bind-key j down article
#bind-key k up article
bind-key j next article
bind-key k prev article
bind-key UP prev article
bind-key DOWN next article
bind-key ENTER down article
bind-key \ up article
bind-key SPACE pagedown article
bind-key , pageup article
bind-key ; macro-prefix

# -- bookmarks -----------------------------------------------------------------

bookmark-cmd "~/bin/bookmark.sh"
bookmark-interactive no
bookmark-autopilot yes
bind-key b bookmark

# -- colours ------------------------------------------------------------------
# Dark solarized color scheme for newsbeuter, based on
# Ethan Schoonover's Solarized.
#
# In order to use this color scheme, you must first configure
# your terminal emulator to use the Solarized palette.
# See <http://ethanschoonover.com/solarized/> for more information.

#color background   default   default
#color listnormal   default   default
#color listfocus    black     yellow
#color info         default   black
#color article      default   default

#color background           color255 yellow
#color listnormal           color255 color0
color listfocus            color0 yellow bold
#color listnormal_unread    color0 yellow bold
color listfocus_unread     color0 yellow bold
color info                 color255 color31
#color article              color255 color0

# highlights
highlight article "^(Date):.*$" green default
highlight article "^(Title):.*$" yellow default
highlight article "https?://[^ ]+" red default
highlight article "\\[image\\ [0-9]+\\]" green default
highlight article "^(Author):.*$" blue default

urls-source "freshrss"
freshrss-url "https://[internal-page]/api/greader.php"
freshrss-login "user"
freshrss-password "pass"
freshrss-min-items 200
freshrss-flag-star "s"
freshrss-show-special-feeds "false"

refresh-on-startup no
reload-threads 3

Steps to reproduce the issue:

Configure an internal FreshRSS installation.

Point newsboat to the FreshRSS installation.

No links are displayed from feeds. Footnote links still display.

Other info you think is relevant:
I have attached screenshots comparing what newsboat shows with FreshRSS and with a direct URLs configuration, for the well-known site Daring Fireball.
With FreshRSS

With urls file

The images above show that there is no 'Link' attribute on the FreshRSS output.

Thanks for the report!
@PetraOleum, apparently this block doesn't always work:
https://github.com/newsboat/newsboat/blob/9c6233a5941b13fc5608b9a731feabc44ff2ace5/src/freshrssapi.cpp#L471-L479
Perhaps we could take the link from some other field, like id that we use for GUID?

If you don't mind I'll look at this in the weekend. In the worst case it's possible that freshrss just isn't collecting the link itself - I'll have to do some testing.

Sounds good, thanks!

Hi, I just realised I was running an outdated version of FreshRSS. I've since updated and links are rendering fine. Happy to close this issue, and thanks for your prompt response.


CodeQL workflow and issue and PR templates

Introduces checks that will build the code and perform code analysis on PRs and on the master branch.
Also introduces PR and issue templates.

Thanks guys for doing this whole effort!
Civ1 still my favourite game.

Thanks guys for doing this whole effort! Civ1 still my favourite game.

Thanks, mine too ;)


WidgetMakeCommand.php missing "/"

Installed and tried to create a widget.  Got error stating stub could not be found.
Works with:
line 39:
 return __DIR__.'../../../resources/stubs/widget.stub';
changed to:
 return __DIR__.'/../../../resources/stubs/widget.stub';

Thank you, fixed in 9d47f690fb22ba8f230c719292e2a322aa015340


Multi-Root Workspace: Exclude Subfolders

Since vscodebot states:
We look at the number of votes the issue has received and the number of duplicate issues filed
I'm asking this again even if there's already this feature request.
We should be able to have the ability to hide subfolders in workspaces settings. Possibly on a directory basis.
Just like this:
{
  "folders": [
    {
      "path": "foo",
      "ignore": [
        "baz/"
      ]
    },

Sublime Text allow this kind of configuration. Can't believe we cannot do on vs code.
Thank you

You can use files.exclude setting for excluding sub folders.
cc: @isidorn


Move to abstract types

Currently the code is poorly written in a way that uses Float64 or Complex{Float64}. This should be changed to use abstract types. This happens in both function arguments as well as when allocating arrays such as in init_sim.

Abstract types are horrible for performance so I moved to parametric structs. Fixed in bf5aceae267b370fdb4b9a5cddddb06076978b68


How to resize animated webp/gif?

Hi guys, I'm trying to resize an animated webp but no success.
I saw today that some new API calls are being developed, but I thought that the animated resizing functionality already works in some way.
A little sample code:
      sharp("originalFilePath.webp", { animated: true })
        .resize({
          width: 200,
          height: 200,
        })
        .webp({
          pageHeight: 200,
          loop: 0,
          delay: 130,
          force: true,
        })
        .toFile("resizedFilePath.webp", (err, info) => {
          if (err) {
            console.error(err);
          }
         console.log('file resized')
        });

My objective is to resize and convert animated images back and forth using webp and gif extensions to accomplish a more cheaper bandwidth cost for my current project.
Thank you in advance for your time.

Hi, if you're resizing n pages at once you'll need to set the height to a multiple of n:
  .resize({
-    width: 200,
+    width: 200 * n,
    height: 200,
  })

I've tried that, but the animation changed completely. Look like an animation of first page moving from down to top.

Sorry, I got width and height the wrong way round in the diff, it should be:
  .resize({
    width: 200,
-   height: 200,
+   height: 200 * n,
  })

Yeah, I've noticed that but the result is the same.

You're probably seeing the effects of rounding "errors" at the top and bottom of each page/frame, which is pretty much why #2275 exists as a future possible enhancement.
Are you able to provide a sample input file? You'll probably only get any success when resizing to exactly 1/2 or 1/4 etc. of the original dimensions as things stand.

I've noticed that if the new size is below 50% the glitch appears. Above 50% the image is resized correctly.
The Image is here

Also, for some reason resized images are heavier than the original.
Original:

512x512: 32k
Resized:
512x512: 60k
300x300: 92k
256x256: 68k

Please subscribe to #2275 for updates.

Ok, thank you very much @lovell.

is the problem solved?


Include subsumer classes in Exomiser output

Exomiser format looks like this:
HP_0000033      MP_0004014      0.47058823529411764     2.6591341898231766      MP_0002160;
HP_0000033      MP_0020998      0.43478260869565216     3.300796258200725       MP_0009198;
HP_0000033      MP_0020957      0.36363636363636365     2.6591341898231766      MP_0002160;
HP_0000033      MP_0020869      0.2127659574468085      3.300796258200725       MP_0009198;
HP_0000033      MP_0020824      0.5     3.300796258200725       MP_0009198;
HP_0000033      MP_0002771      0.4     3.300796258200725       MP_0009198;
HP_0000033      MP_0008995      0.35    2.5184899921386523      HP_0000078;
HP_0000033      MP_0008994      0.4444444444444444      2.6591341898231766      MP_0002160;
HP_0000033      MP_0008984      0.375   3.283570838358968       HP_0000811;

With columns being subject, object, Jaccard, Resnik, and subsumer, in that order.
We would like to include something like this subsumer column in the output, with the knowledge that it may always or nearly always be a Upheno node.

@caufieldjh how far are you going with this exomiser work? We are doing a lot of the same things now - especially setting up these comparative experiments etc. If you continue investing in things like converting OAK semantic similarity profiles to Exomiser tables, then we absolutely should add this to the ETL preprocessing in pheval rather than here..

This isn't the OAK semantic similarity, it's a faster implementation that can be part of the usual Monarch KG build. If that essentially mean "run it through Pheval" then that's entirely doable, but for now we have a few different use cases and one of them is just seeing if we're improving on existing phenodigm scores. That use doesn't really require a whole ETL pipeline. In the slightly longer term though, sure, it can absolutely be used in the course of building Pheval comparisons.

I don't care that much if it's going through oak, but it should export the oak semantic similarity format, not some custom table. In any case, this is all too close to home - We have a concrete mandate to implement what you are doing in PhEval, and I am more than happy for you to implement it, but we need to coordinate the code a bit - for now the most important thing is that semsim exports the standard semantic similarity profile format Chris defined, and I thought this was on someone's priority list?
Is there any way I can get a complete description of where your task starts and where it ends? What are you testing, tuning etc and what is the exact goal?


Default to original

I have found that sometimes images fail to process in the cache but if I set their default as the original image, it acts as a fallback and the image successfully loads.
For example this might fail to load anything:
Eg: https://myimagecache.site.com?url=image.somesite.net/duck.jpg&w=1000&we&n=-1
But this will successfully load the image for the end user:
https://myimagecache.site.com?url=image.somesite.net/duck.jpg&w=1000&we&n=-1&default=https://image.somesite.net/duck.jpg
The issue is now I have a 2x long URL which is ugly and could be an issue if the source image URL was already very long.
It would be good if I could just flag the default to fall back to the source image if something goes wrong.
EDIT:
Iv found a real world example of this happening - this URL will fail to load with a 404 error:
https://images.melonland.net/?url=https%3A%2F%2Fi.redd.it%2Fbest-dinosaur-themed-megazord-v0-bfs51aj5im591.jpg%3Fwidth%3D640%26format%3Dpjpg%26auto%3Dwebp%26s%3D34cf5a6d5bc4dac3e699f8f44c294113e7f4e850&w=1200&fit=inside&we&q=85&il&n=-1
However when you include a default value with the same source URL it works fine:
https://images.melonland.net/?url=https%3A%2F%2Fi.redd.it%2Fbest-dinosaur-themed-megazord-v0-bfs51aj5im591.jpg%3Fwidth%3D640%26format%3Dpjpg%26auto%3Dwebp%26s%3D34cf5a6d5bc4dac3e699f8f44c294113e7f4e850&w=1200&fit=inside&we&q=85&il&n=-1&default=https%3A%2F%2Fi.redd.it%2Fbest-dinosaur-themed-megazord-v0-bfs51aj5im591.jpg%3Fwidth%3D640%26format%3Dpjpg%26auto%3Dwebp%26s%3D34cf5a6d5bc4dac3e699f8f44c294113e7f4e850

Nope you understood perfectly! Your suggestion would be ideal. In my case because Im running a forum, the images being proxied are user input, getting them to display is the priority and errors are not really an issue, as long as most images are working ok.

@kleisauke: Shall we put this on the to-do list?

This has been implemented with commit adc6b9ab6d0be1c0c65b37757577ba5b6db6f122, which has just been rolled out to production. Thanks for reporting this!
Documentation will be updated later to reflect this.

Documentation has been updated.
https://wsrv.nl/docs/format.html#default-image


Use of transform_spec in make_batch_reader leads to tensorflow error when column is missing values

The conversion back to arrow from pandas in ArrowReaderWorker._load_rows() loses type information when all rows in the loaded row group are missing values for a given column.  From the pyarrow.Table.from_pandas documentation:

Be aware that Series of the object dtype don’t carry enough information to always lead to a meaningful Arrow type. In the case that we cannot infer a type, e.g. because the DataFrame is of length 0 or the Series only contains None/nan objects, the type is set to null. This behavior can be avoided by constructing an explicit schema and passing it to this function.

The result is the following error when reading the corresponding batch in the TensorFlow Dataset:
InternalError: Unsupported object type NoneType

Example
from pathlib import Path
import numpy as np
import pandas as pd
import petastorm
from petastorm.unischema import UnischemaField
from petastorm import tf_utils
from petastorm.transform import TransformSpec

# Create parquet dataset
data_path = Path('/path/to/data.parquet')
data_pd = pd.DataFrame({'s': ['A', None, None, None]})
data_pd.to_parquet(data_path, row_group_size=2)

# Read with petastorm
noop_transform_spec = TransformSpec(lambda x: x, edit_fields=[UnischemaField('s', np.str_, (), nullable=True)])

reader = petastorm.make_batch_reader(data_path.as_uri(),
                                     workers_count=1,
                                     shuffle_row_groups=False,
                                     num_epochs=2,
                                     transform_spec=noop_transform_spec)

reader.next() # output: inferred_schema_transformed_view(s=array(['A', 'None'], dtype='<U4'))
reader.next() # output: inferred_schema_transformed_view(s=array([None, None], dtype=object)) <-- type is missing

# Read with tensorflow
dataset = tf_utils.make_petastorm_dataset(reader)
dataset_itr = dataset.as_numpy_iterator()

dataset_itr.next() # output: inferred_schema_transformed_view(s=array([b'A', b'None'], dtype=object))
dataset_itr.next() # InternalError: Unsupported object type NoneType

Workaround
Modify all TransformSpec funcs to replace string columns missing all values with None strings.
def transform_spec_with_workaround(rows_pd: pd.DataFrame) -> pd.DataFrame:
    ...  # custom transformation

    for c in rows_pd.columns:
        if rows_pd[c].dtype == object and rows_pd[c].isnull().all():
            rows_pd[c] = pd.Series(len(rows_pd) * ['None'])

    return rows_pd

Full Trace
---------------------------------------------------------------------------
InternalError                             Traceback (most recent call last)
~/conda/udrgpu2_86_0/lib/python3.6/site-packages/tensorflow/python/eager/context.py in execution_mode(mode)
   2101       ctx.executor = executor_new
-> 2102       yield
   2103     finally:

~/conda/udrgpu2_86_0/lib/python3.6/site-packages/tensorflow/python/data/ops/iterator_ops.py in _next_internal(self)
    757             output_types=self._flat_output_types,
--> 758             output_shapes=self._flat_output_shapes)
    759

~/conda/udrgpu2_86_0/lib/python3.6/site-packages/tensorflow/python/ops/gen_dataset_ops.py in iterator_get_next(iterator, output_types, output_shapes, name)
   2609     except _core._NotOkStatusException as e:
-> 2610       _ops.raise_from_not_ok_status(e, name)
   2611     except _core._FallbackException:

~/conda/udrgpu2_86_0/lib/python3.6/site-packages/tensorflow/python/framework/ops.py in raise_from_not_ok_status(e, name)
   6842   # pylint: disable=protected-access
-> 6843   six.raise_from(core._status_to_exception(e.code, message), None)
   6844   # pylint: enable=protected-access

~/conda/udrgpu2_86_0/lib/python3.6/site-packages/six.py in raise_from(value, from_value)

InternalError: Unsupported object type NoneType
	 [[{{node PyFunc}}]] [Op:IteratorGetNext]

During handling of the above exception, another exception occurred:

InternalError                             Traceback (most recent call last)
<ipython-input-45-55cc7ab782db> in <module>
----> 1 dataset_itr.next()

~/conda/udrgpu2_86_0/lib/python3.6/site-packages/tensorflow/python/data/ops/dataset_ops.py in next(self)
   3778
   3779   def next(self):
-> 3780     return nest.map_structure(lambda x: x.numpy(), next(self._iterator))
   3781
   3782   def __next__(self):

~/conda/udrgpu2_86_0/lib/python3.6/site-packages/tensorflow/python/data/ops/iterator_ops.py in __next__(self)
    734
    735   def __next__(self):  # For Python 3 compatibility
--> 736     return self.next()
    737
    738   def _next_internal(self):

~/conda/udrgpu2_86_0/lib/python3.6/site-packages/tensorflow/python/data/ops/iterator_ops.py in next(self)
    770   def next(self):
    771     try:
--> 772       return self._next_internal()
    773     except errors.OutOfRangeError:
    774       raise StopIteration

~/conda/udrgpu2_86_0/lib/python3.6/site-packages/tensorflow/python/data/ops/iterator_ops.py in _next_internal(self)
    762         return self._element_spec._from_compatible_tensor_list(ret)  # pylint: disable=protected-access
    763       except AttributeError:
--> 764         return structure.from_compatible_tensor_list(self._element_spec, ret)
    765
    766   @property

~/conda/udrgpu2_86_0/lib/python3.6/contextlib.py in __exit__(self, type, value, traceback)
     97                 value = type()
     98             try:
---> 99                 self.gen.throw(type, value, traceback)
    100             except StopIteration as exc:
    101                 # Suppress StopIteration *unless* it's the same exception that

~/conda/udrgpu2_86_0/lib/python3.6/site-packages/tensorflow/python/eager/context.py in execution_mode(mode)
   2103     finally:
   2104       ctx.executor = executor_old
-> 2105       executor_new.wait()
   2106
   2107

~/conda/udrgpu2_86_0/lib/python3.6/site-packages/tensorflow/python/eager/executor.py in wait(self)
     65   def wait(self):
     66     """Waits for ops dispatched in this executor to finish."""
---> 67     pywrap_tfe.TFE_ExecutorWaitForAllPendingNodes(self._handle)
     68
     69   def clear_error(self):

InternalError: Unsupported object type NoneType
	 [[{{node PyFunc}}]]

Thanks for bringing up the issue.
I tried forcing a strict schema type when converting back from pandas to pyarrow table here.  Unfortunately the "trip" to pandas and back is not transparent. One type that ended up being tricky is a pa.timestamp. I was not able to make it work without implementing some weird conversion code which I am not sure if it will be robust enough.
Another approach that I tried is to use pyarrow.Table type as an argument to TransformSpec function (instead of pandas dataframe). However, working with pyarrow.Table type in the transform spec function appears to be inconvenient since pa.Table is immutable and pandas API is much more convenient for a transformation implementation.
So after doing all this, I would suggest sticking with the current implementation. While it's not perfect, I was not able to find a better alternative that would not require implementation of potentially non robust code.
Would appreciate your thought and suggestions on this matter.

Thanks for looking into this!  What was the issue with pa.timestamp?  I'm not seeing the timestamp-specific conversion code in https://github.com/uber/petastorm/pull/750.
For our purposes, using the proposed workaround is not a big deal as we only use a single TransformSpec to implement nested array support as described here.  The nested array support via TransformSpec is itself a bit of a hack.  Has any thought been given to natively supporting nested arrays?

The issue with timestamps I ran into was the automatic conversion of the timestamp into a datetime object - it would not be automatically converted back into pa.timestamp64. However, I just noticed that there is a date_as_object=False object (*.to_pandas(date_as_object=False)) that let me keep dates from being converted to datetimes. Reopened the #750 - let's see if  I can get all tests to pass now.


Unable to run npm start

from couple of days ago unable to run npm start due to this error?

@sky107 I encountered this problem with a different app. I'm not sure if the solution applies here.

Delete all node modules
Ensure you are using node version 15 or higher (required by some traccar dependencies)
Run npm install

Hopefully this helps

I have tried to run npm run build
The build completes but when I run the build. It just doesn't come up.
The UI is blank

What exactly are you trying to run?

I had to install @material-ui/core, @material-ui/icons and eslint-plugin-jsx-a11y to get the page to load

You shouldn't need any additional dependencies.


Range types WIP

This is a draft mostly to get some feedback on how we want to tackle certain technical aspects of range types.

PostgreSQL comes with the following built-in range types:

int4range — Range of integer, int4multirange — corresponding Multirange
int8range — Range of bigint, int8multirange — corresponding Multirange
numrange — Range of numeric, nummultirange — corresponding Multirange
tsrange — Range of timestamp without time zone, tsmultirange — corresponding Multirange
tstzrange — Range of timestamp with time zone, tstzmultirange — corresponding Multirange
daterange — Range of date, datemultirange — corresponding Multirange

They have their own OID and we probably want to map corresponding range<blah> types in EdgeDB to those. Unlike arrays, there does not seem to be a nice reference to them in pg_type like a typarray column for arrays. So we might want to pre-generate ranges for our built-in scalars ourselves and have a few more built-in range types (with stable OIDs) than Postgres.

I haven't touched any aspect of this that relates to protocol when sending range values over the wire (I assume we send them as Postgres would in binary format with minimal meddling, though). Our clients will need to learn about these types, too. That's another aspect that this PR does not touch at the moment. In practical sense, it's useful for testing to have clients that can receive these values, but not crucial as values can be compared inside EdgeDB instead.

Integer ranges are peculiar in Postgres in that they always try to get into the "[a,b)" format, no matter how you actually specify the boundaries. Since integers are discrete, this trick is generally always possible. E.g.
V2f145d6d13_edgedb=# select int8range(1, 4, '[]');
 int8range
-----------
 [1,5)
(1 row)

V2f145d6d13_edgedb=# select int8range(1, 4, '(]');
 int8range
-----------
 [2,5)
(1 row)

This does not affect unbounded ranges as infinity is not included in the bound:
V2f145d6d13_edgedb=# select int8range(NULL, 4);
 int8range
-----------
 (,4)
(1 row)

V2f145d6d13_edgedb=# select lower_inc(int8range(NULL, 4));
 lower_inc
-----------
 f
(1 row)

This feature may catch someone by surprise, but is not fundamentally wrong as the produced ranges are mathematically equivalent.
This is not happening for a contiguous type:
V2f145d6d13_edgedb=# select numrange(1.1, 4.4);
 numrange
-----------
 [1.1,4.4)
(1 row)

V2f145d6d13_edgedb=# select numrange(1.1, 4.4, '()');
 numrange
-----------
 (1.1,4.4)
(1 row)

What's our formalism for declaring a custom range type? At the moment we can just start with ranges for built-in scalars, but do we want to have more customization available in principle? Is there going to be a create ... type that does this job?
Currently all built-in range types are declared implicitly, much like arrays, but we might want explicit declarations even for std to have proper control over canonical or subtype_diff functions (subtype_diff, for example has to return a float8 in all cases, so just automarically grabbing - operator won't quite work and it makes for efficient indexing, so omitting it is not a great idea).
So would these be a new type, like create range type Foo extending range<float64> or would they be a flavour of create scalar type like enums currently are?

Casting.

We obviously want a textual representation of our intervals and the ability to cast to and from str. We also probably want that to be consistent with json. To disambiguate between an array [1.2, 3.4] and range(1.2, 3.4, inchigh := true) we can make the string representation simply be identical to a constructor call. That's verbose, so maybe this could be a good use for [a..b) format instead.

Casting between range types is evidently not supported in Postgres:
V2f145d6d13_edgedb=# select int4range(1, 4) :: int8range;
ERROR:  cannot cast type int4range to int8range
LINE 1: select int4range(1, 4) :: int8range;

This is a niche thing, so we can probably omit casting between ranges of compatible types (like within integer ranges or within float ranges). In principle, though, we could define casts for these if this kind of convenience is needed.

This is a draft mostly to get some feedback on how we want to tackle certain technical aspects of range types.

There is a reverse relationship via pg_type.rngsubtype.

What's our formalism for declaring a custom range type?

There is none, range types are supposed to be automatically defined for all point scalars.

We obviously want a textual representation of our intervals and the ability to cast to and from str.

Um, we don't have text I/O for other container types, so I think the need to support direct string cast for range types isn't as obvious.  I would punt this.

Casting between range types is evidently not supported in Postgres

This is not a problem.  There are other casts that aren't supported directly which we implement via eta-expansion (e.g. tuples etc).


WW-5339 Add option to block custom OGNL maps

WW-5339
Having consulted with a security researcher - there is still some risk here if there exists a custom map implementation whose java.util.Map methods pose some risk. This is because ognl.MapPropertyAccessor#getProperty does not consult the MemberAccess policy for the inherent java.util.Map methods.
Given Struts does not rely on custom OGNL Map implementations, we should add an option to block this capability.

@lukaszlenart
Done :)
https://github.com/apache/struts-site/pull/215

Just tested the Showcase app and it doesn't look good ;-)
[WARN ] ognl.SecurityMemberAccess (SecurityMemberAccess.java:205) - Declaring class [interface java.util.Map$Entry] of member type [public abstract java.lang.Object java.util.Map$Entry.getKey()] is not allowlisted!
[WARN ] ognl.SecurityMemberAccess (SecurityMemberAccess.java:163) - Access to non-public [final java.lang.Object java.util.HashMap$Node.key] is blocked!
[WARN ] ognl.SecurityMemberAccess (SecurityMemberAccess.java:163) - Access to non-public [protected java.lang.String org.apache.struts2.components.UIBean.key] is blocked!
[WARN ] ognl.SecurityMemberAccess (SecurityMemberAccess.java:205) - Declaring class [interface java.util.Map$Entry] of member type [public abstract java.lang.Object java.util.Map$Entry.getValue()] is not allowlisted!
[WARN ] ognl.SecurityMemberAccess (SecurityMemberAccess.java:163) - Access to non-public [java.lang.Object java.util.HashMap$Node.value] is blocked!
[WARN ] ognl.SecurityMemberAccess (SecurityMemberAccess.java:163) - Access to non-public [protected java.lang.String org.apache.struts2.components.UIBean.value] is blocked!
...


BrambleUrlCodeHints should be recursive

When we suggest URLs, we do so only in the current dir.  We should be walking the entire project dir and finding all possible options.  We can do this with Filer and sh.find(): https://github.com/filerjs/filer#find.

The fix would mean changing how BrambleUrlCodeHints.prototype._getUrlList works, https://github.com/humphd/brackets/blob/bramble/src/extensions/default/BrambleUrlCodeHints/main.js#L147


Fix typo

Change word exmaple for example
Motivation
Fix a typo on the Docs

@facebook-github-bot shipit


Added methods to read and delete app/user id

Added methods and tests for reading and deleting from the app-id and user-id endpoints

Thank you!


Add a GUID to the process start message

Needed for Edge integration.
Details forthcoming.

we want to add a machine/host id. so let's try to add a machine_id key with the value from /etc/machine-id to the process start message.


iOS target is installed, yet not found?

I've followed what seems to be the standard process of rustup target add... and yet cannot build those targets. (I have the Xcode toolchain installed, including command-line tools. I build plenty of open source using configure and make so that's not an issue.) Instead of further description, let me just illustrate. Here's what's installed:
$ rustup show
Default host: x86_64-apple-darwin

installed toolchains
--------------------

stable-x86_64-apple-darwin (default)
nightly-x86_64-apple-darwin

installed targets for active toolchain
--------------------------------------

aarch64-apple-ios
armv7-apple-ios
armv7s-apple-ios
i386-apple-ios
x86_64-apple-darwin
x86_64-apple-ios

active toolchain
----------------

stable-x86_64-apple-darwin (default)
rustc 1.32.0 (9fda7c223 2019-01-16)

Yet if I try to build using one of the targets I added:
$ cargo build --target=aarch64-apple-ios
error: failed to run `rustc` to learn about target-specific information

Caused by:
  process didn't exit successfully: `rustc - --crate-name ___ --print=file-names --target aarch64-apple-ios --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro` (exit code: 1)
--- stderr
error: Error loading target specification: Could not find specification for target "aarch64-apple-ios"
  |
  = help: Use `--print target-list` for a list of built-in targets

And, in fact:
$ rustc --print target-list | grep ios
$

And of course, I was trying cargo-lipo first, and got the same error. Specifying a single specific architecture, creating a new empty library crate & and just adding a minimal [lib] config, were some attempts to troubleshoot this.
So how can this happen? And how to fix it?

Can you check which xcode tools are active? xcode-select --print-path
ios targets will only work with the full xcode install, it won't work with the command-line tools. It can be selected with sudo xcode-select --switch /Applications/Xcode.app/Contents/Developer

Ah, that was it. From the docs I got that I needed a full install with both UI & command-line tools, and that xcode-select needed to have been run. I did not catch that the path needed to point to within the full install. When I get a chance, I will review docs to see if it's my oversight or they need clarification, and put in a PR if I think clarification is needed. Thanks.

Well, haha, the incomplete instructions are not part of this project--just all blog posts etc, so no PR coming...


Fix wording and padding in user_info_popover_content.

Still some issues, e.g. this will break under i18n if the text is long, and it would be nice to center the block of 4 links with respect to the popover. But hopefully an improvement for now.

Automated message from Dropbox CLA bot
@rishig, it looks like you've already signed the Dropbox CLA. Thanks!

Nice, merged, thanks @rishig!

For the docs change, do we also want to suggest that folks use the new fa- class names?

yeah, seems reasonable. Want to take a stab, @adnrs96?

Sure. I can get docs, a update for this.


Make allocators optional for custom struct binding

My custom struct is a wrapper around a [26]u8
pub const ID = struct {
    data: [26]u8,
    // .. various helper methods to generate IDs

    /// Turn ID into a SQL argument
    pub fn sql(self: *const Self) sqlite.Text {
        return sqlite.Text{ .data = &self.data };
    }
}

While exploring things, I noticed that bindField() override exists, and that I could use it to remove the need to write id.sql() in my queries, however, that method interface would not suffice for my usecase:
https://github.com/vrischmann/zig-sqlite/blob/b98ebdf4a9f038ca6322e6091e45c4998efad946/sqlite.zig#L1661
If I wanted to remove id.sql() calls, I would then need to add unnecessary allocators to every SQL call that gives an ID struct as  an argument, even though the memory that I want to pass to SQLite is already available on the stack by the time I want to execute the query. Example
            try db.exec(
                \\ delete from tag_names
                \\ where core_hash = ?
            ,
                .{},
                .{self.core.id}, // self.core.id is a full ID struct, which means its inner [26]u8 is available for us to use directly.
            );

Maybe we could check argument count at comptime and not require allocators if argument count is 1?

While exploring #118, I noticed that passing the entire struct as a parameter to the query turns it into stack memory by the time custom bindField is run, which breaks the assumption I had that bindField would have a custom struct with the lifetime of the entire statement, instead it has the lifetime of the bindField() internal function, so the const pointer that I take in the custom one is completely invalid by the time the query is executed. Ooops.
Not sure how to approach this one other than going back to ID.sql() as with that I know that I am using the ID with the correct lifetime. Suggestions welcome.

Hi. I'm currently triaging the issues, is this issue still relevant ?


SEHE exception

Hello,
I have downloaded your Llama.Examples project, downloaded ggml-model-f32-q4_0.bin from https://huggingface.co/AsakusaRinne/LLamaSharpSamples/tree/v0.3.0/LLaMa/7B
And when I publish code for win-x64 runtime, then I am not able to run it on Windows. I am getting following errror:
Fatal error. System.Runtime.InteropServices.SEHException (0x80004005): External component has thrown an exception.
Repeat 2 times:
--------------------------------
   at LLama.Native.NativeApi.llama_init_backend()
--------------------------------
   at LLama.Native.NativeApi..cctor()
   at LLama.Native.NativeApi.llama_context_default_params()
   at LLama.Utils.llama_init_from_gpt_params(LLama.LLamaParams ByRef)
   at LLama.LLamaModel..ctor(LLama.LLamaParams, System.String, Boolean, System.String)
   at LLama.Examples.ChatSession..ctor(System.String, System.String, System.String[])
   at Program.<Main>$(System.String[])

Hi, which version of LLamaSharp.Backend did you install?

Hi I have installed
<PackageReference Include="LLamaSharp" Version="0.3.0" />
<PackageReference Include="LLamaSharp.Backend.Cpu" Version="0.3.0" />

Unfortunately I couldn't reproduce this error. Is there any other information? For example your cpu and system info, your runtime version, etc..

I tried to run/debug it on Macbook M1 and figured it won't work, so I released it for win-x64 and run it on following system:

Dotnet runtime is 7.0.302 and project's target framework is <TargetFramework>net7.0</TargetFramework>

Mac has not been supported now because I don't have a MacBook and it seems this is the reason for the error. Could you please try building from source with llama.cpp? If time is not limited for you, you can also wait for the new release of LLamaSharp.Backend. Someone in discord promised to compile it in Mac and make a PR.

I will try llama.cpp. But the main thing that bothers me is, that it doesn`t work even on Win-server.

Does it throw the same exception when running on windows server?

This exception reported above is from Windows server.
Exception which I am getting from MacOS is different and I didn't reported because I thought it is not supported. I saw no reason to bother you with M1 issues 😄

I see. How many memories are there in your windows server? I am guessing if it's the memory problem because LLM requires a lot of memories.

21,6 GB total - 18,6 GB free
What is memory requirement?

The memory is enough for 7B and 13B models. Unfortunately I don't have windows server, either. What I can do is using cloud server, but I have only 2-4 GB memories, which is too few to run LLM. I think compiling llama.cpp in debug mode and run it directly will give you more information about this error.

Thanks for Your help.

Thanks to the community effort, LLamaSharp.Backend.Cpu 0.3.1 has included MAC support. I hope it is of some help to you. :)


outgoing_tx_data.to_address should not carry UA

Why is zingolib providing the compound UA in the to_address field, leaving the recipient_ua field empty for a transaction's outgoing_tx_data? I expected the to_address to always be the literal address used in the transaction, and if Zingolib is going to preserve the UA's other receivers in the change memo, that that would only appear under recipient_ua

On matrix, altum suggested that the to_address is dropped on save and replaced with the value from recipient_ua. This is information lossy. It is not reconstructed on deserialization, and I'd argue cannot be, since the receiver that had previously been chosen cannot be reliably guessed from the many receivers in the UA later on, when perhaps zingolib supports more receiver types than it did when creating that transaction.


Debugger is slow since .vsix 5.0.270354

Describe the bug
I try to debug codeunit 99000854 "Inventory Profile Offsetting", Function "CalculatePlanFromWorksheet" and step over function (F10). Every step takes about 3 seconds.
I tested version 5.0.254558 and the debugger is faster. Even the version 5.0.271898 is slow.
To Reproduce
Steps and to reproduce the behavior:

Open Base Application.
Set a breakpoint in the function "CalculatePlanFromWorksheet". E.g. in the line, where the function "DemandToInvtProfile is called.
Press Ctrl+Shift+F5 (Debug without Publishing).
Start the planning worksheet and wait until the debugger hits the breakpoint.
Try to step through the lines. It takes very long.

See gif attached.
A colleague of mine told me that they even have this problem outside of the Base Application.
Expected behavior
Performance like in in vsix 5.0.254558.
Screenshots
With version 5.0.254558:

With latest version (5.0.271898):

5. Versions:

Vsix
5.0.271898
AL Language:
Version: DE Business Central 16.0 (Platform 16.0.11233.12436 + Application 16.0.11240.12474)
Visual Studio Code
Version: 1.44.2 (system setup)

It looks as a big bug, but I wander why other users do not complain about it?

Attached the log file. The file was located at:
C:\Users<user>.vscode\extensions\ms-dynamics-smb.al-5.0.271898\bin\win32
DebuggerServices.log
It seems that the latest version 5.0.277602 improved the time that the debugger needs to jump from line to line (F10). Nevertheless, still needs long to connect to vscode and not as fast as in the version 5.0.254558.
Attached the file for 5.0.277602 as well:
DebuggerServices.log

Thanks for the log. That explains it. It is a bug.

@AndreasGloeckner
We have pushed a new vsix version where hopefully debugger stack trace handling - the issue seen from your log file -  should be improved.

I am closing this issue because it appears that it has been resolved. Please open a new issue if you believe this has not been resolved and reference the current issue.


add GET timeout to mirror tests

Summary of changes

Changes introduced in this pull request:

Addresses a timeout failure in the CI. Now, each GET will take no more than 30s. Given many futures are spawned in parallel, this should not compound. Note that this may be too little on a horrible internet connection, but on a regular one, it should be fine. The test will fail if both sources are completely down (this will mean we don't have enough redundancy)

Reference issue to close (if applicable)

Closes
Other information and links

Change checklist

[x] I have performed a self-review of my own code,
[x] I have made corresponding changes to the documentation,
[x] I have added tests that prove my fix is effective or that my feature works (if possible),
[x] I have made sure the CHANGELOG is up-to-date. All user-facing changes should be reflected in this document.

This timeout is getting a bit too high. I'm inclined to leave only the check for existence (that is to say, no 404), and allow handle proper replication in https://github.com/ChainSafe/forest-iac/issues/323. What do you think @lemmih?

The reason for the timeouts is most likely bad internet connection for the fuzzy worker (~3 Mbps).


Associate Each Component With a Color

Each component to be drawn onto the graph should have an assigned color for easy identification.

Just implemented!


[Snyk] Security upgrade minio from 7.2.7 to 7.2.11

Snyk has created this PR to fix 1 vulnerabilities in the pip dependencies of this project.
Snyk changed the following file(s):

python/pyfunc-server/requirements.txt

⚠ Warning
merlin-sdk 0.33.0 has requirement protobuf<4.0.0,>=3.0.0, but you have protobuf 4.24.4.
merlin-sdk 0.33.0 has requirement cloudpickle==2.0.0, but you have cloudpickle 1.2.2.
kfserving <IP_ADDRESS> requires minio, which is not installed.
botocore 1.33.13 has requirement urllib3<1.27,>=1.25.4; python_version < "3.10", but you have urllib3 2.0.7.

[!IMPORTANT]

Check the changes in this PR to ensure they won't cause issues with your project.
Max score is 1000. Note that the real score may have changed since the PR was raised.
This PR was automatically created by Snyk using the credentials of a real user.
Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information:
🧐 View latest project report
📜 Customise PR templates
🛠 Adjust project settings
📚 Read about Snyk's upgrade logic

Learn how to fix vulnerabilities with free interactive lessons:
🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.

:tada: Snyk checks have passed. No issues have been found so far.
:white_check_mark: security/snyk check is complete. No issues have been found. (View Details)
:white_check_mark: license/snyk check is complete. No issues have been found. (View Details)

:tada: Snyk checks have passed. No issues have been found so far.
:white_check_mark: code/snyk check is complete. No issues have been found. (View Details)


Viewer role doesn't inherit permissions

Attempted Debugging

[X] I have read the debugging page

Searched GitHub Issues

[X] I have searched GitHub for the issue.

Describe the Scenario
Steps to reproduce:

Create a new user with the Viewer role.
Login as Admin
Enable Public access.
Under Roles, set Viewer role to View All Books.
Create a Book.
Edit Book permissions add Override for Public to View.
Log out.
Log in as the Viewer user.
When accessing /books, cannot see the Book.

Here's a video in case that's helpful.
https://www.loom.com/share/b362b2ee465642d68e721c18193de5cf
Exact BookStack Version
v24.05.3
Log Content
None
Hosting Environment
Production: PHP8.2 (ea-php82) on EL8 cPanel shared hosting
Local: PHP8.1 on MacBookPro (2019) via ngrok

Hi @CdrMarks,
In your demo video, the "Logged in user"/viewer role shows 0 users assigned.
Can you check if the "Ryan" user has roles assigned to them (edit their user profile) otherwise they may not be receiving the permissions you're expected them to have.

Nice catch @ssddanbrown.  I associated the user with the role and all is working as expected.
Thanks!


Minor bug with empty files: QTextCursor::setPosition: Position '-1' out of range

If an empty file is opened with Spyder when spyder is launched, a message is printed:
QTextCursor::setPosition: Position '-1' out of range
This message can be suppressed by changing one line in plugins/editor/widgets/codeeditor.py:
(around line 4973)
# Text might be after a dot '.'
if text == '':
#was: cursor.setPosition(pos - 1, QTextCursor.MoveAnchor)
cursor.setPosition(max(0, pos - 1), QTextCursor.MoveAnchor)
Hope that helps.
Thank you

Hi @mvillion thank you for the feedback and giving a possible solution to the issue! I think I have been seeing that message around too but didn't know from where that was coming from :)
Would you like to submit a PR with your proposed fix? If that is the case I think this should go to the 5.x branch and you can check the contribution guide for more info about how to do the PR at: https://github.com/spyder-ide/spyder/blob/5.x/CONTRIBUTING.md (if you have any question feel free to ask here)
Otherwise we can try to do the PR with your proposed fix :+1:
Let us know!

Hello,
I am not very familiar with writing PR on github. As this is a very small change, I am hoping that maybe you can fix it while solving another bug?

No problem! I will then submit a PR with your proposed fix and put you as a co-author there :+1:


[Security] Workflow main.yml is using vulnerable action actions/checkout

The workflow main.yml is referencing action actions/checkout using references v1. However this reference is missing the commit a6747255bd19d7a757dbdda8c654a9f84db19839 which may contain fix to the some vulnerability.
The vulnerability fix that is missing by actions version could be related to:
(1) CVE fix
(2) upgrade of vulnerable dependency
(3) fix to secret leak and others.
Please consider to update the reference to the action.

Feel free to open a PR to update the github config, but otherwise this looks like it may not really have much effect on this repository.


allow feature-gated integration tests for unimplemented functionality

Summary
Allows use of integration tests feature gated by the "unimplemented" feature e. g.
#[cfg(feature="unimplemented")]
#[test]
fn sub_num_hex_float_lower() {
    expect_stdout(vec!["%a", ".875"], "0xep-4");
}

Also adds make SPEC=y option (e.g. make UTILS='...' SPEC=y test) that will include these tests and include the --no-fail-fast option.
Motivation
Enable writing of integration tests at a mental distance from the implementation
Writing tests for implemented functionality only requires knowing which features are implemented which usually entails an intimate knowledge of the implementation
Enable eventual automatic enumeration of which features of a utility are implemented.
Because good practice is to write integration tests for each feature implemented anyway, if we allow writing the tests before implementation, this allows us to express (through code and automatically through a nice-looking CI results page if desired) what features are and are not yet implemented without the need to, in addition to writing the inevitable tests, maintain a separate, outside-the-source-code checklist or set of issue tracking issues.
This would also be the first step if we wanted to eventually make very rough, ballpark automatic estimates of how close a utility is to completion; we could use these estimates to communicate to enthusiast end-users which utilities are most ready for them to try out and test.
Enable separate pull requests for tests and implementation while still demonstrating implementation in the implementation PR.
I have no preference on whether that's a convention or not, but this makes conventions like that possible. By providing an implementation of a feature (e.g. cat -n) and simultaneously removing the "unimplemented" feature gate from the cat -n tests, if the PR passes tests it means the implementation is successful in regard to those tests.

Probably a good idea. Maybe this should use the ignore attribute.
SPEC is a bit unintuitive, though i lack ideads for a better name.

I should clarify that. It's purely a semantic marker as to why it's ignored. Ignored is also used for

computationally expensive things
a category of tests which could be, fuzzily, conceived as different than unimplemented: known bugs, which indicate required maintenance vs new development.

By specifying the ignore reason through feature-gating we can decide to run computationally expensive tests without having to remember / compare against comments which ones are failing due to being unimplemented.

hmm, actually you are right, it probably should be ignore until we actually encounter a need to differentiate them.

I was thinking about replacing
#[cfg(feature="unimplemented")]

with something like
#[cfg_attr(not(feature="unimplemented"), ignore)]

That way the tests are still compiled even if not run (which avoids bitrot),
and you can see in the output that there are ignored tests.
Since the feature is only relevant for the tests maybe test_unimplemented is
a better name. Or to be in line with the SPEC=y arg test_spec.

alright, went with test_unimplemented

lineitem fixed


[The One Ring Second Edition] fix roll button names

Changes / Comments

The roll buttons can now be called with chat commands or dragged to the macro quickbar.

Roll20 Requests
Comments are very helpful for reviewing the code changes. Please answer the relevant questions below in your comment.

[x] Does the pull request title have the sheet name(s)? Include each sheet name.
[x] Is this a bug fix?
[ ] Does this add functional enhancements (new features or extending existing features) ?
[ ] Does this add or change functional aesthetics (such as layout or color scheme) ?
[ ] If changing or removing attributes, what steps have you taken, if any, to preserve player data ?
[ ] If this is a new sheet, did you follow Building Character Sheets standards ?

If you do not know English. Please leave a comment in your native language.

Character Sheet Info Roll20 Internal Use only.


Use Resty client for tendermint

Overview
In the Cosmos SDK Unchained implementation, we are currently using a tendermint client generated from the Cosmos SDK Open API Spec. This helped us get off the ground, and provides value in terms of generated models, however it also removes some of our control with regard to request/response handling.
The Resty http client package makes it easy to interact with remote http services, has built in retry capabilities, and requires no extra build time activities. We're already using resty in the Cosmos implementation. Remove all references to the OpenAPI generated tendermint client in Unchained. We will continue using OpenAPI to generate the Unchained server spec.
Reference
The generated client as well as its Makefile reside here: https://github.com/shapeshift/unchained/tree/develop/go/pkg/tendermint
The tendermint client to remove is instantiated here: https://github.com/shapeshift/unchained/blob/22f417c79b21ea8a574eb989b56eaf291888b176/go/pkg/cosmos/cosmos.go#L75
and injected to the HTTPClient https://github.com/shapeshift/unchained/blob/22f417c79b21ea8a574eb989b56eaf291888b176/go/pkg/cosmos/cosmos.go#L87
It is then used downstream such as:
https://github.com/shapeshift/unchained/blob/22f417c79b21ea8a574eb989b56eaf291888b176/go/pkg/cosmos/history.go#L38
Here's an example of the Cosmos (LCD) Resty client being used to retrieve account data:
https://github.com/shapeshift/unchained/blob/22f417c79b21ea8a574eb989b56eaf291888b176/go/pkg/cosmos/account.go#L30
Note the definition of the Account type on Line 18. We'll need to create models for the types we are consuming from the Tendermint API. See the existing generated model for TxSearchResponseResult for reference. https://github.com/shapeshift/unchained/blob/22f417c79b21ea8a574eb989b56eaf291888b176/go/pkg/tendermint/client/model_tx_search_response_result.go#L18
Acceptance Criteria

The generated tendermint client here no longer exists.
All functionality that leverages tendermint continues to function as intended (particularly Tx History)

Ownership

[X] As the sponsor of this bounty I will review the PR

Bounty Hunters

[ ] Join the ShapeShift Discord
[ ] Check out the Contribution Guidelines
[ ] Include an expected completion timeline in the work plan when you apply for this bounty on Gitcoin!

@trungtin - do you have experience writing Go?

@0xean yep I have 2 yrs of experience with golang

awesome, thanks! approving you now.

@trungtin - thank you so much for the PR, will get it paid out immediately! Would love if you would come introduce yourself in our discord and pick up some more bounties.


Dev cognitiveservices form recognizer v2.1 preview.2

What's the change
This change updates CognitiveServices/FormRecognizer layout API to add "boundingBox" property to "DataTable" object.
Why make this change

API consistency.
All other entities in layout API's output, including text lines, words, selectionMarks, even tableCell have "boundingBox" property, except for table itself.
Ease of use.
There are feedback after rolling out of layout API, both internally and externally, that the bounding box of table will be a welcome addition to the API. Without it, users have to calculate convex hull of all enclosed table cells, that is not only inaccurate, but also difficult for developer.

How the change is made

API verison bump
This new property will start to appear in 2.1-preview.2. OpenAPI Hub was used to copy existing API definition and examples, update readme.md, and change API version string in all related files.
Adding new "boundingBox" property
This is a simple change to add one more property "boundingBox" into existing "DataTable" definition. The "boundingBox" shares the same definition as any other "boundingBox" in layout API.

/azp run automation - sdk

/azp run automation - sdk

@xingong202 is the idea to merge this change as part of https://github.com/Azure/azure-rest-api-specs/pull/11088 or manage it in separate PRs?
Also, do you have an ETA on when the change is going to be merged (either here or in the other PR)?

Closing it due to inactivity, feel free to reopen (after rebase with master) or a new PR with the changes.


Add rudimentary admin interface for Datasets

Some admin interface stuff I put together to test out #6. It's very minimal, but better than what we have now :slightly_smiling_face:

Seems like there's some build errors, let's hold off merging until those are fixed.

Same as the other PR, we can merge this whenever you're ready.