New Data Integration

Dear Saturn team,
Say we have some mouse in-house datasets that we'd like to integrate with your mammalian cell atlas under the same embedding. Without re-training with our datasets, is there any quick way that we can find the macrogene values for each our cells? How can leverage the genes_to_macrogenes.pkl file of the mammalian cell atlas together with the count matrix of our own data?
Thanks

You could use the centroids to take a weighted average of expression.
However, I would recommend retraining.

Could you please elaborate on how exactly we can "use the centroids to take a weighted average of expression", in particular the weighted average part? It seems in the manuscript that the macrogene expression values $e_{c}$ is defined by $e_{c}=ReLU(LayerNorm(W^T_{s}log(X^s_{c}+1)))$. Is this what you mean by "weighted average".
Thanks!

Yes. Since you are not using these as inputs to a neural network, you can just ignore the ReLU and LayerNorm parts.


Two-level indexing for parallelization of I/O

At the moment, yt employs a global index to locate data. For many datasets (incl. RAMSES), data are ever spread within files with a mapping that relies on some space-filling process (e.g. the Hilbert curve or the Z-curve).
This mapping is typically lightweight and could be used as an intermediate coarse-grained indexing (similar to what has been done with SPH, as far as I could understand).
Having a lightweight coarse-grained indexing (with no false-negative!) would allow it to be copied on however many MPI tasks yt is running on. As a second step, the current indexing (from file to position in file) would allow finer refinement (do we need to read any data? where is it located on disk?). This would allow distributing the list of files to be read to the different tasks deterministically based on the intersection with the coarse index, and each task would subsequently read the files that actually intersect.
Some of the code required for this is already in place for the RAMSES dataset (https://github.com/yt-project/yt/pull/4734 for having a two-level index, https://github.com/yt-project/yt/pull/4730 for how one could use this to parallelize I/O).

We should do this.  I had an attempt at something related but not identical that I can no longer find; essentially, it was to make .index a list instead of a single attribute, and then have multiple grid index objects that could live on it.  Same for particles, which are already closer to that.
Perhaps what we could try:

Make .index a wrapper, which acts basically as an index collection.  It would need to respond to all of the same API calls, but we'd want to explore how the attributes that are different between different index types would be supplied.
Have it allow multiple index objects, for grids, cells, particles, etc.  And it can have multiple of each type.  (This would enable overlaying particles on grids more easily, for instance.)
Then this top-level index could mediate the coarse indexing, which particles handle already.


List of API supported language codes

There is a list at https://www.modernmt.com:

Albanian, Arabic, Bosnian, Bulgarian, Catalan, Chinese, Chinese Traditional, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, German, Greek, Hebrew, Hungarian, Icelandic, Indonesian, Italian, Japanese, Korean, Latvian, Lithuanian, Macedonian, Malay, Maltese, Norwegian Bokmal, Norwegian Nynorsk, Polish, Portuguese, Portuguese Brazil, Romanian, Russian, Serbian Latin, Slovak, Slovenian, Spanish, Spanish Latin America, Swedish, Thai, Turkish, Ukrainian, Vietnamese.

There is a list at https://github.com/modernmt/modernmt/blob/2bbad392209d900c3290a88d7217eef72bee22a3/src/textprocessing/src/main/java/eu/modernmt/cleaning/filters/lang/OptimaizeLanguageFilter.java
    private static final HashSet<String> SUPPORTED_LANGUAGES = new HashSet<>(
            Arrays.asList("af", "an", "ar", "ast", "be", "br", "ca", "bg", "bn", "cs", "cy", "da", "de", "el", "en",
                    "es", "et", "eu", "fa", "fi", "fr", "ga", "gl", "gu", "he", "hi", "hr", "ht", "hu", "id", "is",
                    "it", "ja", "km", "kn", "ko", "lt", "lv", "mk", "ml", "mr", "ms", "mt", "ne", "nl", "no", "oc",
                    "pa", "pl", "pt", "ro", "ru", "sk", "sl", "so", "sq", "sr", "sv", "sw", "ta", "te", "th", "tl",
                    "tr", "uk", "ur", "vi", "wa", "yi", "zh"));

But the two lists are not the same.  The second list has codes like ast for Asturian, which is not included in the other list.  There are always some nuances around languages with variants, like Chinese, Spanish, Portuguese and Norwegian, sometimes an API requires the variant when a target language but combines them when a source language, so it would be good to have the actual list and definitive wording on questions like whether pt is supported and whether it defaults to pt-BR or pt-PT.
The right place would be https://www.modernmt.com/api/ under General aspects, as languages are used by many API calls.

Hi @bittlingmayer ,
there is a bit of confusion here, at it is definitely our fault. The problem is that in the could-based service documentation, an important API is missing: https://api.modernmt.com/languages
The result is the complete list of language combinations supported by the cloud service - we will add that API as soon as possible to the online docs.
However, please notice that the cloud service and the open-source technology are two distinct and different things.
ModernMT open-source is a "technology" for machine translation, so it does not have a "list" of supported languages. In fact, the languages a ModernMT engine supports depend exclusively n how many (and which) models you have trained. If I want to support English-to-Italian and French-to-English you need to use ModernMT open-source to train 2 models on your data en-it and fr-en.
The ModernMT cloud service is a service that uses the ModernMT machine translation technology with some enhancements plus a large number of models trained on billions of high-quality parallel text. So ModernMT cloud service offers the language combinations for which we have trained and provided a neural model.
Finally, that list you found at OptimaizeLanguageFilter.java is just the list of languages supported by the OptimaizeLanguageFilter component, which is a text filter we use during training to remove sentences in a wrong language. So they are not correlated to languages supported for translation.
I hope I have covered all your points! I'm closing this issue right now just because is not really related to ModernMT open-source and because it can create confusion by mixing the two different things. Anyways, feel free to post new messages if I have not answered completely.

Thanks, Davide, makes sense.


How to insert json data to jsonb column

Hello,
I have Postgres DB and a table with jsonb-type field in it.
I need to insert a row to it and can not do it with error "Could not serialize value"
    settings: {
      id: Generated<number>;
      application_id: number;
      data: Record<string, any>
      created_at: Generated<Date>;
      updated_at: Generated<Date>;
    }

    await db.insertInto('settings')
    .values([{
        application_id: 1,
        data: {
            'test': 'test',
            'com': 'test'
        }
    }]).execute()

Is any way to do it with kysely?
Thank you!

Hey :wave:
There is currently no built-in way to do it.
Try this.

@igalklebanov Thank you!

Hello,
I tried

Hey 👋
There is currently no built-in way to do it.
Try this.

Hello, @igalklebanov and everybody else.
I tried the above solution, but it didn't work. So I tested the following and now it works.
function json<T> (object: T): RawBuilder<T> {
  return sql`cast (${JSON.stringify(object)} as jsonb)`
}


Using byte array from C SDL to display a frame

Hello guys,
This is more of a question rather than a proper issue but I've been trying for a while and I can't get my head around this:
(Bit of background)
I've got a C .dll that is running an NES emulator, in the past, I used SDL to present the frames but I wanted widgets and whatnot so I moved it to Avalonia. I've got my code so that C builds a frame based on a byte[] and sends it to C# to be displayed. That byte[] is formed of 3 bytes per pixel (RGB).
What I get on the C# side is a struct with a IntrPtr and a size int component.
(Now for the problem/question)
I've created a view with this component in it:
<Image Source="{Binding Patterntable}" Width="512" Height="240"></Image>
And I tried to crete a bitmap from Avalonia.Media.Imaging; using different constructors. None of them worked. For instance the one that receives an PixelFormat, IntPtr, Width, Height, Stride comes back as null when I pass the pointer that I get from C.
The one that receives a Stream which I create by marshalling the IntPtr throws an exception saying "Can't create the Bitmap with the given information"
Should I use a different Bitmap? Maybe one from SharpDX? and how do I hook that one in XAML? I had a look at the samples but I still can't get it to work.
I would really appreciate help here if you don't mind :)

I can, no problem at all. Just add another byte to each pixel. Still, how would that work? Would I be able to use Avalonia.Media.Imaging.Bitmap then?

    public class MyBitmapControl : Control
    {
        private WriteableBitmap _bitmap =
            new WriteableBitmap(new PixelSize(255, 255), new Vector(96, 96), PixelFormat.Bgra8888);

        public override unsafe void Render(DrawingContext context)
        {
            uint color = 0xff000000u;
            using (var l = _bitmap.Lock())
            {
                var ptr = (uint*)l.Address;
                for(var x=0; x<l.Size.Width; x++)
                for (var y = 0; y < l.Size.Height; y++)
                {
                    color++;
                    ptr[y * l.RowBytes / 4 + x] = (uint)color;
                }
            }
            context.DrawImage(_bitmap, 1, new Rect(_bitmap.Size), Bounds, BitmapInterpolationMode.HighQuality);
        }
    }

Hey! Thanks a lot for the example!
I assume that would create a MyBitmapControl that I can use in my XAML. Would the bindings still work? so instead of having
        public Bitmap Patterntable
        {
            get {return patterntable;}
            set {this.RaiseAndSetIfChanged(ref patterntable, value);}
        }

I would now have
        private WriteableBitmap patterntable;
        public WriteableBitmap Patterntable
        {
            get {return patterntable;}
            set {this.RaiseAndSetIfChanged(ref patterntable, value);}

}

The problem with WriteableBitmap is that currently you have to call InvalidateVisual on the control every time when bitmap gets updated. It's a known issue that we hope to get fixed soon-ish.
That's why I've provided you with the example with a custom control.
It's also better to reuse WriteableBitmap instances, since otherwise GC might kick in too late, so your app will be wasting memory.
You can create a new bitmap each time and dispose old instances, but it has some performance penalty, but you'll be able to reuse existing Image control this way.

Ahhh I see. Yup no problem at all calling InvalidateVisual. Thanks a lot for the example sir, you've been really helpful. Let's see if I can make this work once and for all :)
Kudos to you!


RTL

How to RTL the chart?

@AmineHamrouni what are you trying to achieve? RTL text? RTL scales?

@etimberg  Both text and scales

To change the text you can create your canvas like so:
<canvas dir="rtl">

There is no good way to reverse the scales in version 1. In version 2 some scales can be reversed.

THANK YOU VERY MUCH :+1:

Is there any RTL support solution for scales and legends and ...
Version: 2.1.3

i have problem in legend title, in rtl direction.
in rtl, first should be bullet and then title.

@etimberg : given solution is not working ng2-charts or chart.js

public barChartOptions = {
tooltips: {
rtl: true
}
};


Bump rust toolchain to nightly 2023-06-23

Fixes ICE in the Github CI
NodeKey enum size decrease

Is there any insight into exactly what is triggering this ICE? I have a nightly build of buck2 on Linux that gets updated every day in this repository, but I haven't seen any ICEs at all in my builds, despite the fact this seems completely reproducible and happens across multiple OSs/architectures.

I honestly have no idea. I tried permuting a few things before attempting a toolchain upgrade which seems to have removed the ICE entirely. (looks like I should fix the other builders though too)

Bumping is not that easy, because we synchronize the version with the version used in internal non-cargo builds (in particular, size changes will break internal builds).

We have an upgrade to 1.71.0-beta.3 in progress, because of the ICEs with 1.70.0-stable. That will fix the GitHub CI — hopefully by early next week. @diliop

23efaf53ecf6737cb215686e244ff99874f647be

CI builds are working again:


Why new_relixir in this line?

https://github.com/nogates/discorelic/blob/b6629e3e8f5447f9cfe852975d65cc05646a3e72/lib/discorelic.ex#L40

@evg2108 good question! I guess I was doing some tests with the new_relixir app while I was creating this... I think it can be safely removed. Will create a PR at some point probably, although to be completely honest, I haven't used NR in a while and I am kind of not maintaining this project anymore.
Feel free to create a PR by yourself, if the CI passes, I am happy to merge


Don't run filtered out tests with dotnet test

Tested this change in PR https://github.com/dotnet/maintenance-packages/pull/115, but didn't help.

Hmm this should work but let me double check.

This now makes the following attributes work:

SkipOnCIAttribute
SkipOnMonoAttribute
ActiveIssueAttribute
SkipOnPlatformAttribute
PlatformSpecificAttribute
SkipOnTargetFrameworkAttribute
SkipOnCoreClrAttribute
ConditionalClassAttribute


Fix visuals of tutorial notebooks

🚀 Feature Request
There are some problems with visual representation of the tutorial notebooks in the documentation, we should fix them and unify the style of notebooks.
Proposal
Problems in tutorials:

Poor representation of dataframes: bad alignment, it was better in the old version of documentation
Not working table of contents

It isn't critical because TOC is displayed on the right panel of the web-site

Not working binder button
Inconsistent naming of sections across different tutorials, there should be some standardized way

If we use digits we should use it everythere.
We should use the same title hierarchy.

Inconsistent placing of imports. We should place imports the same way in each tutorial.

I'm not sure is that necessary or not.

Inconsistent metion of extras necessary to install. It should be at the very top of the tutorial for each of them.

Add created specification for styling of notebooks into contribution guide.
Test cases
No response
Additional context
Developing in docs-prototype branch.

Closed by #1330.


Use --filter=blob:none to clone CPython faster

# Before
scripts/benchmarks/setup.sh  72,98s user 9,59s system 11% cpu 11:31,73 total
# After
scripts/benchmarks/setup.sh  10,70s user 2,77s system 6% cpu 3:33,95 total

thanks, that's a huge speedup!
i have to try this in the ecosystem ci docker too, cloning is currently the bottleneck there

I can make PR for this too.

The solution used for the ecosystem is much faster than --filter=blob:none.

do you mean the settings below? i'm unfortunately not familiar enough with the internals of git to compare them. https://github.com/charliermarsh/ruff/blob/a32617911a36998d2ad88ee7bafcd7c9e01720f1/scripts/check_ecosystem.py#L43-L54

do you mean the settings below? i'm unfortunately not familiar enough with the internals of git to compare them.
https://github.com/charliermarsh/ruff/blob/a32617911a36998d2ad88ee7bafcd7c9e01720f1/scripts/check_ecosystem.py#L43-L54

Yes, that is what I checked.


Namespaces, packages, Nuget naming

We need a coherent Namespace/package categorization and naming convention. We want different hardware to be split up into different packages. However, there is a lot of code reuse between hardware. Further, we want to follow existing package and namespace naming conventions. MS recommends the following:
<Company>.(<Product>|<Technology>)[.<Feature>][.<Subnamespace>]
What is Product/Technology here?

The targeted technology (Bonsai)?
The spec (ONI)?
The specific hardware (Headstage64)?
Some combination of them?

What do we do with reused, highly-specific code? e.g. ConfigureBNO055?
Please put your proposals below.

Goncalo suggests the following:
OpenEphys.Oni (clroni)
OpenEphys.Oni.Bonsai (core bonsai lib)
OpenEphys.Onix (reusable ONIX components)
OpenEphys.Onix.Headstage64 (example ONIX product)
Jon pretty much agrees with it.

We have created a workflow, utilizing Github Actions, to automatically build/test packages before uploading to NuGet. This is shown in OpenEphys.Sockets.Bonsai.
The general flow is to create a branch or a fork of the repository, make changes, then open a PR to the main branch. After opening the PR, actions are automatically run that verify a version number is increased, and the package is built correctly (i.e., without errors). This is enforced by a number of rulesets that protect the main branch from direct commits, require the test/build to pass before merges can happen, and also enforce a linear commit history to make it easier to go back to a previous version if necessary.
Any updates or recommendations are welcome.

We also need to define conventions for naming nodes that produce and accept data

Right now, most nodes that produce data have Data suffix, but its not consistent
I prefer low level types to have a name like Bno055 -> Bno055Hardware and the data stream is named Bno055 without Data suffix.

In any case, we need a consistent scheme.

@jonnew @aacuevas To follow-up on this I think what we need to agree on now is a partition of which devices go into the common package, and which go into the headstage-specific packages.
Below is a working draft proposed breakdown:
Legend
📦 (package)
🖲 (headstage / hub)
⚙ (device)
Proposed Breakdown
📦OpenEphys.Onix
┣ 🖲BreakoutBoard
┣ ⚙FmcLinkController
┣ ⚙HarpSyncInput
┣ ⚙DS90UB9x
┣ ⚙AnalogIO
┣ ⚙Heartbeat
┣ ⚙LoadTester
┣ ⚙MemoryMonitor
┗ ⚙Test0
📦OpenEphys.Onix.Devices
┣ ⚙Rhd2164
┣ ⚙Bno055
┗ ⚙TS4231
📦OpenEphys.Onix.Headstage64
┣ 🖲Headstage64
┗ ⚙Headstage64ElectricalStimulator
📦OpenEphys.Onix.NeuropixelsV2
┣ 🖲NeuropixelsV2eHeadstage
┣ 🖲NeuropixelsV2eBetaHeadstage
┣ ⚙NeuropixelsV2e
┣ ⚙NeuropixelsV2eBeta
┣ ⚙NeuropixelsV2eLinkController
┗ ⚙NeuropixelsV2eBno055

OpenEphys.Onix.Devices
So the idea for OpenEphys.Onix.Devices is to place common stuff. In one sense I agree, in another sense I don't.

BNO055: Basically agree
RHD2164: gateware implementation is super old and limited. There are headstage-64's in the field that use it, but if we make another FPGA-based headstage with this chip, we will probably change some things that would result in a major revision. For gateware this means a different device entirely, that is aimed at the same physical chip.
Heartbeat: We plan on adding this to all FPGA based headstages, feels like it should go in this library and that OpenEphys.Onix should depend on it.

OpenEphys.Onix

Breakoutboard is a beta test device indicating that this package should be called OpenEphys.Onix.FMC
Loadtester, Test0, and Heartbeat are all generic and used in places other than base system

The take home: I see what you are doing, but to be honest, I think that common Devices should be in their own repo, which is never transformed to Nuget but instead, is brought into packages that need the devices as submodules. This will prevent the scattering of devices all across nuget packages and will provide some coherence in terms of repo locations.

@jonnew This all sounds good, I have updated the above proposal to reflect your suggestions, and I have removed the Devices package. My only remaining question is around the FmcLinkController but we can discuss this in a bit and come back to update the breakdown.

A discussion which came up is how to organize the upcoming NeuropixelsV2 which does not depend on the passthrough device in this package organization. One proposal is to make all the "e" devices be dependent on FmcHost (or another name we come up with for this host) and incorporate that into the package namespace, e.g. OpenEphys.Onix.FmcHost.NeuropixelsV2 would be a package containing all passthrough NeuropixelsV2 devices.

Regarding the last message, the rationale is this: Right now devices that follow the oni spec work perfectly across different hosts (e.g.: the bno055, which I've used on the usb acq. board). However, anything that is passthrough-based is tightly tied to the current host.
Our current plan is to deprecate those, enforcing every device to be a proper oni device (however the hardware wants to accomplish this, in onixv2 with the modules, for example), so those would be normal devices that can be in their normal namespaces. But current passthrough devices, for clarity, we think might be better if tightly tied to the host that they required.

An alternative breakdown is to just keep the current mono-package and rename it to FmcHost to imply that all devices depend on the FMC architecture.
Alternative Breakdown
📦OpenEphys.Onix
📦OpenEphys.Onix.FmcHost
┣ 🖲BreakoutBoard
┣ ⚙FmcLinkController (?)
┣ ⚙HarpSyncInput
┣ ⚙DS90UB9x
┣ ⚙AnalogIO
┣ ⚙DigitalIO
┣ ⚙MemoryMonitor
┣ 🖲Headstage64
┣ ⚙Rhd2164
┣ ⚙Bno055
┣ ⚙TS4231
┣ ⚙Headstage64ElectricalStimulator
┗ ⚙Headstage64OpticalStimulator
┣ 🖲NeuropixelsV2eHeadstage
┣ 🖲NeuropixelsV2eBetaHeadstage
┣ ⚙NeuropixelsV2e
┣ ⚙NeuropixelsV2eBeta
┣ ⚙NeuropixelsV2eLinkController
┗ ⚙NeuropixelsV2eBno055

I think I'm leaning towards:
OpenEphys.Onix.PCIe
Our next gen system is also PCIe, but it does this over an IPASS cable, so that one probably would be OpenEphys.Onix.IPASS or more locally devided into different packages as we have discussed above.

@jonnew the spelling itself is fine, just as you say there might be other PCIe devices in the horizon. More importantly, because ONI is a protocol, it could give the wrong impression that this is all of ONI(x) over PCIe, which it is, but not in that general sense yet, so I wonder if this could bring other problems down the line if we reserve such a general namespace.
I am also thinking about namespace hierarchy. Down the line the modular architecture is still the goal so having OpenEphys.Onix as the core namespace and having sub-namespaces for different "families" makes sense. However, there is no modular OpenEphys.Onix yet, so there are advantages in considering the current system as a self-contained product which is modular within its own product line.
Because of the above I would actually not use OpenEphys.Onix at all yet and instead would form a compound word. My current preference would be something like:

OpenEphys.OnixBreakout
OpenEphys.OnixV1

After sleeping on it, i think the name should be:
OpenEphys.Onix1
Its the first in what i envision will be a collection of onix acquisition systems.

@jonnew Sounds good, my only clarification question is to confirm the capitalisation.
My original suggestion was OniX1 to emphasise that Oni is the standard and X1 is the system model name. I like it better than leaving the 1 in isolation and also means people can refer to "the X1 system", "the X2 system" etc, but maybe that creates confusion with the phonetics of reading the name.
The best example of similar name with or without X and numbers:

DirectX 12 (DX12)
Direct3D 12 (D3D12)


fix(schema): add 'tsvector' to string types

Text Search Vectors are a complex type inside of Postgres, but can generally be expressed as strings within JS.

thanks!


Attractor detection refactoring

Documentation in progress...

@daemontus any thoughts on how to reconcile this with #115? I'm thinking merge this first, then revert #115 to before the package rename and merge that, then make an entirely new pr just for the renaming. Not sure if there's a better way.

I just tried to rebase it locally from #115 and it seems that we can first do #115 and then I'll just update this PR with the new name. Interestingly, other than the the newly created files/functions, most of the changes merged without conflicts... so it should be an easy fix :)

This should now be fully rebased with the latest name change. (bcc: @jcrozum)

@jcrozum I think I resolved the issue with the "constant configuration values": SuccessionDiagram now has a configuration dictionary that stores all of these and you can pass it as a constructor parameter. As such, if you serialize the diagram, the "configuration" is preserved. Furthermore, these are no longer global module "constants" so we don't have to worry about users needing to access/modify them.
Please have a quick look if that looks reasonable to you, and if yes, feel free to merge the PR.

Looks good, I'll merge it.


TypeError: config._d.getTime is not a function. (In 'config._d.getTime()', 'config._d.getTime' is undefined)

Describe the bug
This error pops up : TypeError: config._d.getTime is not a function. (In 'config._d.getTime()', 'config._d.getTime' is undefined)

node_modules/moment/moment.js:238:47 in Moment
node_modules/moment/moment.js:3202:26 in clone
node_modules/moment/moment.js:3332:22 in toISOString

To Reproduce
no idea yet how to reproduce ...
Expected behavior
It should work as previously with version 2.23.0
Screenshots

Smartphone (please complete the following information):

Device: iPhone5S
OS: ios 11.4.1

Moment-specific environment
"moment": "2.24.0" within expo sdk22 app
Additional context
`
TypeError: config._d.getTime is not a function. (In 'config._d.getTime()', 'config._d.getTime' is undefined)
This error is located at:
in FameMore (at connectAlert.js:11)
in ConnectedAlert (at withNavigation.js:17)
in withNavigation(ConnectedAlert) (created by Connect(withNavigation(ConnectedAlert)))
in Connect(withNavigation(ConnectedAlert)) (at FameTop.js:122)
in RCTView (at View.js:44)
in FameTop (at withNavigation.js:17)
in withNavigation(FameTop) (at connectAlert.js:11)
in ConnectedAlert (created by Connect(ConnectedAlert))
in Connect(ConnectedAlert) (at FamePensees.js:41)
in RCTView (at View.js:44)
in FamePensees (created by Connect(FamePensees))
in Connect(FamePensees) (at FameHomeView.js:309)
in RCTView (at View.js:44)
in CellRenderer (at VirtualizedList.js:687)
in RCTScrollContentView (at ScrollView.js:852)
in RCTScrollView (at ScrollView.js:977)
in ScrollView (at VirtualizedList.js:1062)
in VirtualizedList (at FlatList.js:662)
in FlatList (at FameHomeView.js:475)
in RCTView (at View.js:44)
in FameHomeView (at connectAlert.js:11)
in ConnectedAlert (created by Connect(ConnectedAlert))
in Connect(ConnectedAlert) (at ReactMeteorData.js:88)
in ReactMeteorDataComponent (at Home.js:339)
in RCTView (at View.js:44)
in KeyboardAvoidingView (at Home.js:333)
in RCTView (at View.js:44)
in Container (at Home.js:330)
in Home (at connectAlert.js:11)
in ConnectedAlert (at withNavigation.js:17)
in withNavigation(ConnectedAlert) (created by Connect(withNavigation(ConnectedAlert)))
in Connect(withNavigation(ConnectedAlert)) (at SceneView.js:17)
in SceneView (at CardStack.js:466)
in RCTView (at View.js:44)
in AnimatedComponent (at Card.js:12)
in Card (at PointerEventsContainer.js:39)
in Container (at CardStack.js:498)
in RCTView (at View.js:44)
in RCTView (at View.js:44)
in CardStack (at CardStackTransitioner.js:67)
in RCTView (at View.js:44)
in Transitioner (at CardStackTransitioner.js:19)
in CardStackTransitioner (at StackNavigator.js:41)
in Unknown (at createNavigator.js:13)
in Navigator (at createNavigationContainer.js:226)
in NavigationContainer (at AppWithNavigationState.js:66)
in App (created by Connect(App))
in Connect(App) (at app/index.js:242)
in RCTView (at View.js:44)
in AlertProvider (at app/index.js:241)
in Provider (at app/index.js:240)
in AppContainer (at ReactMeteorData.js:88)
in ReactMeteorDataComponent (at withExpoRoot.js:22)
in RootErrorBoundary (at withExpoRoot.js:21)
in ExpoRootComponent (at renderApplication.js:34)
in RCTView (at View.js:44)
in RCTView (at View.js:44)
in AppContainer (at renderApplication.js:33)
Moment
moment.js:238:47
clone
moment.js:3202:26
toISOString
moment.js:3332:22
send
socket.js:15:38

ddp.js:44:25
process
queue.js:21:32
push
queue.js:16:4
method
ddp.js:108:27
_default
Call.js:9:29
componentDidMount
FameMore.js:37:18
commitLifeCycles
ReactNativeRenderer-dev.js:12144:10
commitAllLifeCycles
ReactNativeRenderer-dev.js:13496:8
invokeGuardedCallbackImpl
ReactNativeRenderer-dev.js:92:15
invokeGuardedCallback
ReactNativeRenderer-dev.js:306:36
commitRoot
ReactNativeRenderer-dev.js:13702:8
completeRoot
ReactNativeRenderer-dev.js:15269:13
performWorkOnRoot
ReactNativeRenderer-dev.js:15197:21
performWork
ReactNativeRenderer-dev.js:15090:24
performSyncWork
ReactNativeRenderer-dev.js:15047:14
batchedUpdates$1
ReactNativeRenderer-dev.js:15315:21
batchedUpdates
ReactNativeRenderer-dev.js:2616:31
_receiveRootNodeIDEvent
ReactNativeRenderer-dev.js:2711:17
receiveTouches
ReactNativeRenderer-dev.js:2787:28
__callFunction
MessageQueue.js:349:47

MessageQueue.js:106:26
__guard
MessageQueue.js:297:10
callFunctionReturnFlushedQueue
MessageQueue.js:105:17
`

I have now the code that generates this error
moment(latestExpiresDate).isAfter(moment())

with latestExpiresDate as undefined ...

I've managed to reproduce this as well on my end... and I also found the source of the bug... and I also patched it and submitted a pull request https://github.com/moment/moment/pull/5145.
This bug occurs during console.log, util.inspect, or a JSON.stringify call on a moment object with an invalid date (e.g. as @lc3t35 mentioned above https://github.com/moment/moment/issues/5011#issuecomment-469015425 and provided an example).
The error is due to the isValid method returning false positives when m._d == 'Invalid Date' or simply when m._d is not a valid Date instance.
Here was the fix if you want to hardcode a test on your end:

vim ./node_modules/moment/moment.js

    function isValid(m) {
+      if (!isDate(m._d)) {
+        m._isValid = false;
+        return m._isValid;
+      }
        if (m._isValid == null) {
            var flags = getParsingFlags(m);
            var parsedParts = some.call(flags.parsedDateParts, function (i) {
                return i != null;
            });

I've published @ladjs/moment in the meanwhile until https://github.com/moment/moment/pull/5145 is merged and a version bump is released to npm.
Thanks!

I'm having a hard time reproducing this problem. Does it reproduce on all environments?
> moment(undefined).isAfter(moment())
false
> moment(null).isAfter(moment())
false

@lc3t35 could you provide another example? We could not reproduce the one listed.
Does 2.26.0 solve your issue?

Hello. the new version did not solved the issue.

@opravil-jan - can you post another example? We cannot reproduce it.

I will close this. Please re-open if you have another test case.

@marwahaha - i'm not sure who this falls on, but i encountered a similar problem.  i suspect it is due to firebase logger's removeCircular method which somehow puts moment.js into a bad state.
https://github.com/firebase/firebase-functions/blob/master/src/logger.ts#L33

i think i have a fix for this, pr coming

although this would stop the crash i am seeing, i don't think it is the correct result, since there should be a valid date in there.  it is probably better to have moment blow up.
https://github.com/moment/moment/commit/1ca8689dc4623ba4498835039f51f7e077b51f61

Is there a solution for this yet? I only get it when I try to use  var pdf = require('pdf-parse')


[RTL] Update num_req_outstanding check

as discussed at #3911, num_req_outstanding can be equal to MaxOutstanding
Signed-off-by: Weicai Yang<EMAIL_ADDRESS>

just solved the conflict, no other change.


Added a HTML5 video player

I just added a simple HTML5 video player.

I added my widget to the widget overview and did a few other things as you can see.

Very good work! Thank you for your contribution. ;-)


GEODE-4952: Remove unused imports from test files.

Thank you for submitting a contribution to Apache Geode.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

[x] Has your PR been rebased against the latest commit within the target branch (typically develop)?

[x] Is your initial contribution a single, squashed commit?

[x] Does gradlew build run cleanly?

[n/a] Have you written or updated unit tests to verify your changes?

[n/a] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and
submit an update to your PR as soon as possible. If you need help, please send an
email to<EMAIL_ADDRESS>

Precheckin green, but I'm going to wait on this churn until we cut a 1.6 branch.


Bootstrap 4.0.0 stable

I'm opening this issue to discuss how to proceed and track what do to.
TODO:

[x] Adapt frontend to remove the Beta warning
[ ] Merge config for bootstrap 3 and 4
[ ] Adapt frontend to only show the latest version
[ ] Adapt tests for the above stuff
[ ] Update the symlink for latest version

@XhmikosR re-open if this isn't finished.


:sparkles: Image logging

FYI you'll want to add a section here with PIL as a dependency:
https://github.com/whylabs/whylogs-python/blob/mainline/setup.cfg#L87

FYI you'll want to add a section here with PIL as a dependency:
https://github.com/whylabs/whylogs-python/blob/mainline/setup.cfg#L87

I noticed notebooks/Test.ipynb is committed? Do we have a better name for the notebook?

I noticed notebooks/Test.ipynb is committed? Do we have a better name for the notebook?


[apiv2] can't build successfully an image from the given Dockerfile

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)
/kind bug
Description

Steps to reproduce the issue:

./bin/podman system service --timeout 0 tcp:<IP_ADDRESS>:8080

to create a Dockerfile

curl -v -X POST "http://localhost:8080/v1.40/build?dockerfile=./Dockerfile"|jq

Describe the results you received:
1. can successfully build an image from Dockerfile by podman cmdline
[fedora@fedora32 containers]$ ./libpod/bin/podman build -f Dockerfile
STEP 1: FROM busybox
Getting image source signatures
Copying blob d9cbbca60e5f done
Copying config 78096d0a54 done
Writing manifest to image destination
Storing signatures
STEP 2: COPY ./hello /hello
--> 09ed0df702c
STEP 3: RUN cat /hello
hello
STEP 4: COMMIT
--> a074a7c68b1
a074a7c68b17ce7cc941182b7a3833603a850f14243c34442c2ab693b0fdd68f

[fedora@fedora32 containers]$ ./libpod/bin/podman images|grep busybox
docker.io/library/busybox                 latest  78096d0a5478  5 days ago   1.44 MB

[fedora@fedora32 containers]$ ./libpod/bin/podman rmi busybox
Untagged: docker.io/library/busybox:latest

2. to build image w/ the same Dockerfile by curl
[fedora@fedora32 containers]$ curl -v -X POST "http://localhost:8080/v1.40/build?dockerfile=./Dockerfile"|jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying ::1:8080...
* connect to ::1 port 8080 failed: Connection refused
*   Trying <IP_ADDRESS>:8080...
* Connected to localhost (<IP_ADDRESS>) port 8080 (#0)
> POST /v1.40/build?dockerfile=./Dockerfile HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.69.1
> Accept: */*
>
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Mark bundle as not supporting multiuse
< HTTP/1.1 500 Internal Server Error
< Content-Type: application/json
< Date: Tue, 19 May 2020 11:27:18 GMT
< Content-Length: 506
<
{ [506 bytes data]
100   506  100   506    0     0   2032      0 --:--:-- --:--:-- --:--:--  2024
* Connection #0 to host localhost left intact
{
  "cause": "no such file or directory",
  "message": "error building at STEP \"ADD /$SCRIPT_BASE $GOSRC/$SCRIPT_BASE\": no files found matching \"/tmp/libpod_builder367071067/build/contrib/cirrus\": no such file or directory",
  "response": 500
}
{
  "stream": "\nFROM registry.fedoraproject.org/fedora:latest\nENV GOPATH=/var/tmp/go\nENV GOSRC=$GOPATH/src/github.com/containers/libpod\nENV SCRIPT_BASE=./contrib/cirrus\nENV PACKER_BASE=$SCRIPT_BASE/packer\nADD /$SCRIPT_BASE $GOSRC/$SCRIPT_BASE\nSuccessfully built \n"
}

NOTE: it still raised an error like 'no such file or directory' even though a absolute path is given for the Dockerfile, I also tried to use http encode to pass parameter to curl w/o double quota around URL, I got the same result. It is more puzzling that the registry.fedoraproject.org/fedora:latest is pulled on the local.

[fedora@fedora32 containers]$ ./libpod/bin/podman images|grep busybox
[fedora@fedora32 containers]$ ./libpod/bin/podman images|grep fedora
registry.fedoraproject.org/fedora         latest  eb7134a03cd6  4 days ago   208 MB

Describe the results you expected:
Can build successfully an image from the given Dockerfile

Additional information you deem important (e.g. issue happens only occasionally):
[fedora@fedora32 libpod]$ ./bin/podman system service --timeout 0 tcp:<IP_ADDRESS>:8080 &
[1] 109052
[fedora@fedora32 libpod]$ This function is EXPERIMENTAL.
http: superfluous response.WriteHeader call from github.com/containers/libpod/pkg/api/handlers/utils.WriteJSON (handler.go:64)

Output of podman version:
Version:            2.0.0-dev
RemoteAPI Version:  1
Go Version:         go1.14.2
Git Commit:         0f8ad039235137d086b9fea33bb74b362e625cdd
Built:              Tue May 19 11:02:16 2020
OS/Arch:            linux/amd64

Output of podman info --debug:
host:
  arch: amd64
  buildahVersion: 1.15.0-dev
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.15-1.fc32.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.15, commit: 33da5ef83bf2abc7965fc37980a49d02fdb71826'
  cpus: 4
  distribution:
    distribution: fedora
    version: "32"
  eventLogger: file
  hostname: fedora32
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.6.10-300.fc32.x86_64
  linkmode: dynamic
  memFree: 823971840
  memTotal:<PHONE_NUMBER>
  ociRuntime:
    name: crun
    package: crun-0.13-2.fc32.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 0.13
      commit: e79e4de4ac16da0ce48777afb72c6241de870525
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  rootless: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.0.0-1.fc32.x86_64
    version: |-
      slirp4netns version 1.0.0
      commit: a3be729152a33e692cd28b52f664defbf2e7810a
      libslirp: 4.2.0
  swapFree: 0
  swapTotal: 0
  uptime: 195h 29m 51.01s (Approximately 8.12 days)
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - registry.centos.org
  - docker.io
store:
  configFile: /home/fedora/.config/containers/storage.conf
  containerStore:
    number: 6
    paused: 0
    running: 3
    stopped: 3
  graphDriverName: overlay
  graphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: fuse-overlayfs-1.0.0-1.fc32.x86_64
      Version: |-
        fusermount3 version: 3.9.1
        fuse-overlayfs: version 1.0.0
        FUSE library version 3.9.1
        using FUSE kernel interface version 7.31
  graphRoot: /home/fedora/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 9
  runRoot: /run/user/1000/containers
  volumePath: /home/fedora/.local/share/containers/storage/volumes
version:
  Built:<PHONE_NUMBER>
  GitCommit: 0f8ad039235137d086b9fea33bb74b362e625cdd
  GoVersion: go1.14.2
  OsArch: linux/amd64
  RemoteAPIVersion: 1
  Version: 2.0.0-dev

Package info (e.g. output of rpm -q podman or apt list podman):
[fedora@fedora32 libpod]$ git rev-parse HEAD
0f8ad039235137d086b9fea33bb74b362e625cdd

Additional environment details (AWS, VirtualBox, physical, etc.):
Opentack VM

@vrothberg PTAL

The usage of curl is not correct (see https://docs.docker.com/engine/api/v1.40/#operation/ImageBuild). The build endpoint expects a tarfile with the Dockerfile(s) and the entire build context:
 --request POST \
 --header "Content-Type: application/x-tar" \
 --data-binary<EMAIL_ADDRESS>\

I have to leave the keyboard in a bit, so I cannot reproduce but the usage of this issue is wrong and Podman validly errors out.

The usage of curl is not correct (see https://docs.docker.com/engine/api/v1.40/#operation/ImageBuild). The build endpoint expects a tarfile with the Dockerfile(s) and the entire build context:
 --request POST \
 --header "Content-Type: application/x-tar" \
 --data-binary<EMAIL_ADDRESS>\

Thanks, but I haven't found related documentation about 'header Parameters' in http://docs.podman.io/en/latest/_static/api.html#operation/buildImage

It would be great to get these curl commands into a test suite that we could run on each PR.

The usage of curl is not correct (see https://docs.docker.com/engine/api/v1.40/#operation/ImageBuild). The build endpoint expects a tarfile with the Dockerfile(s) and the entire build context:
 --request POST \
 --header "Content-Type: application/x-tar" \
 --data-binary<EMAIL_ADDRESS>\

Thanks, but I haven't found related documentation about 'header Parameters' in http://docs.podman.io/en/latest/_static/api.html#operation/buildImage

That's a fair point. The docs should mention that.

@vrothberg , I still can't get expected result w/ above parameters, maybe, I'm wrong, could you give a cmdline example?  thanks!
$ file /tmp/busybox-latest.tar
/tmp/busybox-latest.tar: POSIX tar archive

$ curl -v -X POST -H "Content-Type: application/x-tar" --data-binary<EMAIL_ADDRESS>http://localhost:8080/v1.40/libpod/build|jq
Note: Unnecessary use of -X or --request, POST is already inferred.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying ::1:8080...
* connect to ::1 port 8080 failed: Connection refused
*   Trying <IP_ADDRESS>:8080...
* Connected to localhost (<IP_ADDRESS>) port 8080 (#0)
> POST /v1.40/libpod/build HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.69.1
> Accept: */*
> Content-Type: application/x-tar
> Content-Length: 1446400
> Expect: 100-continue
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 404 Not Found
< Content-Type: text/plain; charset=utf-8
< X-Content-Type-Options: nosniff
< Date: Thu, 21 May 2020 09:10:24 GMT
< Content-Length: 10
< Connection: close
<
{ [10 bytes data]
  0 1412k  100    10    0     0   5000      0 --:--:-- --:--:-- --:--:--  5000
* Closing connection 0
parse error: Invalid numeric literal at line 1, column 4

@chuanchang You are getting back a 404, v1.40/libpod/build doesn't exist. You most likely ment v1.40/build instead since that are the docker api endpoints.

i also just made a change upstream so libpod/build now exists.

Both the /libpod/build and /build didn't work for me, if my cmdline is wrong please correct me, thanks.
[fedora@fedora32 libpod]$ git rev-parse HEAD
1077d2d0b72d027a01080c0d3ebdb7d99a969661

[fedora@fedora32 containers]$ cat Dockerfile
FROM busybox
RUN echo hello

[fedora@fedora32 containers]$ tar cvf Dockerfile.tar Dockerfile
Dockerfile

[fedora@fedora32 containers]$ file Dockerfile.tar
Dockerfile.tar: POSIX tar archive (GNU)

[fedora@fedora32 containers]$ curl -v -X POST -H "Content-Type: application/x-tar" --data-binary<EMAIL_ADDRESS>http://localhost:8080/v1.40/libpod/build|jq
Note: Unnecessary use of -X or --request, POST is already inferred.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying ::1:8080...
* connect to ::1 port 8080 failed: Connection refused
*   Trying <IP_ADDRESS>:8080...
* Connected to localhost (<IP_ADDRESS>) port 8080 (#0)
> POST /v1.40/libpod/build HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.69.1
> Accept: */*
> Content-Type: application/x-tar
> Content-Length: 10240
>
} [10240 bytes data]
* upload completely sent off: 10240 out of 10240 bytes
* Mark bundle as not supporting multiuse
< HTTP/1.1 500 Internal Server Error
< Content-Type: application/json
< Date: Tue, 26 May 2020 03:48:58 GMT
< Content-Length: 236
<
{ [236 bytes data]
100 10476  100<PHONE_NUMBER>0    979  42489 --:--:-- --:--:-- --:--:-- 43650
* Connection #0 to host localhost left intact
{
  "cause": "no such file or directory",
  "message": "error building at STEP \"ADD /$SCRIPT_BASE $GOSRC/$SCRIPT_BASE\": no files found matching \"/tmp/libpod_builder589466626/build/contrib/cirrus\": no such file or directory",
  "response": 500
}

[fedora@fedora32 containers]$ curl -v -X POST -H "Content-Type: application/x-tar" --data-binary<EMAIL_ADDRESS>http://localhost:8080/v1.40/build|jq
Note: Unnecessary use of -X or --request, POST is already inferred.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying ::1:8080...
* connect to ::1 port 8080 failed: Connection refused
*   Trying <IP_ADDRESS>:8080...
* Connected to localhost (<IP_ADDRESS>) port 8080 (#0)
> POST /v1.40/build HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.69.1
> Accept: */*
> Content-Type: application/x-tar
> Content-Length: 10240
>
} [10240 bytes data]
* upload completely sent off: 10240 out of 10240 bytes
* Mark bundle as not supporting multiuse
< HTTP/1.1 500 Internal Server Error
< Content-Type: application/json
< Date: Tue, 26 May 2020 03:49:16 GMT
< Content-Length: 236
<
{ [236 bytes data]
100 10476  100<PHONE_NUMBER>0   1035  44912 --:--:-- --:--:-- --:--:-- 45947
* Connection #0 to host localhost left intact
{
  "cause": "no such file or directory",
  "message": "error building at STEP \"ADD /$SCRIPT_BASE $GOSRC/$SCRIPT_BASE\": no files found matching \"/tmp/libpod_builder041902975/build/contrib/cirrus\": no such file or directory",
  "response": 500
}

Works for me. Closing. Feel free to reopen if it doesn't work for you.

Works for me. Closing. Feel free to reopen if it doesn't work for you.

It doesn't work well for me, the curl command can't still normally exit after 8 mins.
$ curl -v -X POST -H "Content-Type: application/x-tar" --data-binary<EMAIL_ADDRESS>http://localhost:8080/v1.40/libpod/build|jq
Note: Unnecessary use of -X or --request, POST is already inferred.
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Dload  Upload   Total   Spent    Left  Speed
0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying ::1:8080...

connect to ::1 port 8080 failed: Connection refused
Trying <IP_ADDRESS>:8080...
Connected to localhost (<IP_ADDRESS>) port 8080 (#0)

POST /v1.40/libpod/build HTTP/1.1
Host: localhost:8080
User-Agent: curl/7.69.1
Accept: /
Content-Type: application/x-tar
Content-Length: 10240

} [10240 bytes data]

upload completely sent off: 10240 out of 10240 bytes
100 10240    0     0  100 10240      0     21  0:08:07  0:08:02  0:00:05     0   <------------  Can't normally exit

@chuanchang can you paste the full curl command here?

@chuanchang can you paste the full curl command here?

@baude I still used previous curl command, you can got it from https://github.com/containers/podman/issues/6274#issuecomment-654670255 and https://github.com/containers/podman/issues/6274#issuecomment-633797218, thanks!


runtime error: index out of range

panic: runtime error: index out of range

goroutine 20 [running]:
panic(0x2d5820, 0xc4200121c0)
	/home/travis/.gimme/versions/go1.7.linux.amd64/src/runtime/panic.go:500 +0x1a1
github.com/quii/mockingjay-server/vendor/github.com/quii/jsonequaliser.getJSONNodeFromString(0xc420013ee0, 0x2, 0xc4200ce900, 0x2, 0x8)
	/home/travis/gopath/src/github.com/quii/mockingjay-server/vendor/github.com/quii/jsonequaliser/json.go:14 +0x216
github.com/quii/mockingjay-server/vendor/github.com/quii/jsonequaliser.IsCompatible(0xc420013ee0, 0x2, 0xc4200ce7e8, 0x2, 0x0, 0xc42002c400, 0xc42000f7c0)
	/home/travis/gopath/src/github.com/quii/mockingjay-server/vendor/github.com/quii/jsonequaliser/compatability.go:13 +0x39
github.com/quii/mockingjay-server/mockingjay.checkBody(0xc4200ce7e8, 0x2, 0xc420013ee0, 0x2, 0xc4200ce7e8, 0x2, 0x0)
	/home/travis/gopath/src/github.com/quii/mockingjay-server/mockingjay/compatibility.go:147 +0xca
github.com/quii/mockingjay-server/mockingjay.(*CompatibilityChecker).check(0xc4200c2aa0, 0xc420486f18, 0x7fff5fbff712, 0x22, 0x0, 0x0, 0x0)
	/home/travis/gopath/src/github.com/quii/mockingjay-server/mockingjay/compatibility.go:114 +0x4b2
github.com/quii/mockingjay-server/mockingjay.(*CompatibilityChecker).compatabilityWorker(0xc4200c2aa0, 0xc420774720, 0x1e, 0x0, 0xc42000b2f0, 0x28, 0x0, 0xc420013ea8, 0x3, 0x0, ...)
	/home/travis/gopath/src/github.com/quii/mockingjay-server/mockingjay/compatibility.go:70 +0x72
created by github.com/quii/mockingjay-server/mockingjay.(*CompatibilityChecker).CheckCompatibility
	/home/travis/gopath/src/github.com/quii/mockingjay-server/mockingjay/compatibility.go:56 +0x2c0

Could you supply some more context?
So you have the config?
How do you produce that error?

2.0M 10 May 09:14 example.yaml

./mockingjay-server -config=example.yaml -realURL=http://***.***.***:****

Not sure what the reason could be.
I have a config file of about 2 Mb and ran mockingjay to validate the endpoints.
Could be something to do with control characters in the yaml as well.
[yaml: control characters are not allowed]
Guess it might need a way to handle huge error gracefully?
The json response has a control character that the parser could not handle?

Thanks for raising this, I will try and find time to look into it.
It would be great if you could give me an example of the YAML coming back from the real server so I can try and reproduce it locally :)

Hi @Daniel-ltw sorry for the late reply
I have just pushed a change to improve JSON handling which @gypsydave5 kindly wrote
Could you try again with the newest release? https://github.com/quii/mockingjay-server/releases/tag/1.10.4

Looks better.
The error message do make better sense now.
Cheers.


Empty extracted obj

I am trying to train with my custom data but the extracted object file is empty (it opens but nothing is there). I was using 23 images when I got these results.
First I run the img2poses file:
python scripts/imgs2poses.py ./load/bmvs_dog # images are in ./load/bmvs_dog/images output:
`Need to run COLMAP
I0425 18:00:23.786562 29914 misc.cc:198]
==============================================================================
Feature extraction
==============================================================================
I0425 18:00:24.199863 29918 feature_extraction.cc:255] Processed file [1/23]
I0425 18:00:24.199926 29918 feature_extraction.cc:258]   Name:            000000.png
I0425 18:00:24.199941 29918 feature_extraction.cc:284]   Dimensions:      1600 x 1200
I0425 18:00:24.199954 29918 feature_extraction.cc:287]   Camera:          #1 - SIMPLE_RADIAL
I0425 18:00:24.199970 29918 feature_extraction.cc:290]   Focal Length:    1920.00px
I0425 18:00:24.199992 29918 feature_extraction.cc:303]   Features:        9141
I0425 18:00:24.276440 29918 feature_extraction.cc:255] Processed file [2/23]
I0425 18:00:24.276490 29918 feature_extraction.cc:258]   Name:            000001.png
I0425 18:00:24.276506 29918 feature_extraction.cc:284]   Dimensions:      1600 x 1200
I0425 18:00:24.276520 29918 feature_extraction.cc:287]   Camera:          #1 - SIMPLE_RADIAL
I0425 18:00:24.276533 29918 feature_extraction.cc:290]   Focal Length:    1920.00px
I0425 18:00:24.276556 29918 feature_extraction.cc:303]   Features:        9965
I0425 18:00:24.349610 29918 feature_extraction.cc:255] Processed file [3/23]
I0425 18:00:24.350747 29918 feature_extraction.cc:258]   Name:            000002.png
I0425 18:00:24.350764 29918 feature_extraction.cc:284]   Dimensions:      1600 x 1200
I0425 18:00:24.350772 29918 feature_extraction.cc:287]   Camera:          #1 - SIMPLE_RADIAL
I0425 18:00:24.350781 29918 feature_extraction.cc:290]   Focal Length:    1920.00px
I0425 18:00:24.350801 29918 feature_extraction.cc:303]   Features:        6628
I0425 18:00:24.453512 29918 feature_extraction.cc:255] Processed file [4/23]
I0425 18:00:24.460233 29918 feature_extraction.cc:258]   Name:            000003.png
I0425 18:00:24.460263 29918 feature_extraction.cc:284]   Dimensions:      1600 x 1200
I0425 18:00:24.460271 29918 feature_extraction.cc:287]   Camera:          #1 - SIMPLE_RADIAL
I0425 18:00:24.460280 29918 feature_extraction.cc:290]   Focal Length:    1920.00px
I0425 18:00:24.460299 29918 feature_extraction.cc:303]   Features:        5055
I0425 18:00:24.623417 29918 feature_extraction.cc:255] Processed file [5/23]
I0425 18:00:24.623450 29918 feature_extraction.cc:258]   Name:            000004.png
I0425 18:00:24.623459 29918 feature_extraction.cc:284]   Dimensions:      1600 x 1200
I0425 18:00:24.623463 29918 feature_extraction.cc:287]   Camera:          #1 - SIMPLE_RADIAL
I0425 18:00:24.623468 29918 feature_extraction.cc:290]   Focal Length:    1920.00px
I0425 18:00:24.623483 29918 feature_extraction.cc:303]   Features:        5156
I0425 18:00:24.905103 29918 feature_extraction.cc:255] Processed file [6/23]
I0425 18:00:24.905160 29918 feature_extraction.cc:258]   Name:            000005.png
I0425 18:00:24.905175 29918 feature_extraction.cc:284]   Dimensions:      1600 x 1200
I0425 18:00:24.905187 29918 feature_extraction.cc:287]   Camera:          #1 - SIMPLE_RADIAL
I0425 18:00:24.905200 29918 feature_extraction.cc:290]   Focal Length:    1920.00px
I0425 18:00:24.905227 29918 feature_extraction.cc:303]   Features:        2275
I0425 18:00:24.906174 29918 feature_extraction.cc:255] Processed file [7/23]
I0425 18:00:24.906201 29918 feature_extraction.cc:258]   Name:            000006.png
I0425 18:00:24.906210 29918 feature_extraction.cc:284]   Dimensions:      1600 x 1200
I0425 18:00:24.906239 29918 feature_extraction.cc:287]   Camera:          #1 - SIMPLE_RADIAL
I0425 18:00:24.906248 29918 feature_extraction.cc:290]   Focal Length:    1920.00px
I0425 18:00:24.906261 29918 feature_extraction.cc:303]   Features:        3726
I0425 18:00:24.962646 29918 feature_extraction.cc:255] Processed file [8/23]
I0425 18:00:24.962675 29918 feature_extraction.cc:258]   Name:            000007.png
I0425 18:00:24.962684 29918 feature_extraction.cc:284]   Dimensions:      1600 x 1200
I0425 18:00:24.962692 29918 feature_extraction.cc:287]   Camera:          #1 - SIMPLE_RADIAL
I0425 18:00:24.962700 29918 feature_extraction.cc:290]   Focal Length:    1920.00px
I0425 18:00:24.962713 29918 feature_extraction.cc:303]   Features:        5256
I0425 18:00:25.024228 29918 feature_extraction.cc:255] Processed file [9/23]
I0425 18:00:25.024272 29918 feature_extraction.cc:258]   Name:            000008.png
I0425 18:00:25.024281 29918 feature_extraction.cc:284]   Dimensions:      1600 x 1200
I0425 18:00:25.024288 29918 feature_extraction.cc:287]   Camera:          #1 - SIMPLE_RADIAL
I0425 18:00:25.024297 29918 feature_extraction.cc:290]   Focal Length:    1920.00px
I0425 18:00:25.024310 29918 feature_extraction.cc:303]   Features:        8690
I0425 18:00:25.071779 29918 feature_extraction.cc:255] Processed file [10/23]
I0425 18:00:25.071820 29918 feature_extraction.cc:258]   Name:            000009.png
I0425 18:00:25.071830 29918 feature_extraction.cc:284]   Dimensions:      1600 x 1200
I0425 18:00:25.071837 29918 feature_extraction.cc:287]   Camera:          #1 - SIMPLE_RADIAL
I0425 18:00:25.071846 29918 feature_extraction.cc:290]   Focal Length:    1920.00px
I0425 18:00:25.071859 29918 feature_extraction.cc:303]   Features:        9896
I0425 18:00:25.137094 29918 feature_extraction.cc:255] Processed file [11/23]
I0425 18:00:25.137135 29918 feature_extraction.cc:258]   Name:            000010.png
I0425 18:00:25.137148 29918 feature_extraction.cc:284]   Dimensions:      1600 x 1200
I0425 18:00:25.137158 29918 feature_extraction.cc:287]   Camera:          #1 - SIMPLE_RADIAL
I0425 18:00:25.143236 29918 feature_extraction.cc:290]   Focal Length:    1920.00px
I0425 18:00:25.143272 29918 feature_extraction.cc:303]   Features:        10684
I0425 18:00:25.262234 29918 feature_extraction.cc:255] Processed file [12/23]
I0425 18:00:25.262269 29918 feature_extraction.cc:258]   Name:            000011.png
I0425 18:00:25.262279 29918 feature_extraction.cc:284]   Dimensions:      1600 x 1200
I0425 18:00:25.262285 29918 feature_extraction.cc:287]   Camera:          #1 - SIMPLE_RADIAL
I0425 18:00:25.262295 29918 feature_extraction.cc:290]   Focal Length:    1920.00px
I0425 18:00:25.262307 29918 feature_extraction.cc:303]   Features:        9474
I0425 18:00:25.267258 29918 feature_extraction.cc:255] Processed file [13/23]
I0425 18:00:25.267289 29918 feature_extraction.cc:258]   Name:            000012.png
I0425 18:00:25.267298 29918 feature_extraction.cc:284]   Dimensions:      1600 x 1200
I0425 18:00:25.267305 29918 feature_extraction.cc:287]   Camera:          #1 - SIMPLE_RADIAL
I0425 18:00:25.267320 29918 feature_extraction.cc:290]   Focal Length:    1920.00px
I0425 18:00:25.267333 29918 feature_extraction.cc:303]   Features:        9983
I0425 18:00:25.373262 29918 feature_extraction.cc:255] Processed file [14/23]
I0425 18:00:25.373304 29918 feature_extraction.cc:258]   Name:            000013.png
I0425 18:00:25.373312 29918 feature_extraction.cc:284]   Dimensions:      1600 x 1200
I0425 18:00:25.373317 29918 feature_extraction.cc:287]   Camera:          #1 - SIMPLE_RADIAL
I0425 18:00:25.373327 29918 feature_extraction.cc:290]   Focal Length:    1920.00px
I0425 18:00:25.373342 29918 feature_extraction.cc:303]   Features:        10147
I0425 18:00:25.751847 29918 feature_extraction.cc:255] Processed file [15/23]
I0425 18:00:25.751889 29918 feature_extraction.cc:258]   Name:            000014.png
I0425 18:00:25.751904 29918 feature_extraction.cc:284]   Dimensions:      1600 x 1200
I0425 18:00:25.751917 29918 feature_extraction.cc:287]   Camera:          #1 - SIMPLE_RADIAL
I0425 18:00:25.751930 29918 feature_extraction.cc:290]   Focal Length:    1920.00px
I0425 18:00:25.751952 29918 feature_extraction.cc:303]   Features:        10626
I0425 18:00:25.754889 29918 feature_extraction.cc:255] Processed file [16/23]
I0425 18:00:25.754915 29918 feature_extraction.cc:258]   Name:            000015.png
I0425 18:00:25.754923 29918 feature_extraction.cc:284]   Dimensions:      1600 x 1200
I0425 18:00:25.754930 29918 feature_extraction.cc:287]   Camera:          #1 - SIMPLE_RADIAL
I0425 18:00:25.754938 29918 feature_extraction.cc:290]   Focal Length:    1920.00px
I0425 18:00:25.754948 29918 feature_extraction.cc:303]   Features:        8881
I0425 18:00:25.800624 29918 feature_extraction.cc:255] Processed file [17/23]
I0425 18:00:25.800652 29918 feature_extraction.cc:258]   Name:            000016.png
I0425 18:00:25.800662 29918 feature_extraction.cc:284]   Dimensions:      1600 x 1200
I0425 18:00:25.800668 29918 feature_extraction.cc:287]   Camera:          #1 - SIMPLE_RADIAL
I0425 18:00:25.800678 29918 feature_extraction.cc:290]   Focal Length:    1920.00px
I0425 18:00:25.800688 29918 feature_extraction.cc:303]   Features:        6571
I0425 18:00:25.886762 29918 feature_extraction.cc:255] Processed file [18/23]
I0425 18:00:25.886811 29918 feature_extraction.cc:258]   Name:            000017.png
I0425 18:00:25.886821 29918 feature_extraction.cc:284]   Dimensions:      1600 x 1200
I0425 18:00:25.886827 29918 feature_extraction.cc:287]   Camera:          #1 - SIMPLE_RADIAL
I0425 18:00:25.886834 29918 feature_extraction.cc:290]   Focal Length:    1920.00px
I0425 18:00:25.886845 29918 feature_extraction.cc:303]   Features:        4236
I0425 18:00:25.888120 29918 feature_extraction.cc:255] Processed file [19/23]
I0425 18:00:25.888144 29918 feature_extraction.cc:258]   Name:            000018.png
I0425 18:00:25.888152 29918 feature_extraction.cc:284]   Dimensions:      1600 x 1200
I0425 18:00:25.888159 29918 feature_extraction.cc:287]   Camera:          #1 - SIMPLE_RADIAL
I0425 18:00:25.888166 29918 feature_extraction.cc:290]   Focal Length:    1920.00px
I0425 18:00:25.888176 29918 feature_extraction.cc:303]   Features:        2944
I0425 18:00:26.025004 29918 feature_extraction.cc:255] Processed file [20/23]
I0425 18:00:26.029333 29918 feature_extraction.cc:258]   Name:            000019.png
I0425 18:00:26.029357 29918 feature_extraction.cc:284]   Dimensions:      1600 x 1200
I0425 18:00:26.029366 29918 feature_extraction.cc:287]   Camera:          #1 - SIMPLE_RADIAL
I0425 18:00:26.029376 29918 feature_extraction.cc:290]   Focal Length:    1920.00px
I0425 18:00:26.029390 29918 feature_extraction.cc:303]   Features:        2055
I0425 18:00:26.030042 29918 feature_extraction.cc:255] Processed file [21/23]
I0425 18:00:26.030062 29918 feature_extraction.cc:258]   Name:            000020.png
I0425 18:00:26.030069 29918 feature_extraction.cc:284]   Dimensions:      1600 x 1200
I0425 18:00:26.030077 29918 feature_extraction.cc:287]   Camera:          #1 - SIMPLE_RADIAL
I0425 18:00:26.030086 29918 feature_extraction.cc:290]   Focal Length:    1920.00px
I0425 18:00:26.030097 29918 feature_extraction.cc:303]   Features:        2754
I0425 18:00:26.071935 29918 feature_extraction.cc:255] Processed file [22/23]
I0425 18:00:26.071964 29918 feature_extraction.cc:258]   Name:            000021.png
I0425 18:00:26.071970 29918 feature_extraction.cc:284]   Dimensions:      1600 x 1200
I0425 18:00:26.071976 29918 feature_extraction.cc:287]   Camera:          #1 - SIMPLE_RADIAL
I0425 18:00:26.071982 29918 feature_extraction.cc:290]   Focal Length:    1920.00px
I0425 18:00:26.071993 29918 feature_extraction.cc:303]   Features:        4343
I0425 18:00:26.120632 29918 feature_extraction.cc:255] Processed file [23/23]
I0425 18:00:26.120666 29918 feature_extraction.cc:258]   Name:            000022.png
I0425 18:00:26.120671 29918 feature_extraction.cc:284]   Dimensions:      1600 x 1200
I0425 18:00:26.120676 29918 feature_extraction.cc:287]   Camera:          #1 - SIMPLE_RADIAL
I0425 18:00:26.120682 29918 feature_extraction.cc:290]   Focal Length:    1920.00px
I0425 18:00:26.120692 29918 feature_extraction.cc:303]   Features:        7045
I0425 18:00:26.169898 29914 timer.cc:91] Elapsed time: 0.040 [minutes]
Features extracted
I0425 18:00:26.324311 29931 misc.cc:198]
==============================================================================
Exhaustive feature matching
==============================================================================
I0425 18:00:26.533618 29931 feature_matching.cc:234] Matching block [1/1, 1/1]
I0425 18:00:33.660600 29931 feature_matching.cc:47]  in 7.127s
I0425 18:00:33.662482 29931 timer.cc:91] Elapsed time: 0.122 [minutes]
Features matched
I0425 18:00:33.747568 29966 incremental_mapper.cc:225] Loading database
I0425 18:00:33.749192 29966 database_cache.cc:65] Loading cameras...
I0425 18:00:33.749248 29966 database_cache.cc:75]  1 in 0.000s
I0425 18:00:33.749264 29966 database_cache.cc:83] Loading matches...
I0425 18:00:33.750439 29966 database_cache.cc:89]  188 in 0.001s
I0425 18:00:33.750459 29966 database_cache.cc:105] Loading images...
I0425 18:00:33.763101 29966 database_cache.cc:155]  23 in 0.013s (connected 23)
I0425 18:00:33.763129 29966 database_cache.cc:166] Building correspondence graph...
I0425 18:00:33.775928 29966 database_cache.cc:203]  in 0.013s (ignored 0)
I0425 18:00:33.775978 29966 timer.cc:91] Elapsed time: 0.000 [minutes]
I0425 18:00:33.778131 29966 incremental_mapper.cc:265] Finding good initial image pair
I0425 18:00:33.795753 29966 incremental_mapper.cc:289] Initializing with image pair #10 and #15
I0425 18:00:33.798981 29966 incremental_mapper.cc:294] Global bundle adjustment
I0425 18:00:34.108727 29966 incremental_mapper.cc:376] Registering image #14 (3)
I0425 18:00:34.108762 29966 incremental_mapper.cc:379] => Image sees 308 / 2127 points
I0425 18:00:34.562849 29966 incremental_mapper.cc:41] Retriangulation and Global bundle adjustment
I0425 18:00:34.635131 29966 incremental_mapper.cc:376] Registering image #11 (4)
I0425 18:00:34.635155 29966 incremental_mapper.cc:379] => Image sees 554 / 2242 points
I0425 18:00:34.799409 29966 incremental_mapper.cc:41] Retriangulation and Global bundle adjustment
I0425 18:00:34.903867 29966 incremental_mapper.cc:376] Registering image #13 (5)
I0425 18:00:34.903893 29966 incremental_mapper.cc:379] => Image sees 765 / 2088 points
I0425 18:00:35.195473 29966 incremental_mapper.cc:41] Retriangulation and Global bundle adjustment
I0425 18:00:35.335340 29966 incremental_mapper.cc:376] Registering image #12 (6)
I0425 18:00:35.335366 29966 incremental_mapper.cc:379] => Image sees 698 / 1422 points
I0425 18:00:35.551343 29966 incremental_mapper.cc:41] Retriangulation and Global bundle adjustment
I0425 18:00:35.718150 29966 incremental_mapper.cc:376] Registering image #1 (7)
I0425 18:00:35.718178 29966 incremental_mapper.cc:379] => Image sees 650 / 1408 points
I0425 18:00:36.168394 29966 incremental_mapper.cc:41] Retriangulation and Global bundle adjustment
I0425 18:00:36.362512 29966 incremental_mapper.cc:376] Registering image #2 (8)
I0425 18:00:36.362540 29966 incremental_mapper.cc:379] => Image sees 729 / 1875 points
I0425 18:00:36.765982 29966 incremental_mapper.cc:41] Retriangulation and Global bundle adjustment
I0425 18:00:36.991492 29966 incremental_mapper.cc:376] Registering image #9 (9)
I0425 18:00:36.991520 29966 incremental_mapper.cc:379] => Image sees 602 / 1881 points
I0425 18:00:37.261857 29966 incremental_mapper.cc:41] Retriangulation and Global bundle adjustment
I0425 18:00:37.532910 29966 incremental_mapper.cc:376] Registering image #16 (10)
I0425 18:00:37.532940 29966 incremental_mapper.cc:379] => Image sees 735 / 1876 points
I0425 18:00:37.852524 29966 incremental_mapper.cc:41] Retriangulation and Global bundle adjustment
I0425 18:00:38.143537 29966 incremental_mapper.cc:376] Registering image #3 (11)
I0425 18:00:38.143568 29966 incremental_mapper.cc:379] => Image sees 513 / 1190 points
I0425 18:00:38.391364 29966 incremental_mapper.cc:41] Retriangulation and Global bundle adjustment
I0425 18:00:38.691735 29966 incremental_mapper.cc:376] Registering image #8 (12)
I0425 18:00:38.691766 29966 incremental_mapper.cc:379] => Image sees 480 / 1205 points
I0425 18:00:39.022234 29966 incremental_mapper.cc:376] Registering image #17 (13)
I0425 18:00:39.022269 29966 incremental_mapper.cc:379] => Image sees 558 / 1568 points
I0425 18:00:39.535617 29966 incremental_mapper.cc:41] Retriangulation and Global bundle adjustment
I0425 18:00:40.174641 29966 incremental_mapper.cc:376] Registering image #23 (14)
I0425 18:00:40.175036 29966 incremental_mapper.cc:379] => Image sees 649 / 1383 points
I0425 18:00:40.648156 29966 incremental_mapper.cc:376] Registering image #22 (15)
I0425 18:00:40.648204 29966 incremental_mapper.cc:379] => Image sees 446 / 972 points
I0425 18:00:41.058521 29966 incremental_mapper.cc:41] Retriangulation and Global bundle adjustment
I0425 18:00:41.775250 29966 incremental_mapper.cc:376] Registering image #18 (16)
I0425 18:00:41.775292 29966 incremental_mapper.cc:379] => Image sees 508 / 1165 points
I0425 18:00:42.046187 29966 incremental_mapper.cc:376] Registering image #4 (17)
I0425 18:00:42.046231 29966 incremental_mapper.cc:379] => Image sees 356 / 876 points
I0425 18:00:42.296836 29966 incremental_mapper.cc:41] Retriangulation and Global bundle adjustment
I0425 18:00:42.729629 29966 incremental_mapper.cc:376] Registering image #7 (18)
I0425 18:00:42.729662 29966 incremental_mapper.cc:379] => Image sees 405 / 905 points
I0425 18:00:42.853981 29966 incremental_mapper.cc:376] Registering image #21 (19)
I0425 18:00:42.854018 29966 incremental_mapper.cc:379] => Image sees 462 / 741 points
I0425 18:00:42.990330 29966 incremental_mapper.cc:41] Retriangulation and Global bundle adjustment
I0425 18:00:43.400600 29966 incremental_mapper.cc:376] Registering image #19 (20)
I0425 18:00:43.400630 29966 incremental_mapper.cc:379] => Image sees 361 / 680 points
I0425 18:00:43.543157 29966 incremental_mapper.cc:376] Registering image #5 (21)
I0425 18:00:43.543185 29966 incremental_mapper.cc:379] => Image sees 299 / 558 points
I0425 18:00:43.637622 29966 incremental_mapper.cc:41] Retriangulation and Global bundle adjustment
I0425 18:00:44.211854 29966 incremental_mapper.cc:376] Registering image #20 (22)
I0425 18:00:44.211897 29966 incremental_mapper.cc:379] => Image sees 290 / 434 points
I0425 18:00:44.341912 29966 incremental_mapper.cc:376] Registering image #6 (23)
I0425 18:00:44.341940 29966 incremental_mapper.cc:379] => Image sees 179 / 286 points
I0425 18:00:44.450908 29966 incremental_mapper.cc:41] Retriangulation and Global bundle adjustment
I0425 18:00:44.986716 29966 timer.cc:91] Elapsed time: 0.187 [minutes]
Sparse map created
Finished running COLMAP, see /content/instant-nsr-pl/load/scan/colmap_output.txt for logs`

Then I run !python launch.py --config configs/nerf-colmap.yaml --gpu 0 --train with nerf-colmap.yaml configurations as follows. I also tried up_est_method: ground:

name: nerf-colmap-${basename:${dataset.root_dir}}
tag: ""
seed: 42

dataset:
  name: colmap
  root_dir: ./load/scan
  img_downscale: 4 # specify training image size by either img_wh or img_downscale
  up_est_method: camera # if true, use estimated ground plane normal direction as up direction
  center_est_method: lookat
  n_test_traj_steps: 120
  apply_mask: false
  load_data_on_gpu: true

model:
  name: nerf
  radius: 1.0
  num_samples_per_ray: 2048
  train_num_rays: 128
  max_train_num_rays: 8192
  grid_prune: true
  dynamic_ray_sampling: true
  batch_image_sampling: true
  randomized: true
  ray_chunk: 16384
  learned_background: true
  background_color: random
  geometry:
    name: volume-density
    radius: ${model.radius}
    feature_dim: 16
    density_activation: trunc_exp
    density_bias: -1
    isosurface:
      method: mc
      resolution: 256
      chunk: 2097152
      threshold: 5.0
    xyz_encoding_config:
      otype: HashGrid
      n_levels: 16
      n_features_per_level: 2
      log2_hashmap_size: 19
      base_resolution: 16
      per_level_scale: 1.447269237440378
    mlp_network_config:
      otype: FullyFusedMLP
      activation: ReLU
      output_activation: none
      n_neurons: 64
      n_hidden_layers: 1
  texture:
    name: volume-radiance
    input_feature_dim: ${model.geometry.feature_dim}
    dir_encoding_config:
      otype: SphericalHarmonics
      degree: 4
    mlp_network_config:
      otype: FullyFusedMLP
      activation: ReLU
      output_activation: Sigmoid
      n_neurons: 64
      n_hidden_layers: 2

system:
  name: nerf-system
  loss:
    lambda_rgb: 1.
    lambda_distortion: 0.001
  optimizer:
    name: AdamW
    args:
      lr: 0.01
      betas: [0.9, 0.99]
      eps: 1.e-15
  scheduler:
    name: MultiStepLR
    interval: step
    args:
      milestones: [10000, 15000, 18000]
      gamma: 0.33

checkpoint:
  save_top_k: -1
  every_n_train_steps: ${trainer.max_steps}

export:
  chunk_size: 2097152
  export_vertex_color: False

trainer:
  max_steps: 10000
  log_every_n_steps: 200
  num_sanity_val_steps: 0
  val_check_interval: 5000
  limit_train_batches: 1.0
  limit_val_batches: 2
  enable_progress_bar: true
  precision: 16

could u try printing the value of mesh["v_pos"]? after you do mesh=self.helper(...)


fix MDI nerd icons

since MDI icons are deprecated and removed from nerd fonts, most of the icons show question marks and random Arabic letters

latest commit was 2 weeks ago, but it's not related to the project, but to the website

Fixed with 1792bf1. Thanks for the report. The deprecation of mdi fonts went unnoticed by me because I was using the older version of nerd fonts in my system.


feat: add error metrics to target and source

Looks fine, but if these are in hotpaths that trigger a very large number of times per sec, that sprintf is going to show up in alloc graphs.

These are only error metrics, so I think it should avoid hotpaths. Only scenario I can think of is if a node goes down and we keep incrementing failure metric under the background health checker


fix: redirect and canonical urls

Docs updates should not be submitted in this repository
Our doc site content is being automatically updated from Wing repository docs folder.
Please submit any changes to the docs as a PR here

@melissamcewen apparently this fix by @ainvoner fixes the canonical links issue, so no need to make changes in Vercel.


Suggestion - calendar module (not sure how to do it)

I'd like to make a small change to the next release, but I have no clue on how to do it.
Currently, calendar logs by default "Broadcasting: X events."
I'd like to change it to be "Broadcasting: X events from Y"
X: Number of events
Y: Name of calendar.
This would make debugging easier without enabling the debug flag, which massively overdisplays what I'm looking for.
Thanks!

With "Name of calendar" do you mean the url? That would be fairly easy to implement.

Not exactly.
{
 url: "https://calendar.google.com/calendar/ical/en.usa%23holiday%40group.v.calendar.google.com/public/basic.ics",
 name: "Holdays",
},

The "name" here if used, if not, then the URL.

we don't have a name attribute. that is added into config for calendarExt3 and agenda.
you can see the whole config, so given one known value, you can find the rest.

but.  we already have that function
from the doc

sam: This is making logging better - not broadcast purposes. I'm aware It's already there - I already use the name as you can see from my code snip above.

sorry, logging...

	this.broadcastEvents = function () {
		Log.info(`Calendar-Fetcher: Broadcasting ${events.length} events.`);
		eventsReceivedCallback(this);
	};

this statement.

We dont have much meta-information there about the calendar so passing name down there involves a lot of code just for logging purpose. Wouldnt the url be sufficient?

The URL would be a fine backup. The name I'm looking for is in the config.js and passed through to the CALENDAR_EVENTS broadcast array in every event as a named item "calendarName", so I know it's stored somewhere in the code, I just can't find it.

the problem is things are all over the place.
calendar.js does and add calendar() which doesn't pass the name
this goes down to the node helper, which doesn't have the name, which starts an async fetcher, which doesn't get the name..
it gets the events back from the util lib, and sends them up to the socketNotificationReceived in calendar.js, but there is no name in the data.
so all we know is there are a bunch of events to broadcast.
I was about to post the 'simple' code yesterday, but it turned out to need changes in 10 places to pass the name around, just to put it in that message IF it was specified (95% of the time not)

Sam, compared to the raw data put out in debug mode, it's nothing, and it's being logged - if you have access to the log, you also have access to the config.js, no?
Silly questions, for my own benefit and clarification -
the function that creates this log item is doing the actual broadcast, right? Why can't we pull the name out of the broadcast string (calendarName), or is the spaghetti code too much for that, even? are the logging points in the wrong spot?

the function that creates this log item is doing the actual broadcast, right?

no.. it puts out the message(in calendarfetcher.js, which doesn't know the config.js and doesn't have the name) ,
	this.broadcastEvents = function () {
		Log.info(`Calendar-Fetcher: Broadcasting ${events.length} events.`);
		eventsReceivedCallback(this);
	};

and then another function (in node_helper.js, which doesn't know the name) sends the events UP
broadcastEvents: function (fetcher, identifier) {
		this.sendSocketNotification("CALENDAR_EVENTS", {
			id: identifier,
			url: fetcher.url(),
			events: fetcher.events()
		});
	}

to the calendar.js (only thing that can actually broadcast)
if (notification === "CALENDAR_EVENTS") {
			if (this.hasCalendarURL(payload.url)) {
				this.calendarData[payload.url] = payload.events;
				this.error = null;
				this.loaded = true;

				if (this.config.broadcastEvents) {
					this.broadcastEvents();
				}
			}

and IT creates the message that is broadcast
but the message is log since printed
broadcastEvents: function () {
		const eventList = this.createEventList(false);
		for (const event of eventList) {
			event.symbol = this.symbolsForEvent(event);
			event.calendarName = this.calendarNameForUrl(event.url);
			event.color = this.colorForUrl(event.url, false);
			delete event.url;
		}

		this.sendNotification("CALENDAR_EVENTS", eventList);
	}

MAYBE the 'fix' is to move the message to here, at the actual broadcast
then we could use the function to get the name using the url
	calendarNameForUrl: function (url) {
		return this.getCalendarProperty(url, "name", "");
	},

Thanks for the explaination.. I guess the short term solution then would be the URL, with some code streamlining in the future?

so, I tried to implement moving the logging message as proposed above... oops.. calendar.js runs in the browser, SO the message comes out in the developers window, NOT in the terminal window like currently...

Thanks for the explaination.. I guess the short term solution then would be the URL, with some code streamlining in the future?

Yes indeed. Do you want to tackle it?


What is Arg?

Hi, I'm trying Dip. One thing that I wonder is what Arg is, as in
public func resolve<T, Arg1>(tag tag: Tag? = nil, withArguments arg1: Arg1) throws -> T {
    return try resolve(tag: tag) { (factory: (Arg1) -> T) in factory(arg1) }
  }

I can't seem to find where Arg1 is defined

It's a generic type (as defined inside the angle brackets), exactly like the type T; so it can be anything you want!

@AliSoftware: Thanks for your very fast response. I just realized that.


Azure Core beta release preparation for September 2021

Updated CHANGELOGs and READMEs, as well as dependency versions.

/run azp android - communication - ci

/azp run android - communication - ci


Collect diurnal hyperspectral images for calibration

As discussed with @remotesensinglab, @czender, @smarshall-bmr
We need to image the spectralon targets every 15 min throughout the day,
The first priority is a clear day
The second priority is to collect images on cloudy days
These datasets will be used in the hyperspectral calibration.
Details

Scan spectralon target of varying reflectances every 15 min over the course of a clear day
Ideally we will have 30 or 40% reflectance targets
Files should be identified, clipped to the target area, averaged within each band
Next step is to create an algorithm to use these for calibration as a function of zenith angle and downwelling radiance from the Environmental logger as described in #88

Completion Criteria

[ ] set of files for use in calibration

Issue moved to terraref/extractors-hyperspectral #17 via ZenHub


Can't get icons to show in iTerm.

How do I get the icons to work in iTerm? I did run brew cask install font-hack-nerd-font which was successful.

iTerm2 -> Preferences -> Profiles -> Text -> Font -> Change Font
Under Non-ASCII font select Hack


feat(packages/builder): Allow custom clients to inject custom launche…

…rs into electron builds
git cherry-pick 8d469b98a62253eeaed49b2334ea014560576ddf
[cherrypick6 946d6bd53] feat(packages/builder): Allow custom clients to inject custom launchers into electron builds
Fixes #7220

Description of what you did:

My PR is a:

[ ] 💥 Breaking change
[ ] 🐛 Bug fix
[x] 💅 Enhancement
[ ] 🚀 New feature

Please confirm that your PR fulfills these requirements

[x] Multiple commits are squashed into one commit.
[x] The commit message follows Conventional Commits, which allows us to autogenerate release notes; e.g. fix(plugins/plugin-k8s): fixed annoying bugs
[x] All npm dependencies are pinned.

This has received extensive testing on the dev branch.
/lgtm


Fix/build process

Description

This aims to ease the process of building iohook.
Motivation and Context

Before this PR, the install would fail because prebuilts are inexistent. Now, it just informs the user that they have to build it themselves.
The default behavior of npm run build is, as the install script, to look for a main package.json and build the targets listed by the user. If no target found, or no package.json found, it just builds the current platform. This is more convenient, as a user just want to build their needed targets.
A new behavior, used in the CI, is npm run build -- --all that mimic the previous behavior (build all supported targets).
How Has This Been Tested?

Works locally ™
Screenshots (if appropriate):
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] My code follows the code style of this project.
[x] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

Hey guys @ykhwong @marcelblum, I could use an extra pair of eyes here. Thanks for your recent work on this lib. What do you think of this PR? Tried to add more context in the description ☝

Hey good idea, tested working for me! I was definitely a little surprised the first time I ran npm run build and it tried to build for a zillion targets all at once :smile:.
IMHO manual-build.md could still be made more clear:

I think in the Building for specific versions of node section it would help to explicitly state that if you clone the repo and npm install, it automatically tries to download the official prebuild for your detected current platform and extract it into /builds. And then npm run build with no parameters builds for your detected current platform and dumps it into /build.

Also I think at least for Windows & Mac the prerequisites mentioned (e.g. "Install: msys2 with autotools, pkg-config, libtool, gcc, clang, glib, C++ Build Tools") are out of date/no longer necessary? For me all I had to do was clone the repo, npm install and npm run build and it just worked. I believe all that is needed is https://github.com/nodejs/node-gyp and it's platform-specific prerequisites mentioned there in its readme. On Windows for example, when you run the official Node.js installer, if you check the box for "install tools for native modules", everything needed for node-gyp is automagically installed along with Node itself. On Mac, all you need is the xcode command line tools installed AFAICT since a compatible version of Python is already included with macOS.

Regarding the build process, everything looks good to me. Thanks.


Server stores mapname and map_idx. We probably don't need both

Now that we have g_defined_map_datas we probably don't need to store both mapname and mapidx in the database. We can store one and lookup the other as needed.
I believe we communicate with the client primarily via mapidx, so I would say maybe use that? Also, then we're not storing the map-prefix (DefaultMapInstances) in the database. Which is good, in the event that a server operator changes their map prefix at some point after character's have been moving around the world

Per eos428 and #499 we probably want to use MapIDX as it eliminates some other issues.


Question: Understanding Kafka Statistics Event

Description
Hi,
I'm looking to understand the kafka statistic event and the values that come from this.
I'm using kafka inside of Azure Service Fabric with 5 brokers. I have logging wrapped around this statistic events and I'm seeing some behavior I don't quite understand.

Although I have registered 5 brokers, there are a total of 10 brokers from the statistic events.

5 have IP addresses I've registered. 5 have what look like DNS of the SF cluster that weren't registered.

The brokers that are sending and receiving the most messages are the 5 additional brokers and not the 5 original brokers. (5 unregistered brokers have 100,000+ | 5 registered <1000).

From logging, statistic logs that are coming later have lower total messages count sometimes. That is a log at 10:00AM has more messages than a log at 10:01AM, if messages sent and received are total counts how is this possible?

Producer Configuration
{ "bootstrap.servers", _brokerList },
{ "batch.num.messages", 1 },
{ "auto.offset.reset", "smallest" },
{ "socket.nagle.disable", true },
{ "socket.blocking.max.ms", 1 },
{ "queue.buffering.max.ms", 1 },
 { "default.topic.config", new Dictionary<string, object>
         {
               { "acks", 0 }
          }
}

Consumer Configuration
{ "group.id", "some-consumer" },
{ "enable.auto.commit", enableAutoCommit },
{ "bootstrap.servers", brokerList }, //5 registered brokers
{ "statistics.interval.ms", 60000 },
{ "default.topic.config", new Dictionary<string, object>
        {
              { "auto.offset.reset", "smallest" }
         }
}

Checklist
Please provide the following information:

[x] Confluent.Kafka nuget version: 0.11.0
[ ] Apache Kafka version:
[ ] Client configuration:
[x] Operating system: Windows Server 2016
[ ] Provide logs (with "debug" : "..." as necessary in configuration)
[ ] Provide broker log excerpts
[ ] Critical issue

re number of brokers:
https://github.com/edenhill/librdkafka/wiki/FAQ#number-of-broker-tcp-connections

Closing as the above answered the question. Thanks!


save Predictions

Hello
I am trying to save predictions ( predicted values and true values) either as .mat file or .npy file. however, i can't find on the internet how can i saved in tensorflow? can you help me with that?
Thank You

The variable pred in summary, pred, av_test_err, av_test_acc = sess.run([test_summary_op, prediction, _loss, accuracy],  could be directly operated by python, you can concat it with y_test and save as .csv, .mat or .npy.


Import terraform-provider-elasticsearch, update for opensearch

Description
Imports terraform-provider elasticsearch with updates to resource names for removing elasticsearch naming
Issues Resolved

Closes https://github.com/phillbaker/terraform-provider-elasticsearch/issues/299

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Also closes https://github.com/opensearch-project/opensearch-devops/issues/82, which is directly related to the issue already referenced in the description.

Do we want to begin work around completely removing anything with elasticsearch in the name?

This is mostly done in this PR. All resources are now named using opensearch. Looks like I missed the examples directory though, I'll remove those from this PR.

Also closes opensearch-project/opensearch-devops#82, which is directly related to the issue already referenced in the description.
@phillbaker Do we want to begin work around completely removing anything with elasticsearch in the name? My assumption was that your main focus initially is just to get the repo bootstrapped. Then we can focus on refactoring and stripping out some of the elasticsearch-specific stuff. Let me know what you had in mind.

Thanks a lot for your contributions. I will second this. It would be great if we can remove all references to elasticsearch in the name before merging this PR.

@bbarani have you been able to look at the settings on this repo to enable github actions?

@bbarani have you been able to look at the settings on this repo to enable github actions?

I have enabled GitHub actions now. Please let me know if you are still seeing any issues.


Added LanguageViewLocationExpander into framework

cc @rynowak

:watch:

:shipit: from me. Please get signoff from @pranavkm as well

:shipit: once comments are addressed.


Planting Timber Logs the Right Way

Article checklist
ATTENTION: In your PR - add the ISSUE (topic form) # that this PR corresponds to. You can do this by also adding a comment that states "This fixes #(enter your Idea Content Form #)" to link and close your Idea Content (issue) when the PR is merged.
NOTE: (Please ensure that you have only one open issue + linked pull request at a time. This will ensure that we complete the article in a timely manner from inception to publishing.)
If you have not already please go over our Resources Page for more tips and tricks on how to improve our overall technical writing.
Preliminary Checks - Formatting and Structure

[x] Is the article you are submitting an in-depth and unique article? Does it go beyond what is in the official docs and what is covered in other blog sites. See these articles as examples.
[x] Does your article follow any of the suggested structure formats? - see example formats
[x] Is your article properly formatted in Markdown? - see Github Markdown guide
[x] Have you used the correct folder and file structure? - see Contribution Guidelines
[x] Is your article over 750 words? - Tool: Wordcounter.net
[x] Have you used our preferred heading structure? - H3 (###) and up
[x] Does your article provide enough value and detail about your topic? - Articles should be clear, accurate, and fully explained.
[x] Can your article be understood by beginners? - Assume the audience is smart but has no prior exposure to the common terminology in your article.
[x] Have you included a hero image that is 600x400 pixels and have the copyright to use it? - Tools: Pixlr Image Editor for resizing and Unsplash for Creative Commons images
[x] In your PR - add a comment that states "This fixes #(enter your Idea Content Form #)" to link and close your Idea Content (issue) when the PR is merged.

Grammar & plagiarism checks

[x] Have you spellchecked and grammar checked your article? - Tools: Write&Improve - ProWritingAid -  Grammarly
[x] Please place your article through a 3rd party plagiarism checker? We suggest using Quetext, this tool is free and has a daily limit. We typically accept articles with 10% or less.
[x] Have you checked your article for readability? - Tool: Hemmingway
[x] Have you added sources for quotes and images that aren't yours?

Technical checks

[x] Are your code snippets properly formatted for syntax highlighting - see Syntax guide

Contribution guidelines
For first-time contributors and for more details, see Contributing Guidelines
This fixes #2569

@Owallah you should request the topic to be reopened before creating a PR. https://github.com/section-engineering-education/engineering-education/issues/2569

Hello Mr. Odhiambo.
How's the progress so far?

@Owallah you should request the topic to be reopened before creating a PR. #2569

I already requested and it was opened. Thanks for the headsup.

@Owallah
I'll be running a final review on your PR when ready.
Upon running your article through our 3rd party plagiarism checker it seemed to raise a few flags and the % was higher than we typically accept.
Please see attached PDF - and revisit the article to ensure we are contributing wholly unique and original content.
Be sure to see our resources page to see more info on plagiarism and what is considered as such.
Planting Timber Logs the Right Way #3191.pdf

@hectorkambow
I have worked on the plagiarism as you instructed.  I had some issues though, the site quetext couldn't allow me to sign-up. So I used prepostseo instead to check the plagiarism percentage after I was done. Receive my apologies in advance if it wasn't as effective as quetext is.

Good morning @paulodhiambo

@hectorkambow
I have worked on the plagiarism as you instructed. I had some issues though, the site quetext couldn't allow me to sign-up. So I used prepostseo instead to check the plagiarism percentage after I was done. Receive my apologies in advance if it wasn't as effective as quetext is.

That it totally acceptable - we encourage you to use any tool as long as it helps out.
We will re-check it 👍

@hectorkambow
Thank you.

Hello @paulodhiambo

What's the progress so far?

Good morning @paulodhiambo
I hope you had a great weekend.

@paulodhiambo
Is there anything I'm supposed to do at this stage of plagiarism check?

@paulodhiambo
Thank you

You don't have to do anything.

@paulodhiambo
It's okay

Planting Timber Logs the Right Way #3191 2nd.pdf
Upon running your article through our 3rd party plagiarism checker it seemed to raise a few flags and the % was higher than we typically accept.
Please see attached PDF - and revisit the article to ensure we are contributing wholly unique and original content.
Be sure to see our resources page to see more info on plagiarism and what is considered as such.
Seeing the same % 🤔

Good morning sir @hectorkambow ,
I have seen the message you have sent above. Going through the document, I have noticed that the plagiarism checker is flagging most of my own written code. I guess this is what is raising the percentage.
I am however going to look at the text part and try and change a few things.
Thank you.

Hello @hectorkambow and @paulodhiambo
I have done a few changes to the article. This is in reference to the message @hectorkambow sent earlier.
As stated earlier, in the document sent, most of my code was being flagged. This can however not be simplified further as it will distort the code.
Again some of the content which I have stated From the official documentation has been flagged. This too cannot be simplified further as it is exactly what the official documents say. If I change that, that portion will lose meaning.
Looking forward to your reply. Thank you!

Hello @paulodhiambo
Any progress so far?

@hectorkambow will check the plagiarism again.

It's okay @paulodhiambo

Hello @hectorkambow , @WanjaMIKE , @paulodhiambo
Good morning.

Any progress on the plagiarism check?

@Owallah Some minor issues still appear to be flagged in the plagiarism report. Could you check them out.
Planting Timber Logs the Right Way #3191.pdf

Hello @WanjaMIKE I have received your message. I have viewed the document. I have one problem though

Most of what the plagiarism check has flagged is the code. I can however not change that.

Most of it is autogenerated by Android Studio.
an example is this
<?xml version="1.0" encoding="utf-8"?> <androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android" xmlns:app="http://schemas.android.com/apk/res-auto" xmlns:tools="http://schemas.android.com/tools" android:layout_width="match_parent" android:layout_height="match_parent" tools:context=".MainActivity">
Changing that will cause errors to the whole xml code

Most of it is autogenerated by Android Studio.
an example is this
<?xml version="1.0" encoding="utf-8"?> <androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android" xmlns:app="http://schemas.android.com/apk/res-auto" xmlns:tools="http://schemas.android.com/tools" android:layout_width="match_parent" android:layout_height="match_parent" tools:context=".MainActivity">
Changing that will cause errors to the whole xml code

Yes, You can leave the boilerplate code. Only the minor sentences need to be changed. Thanks

Hello @WanjaMIKE
I have done changes. Please check it out

Hello @paulodhiambo

@paulodhiambo - please let me know if you believe the article is ready for a final review

@paulodhiambo
I believe Mike ran a plagiarism check on this yesterday - does it need another?

@WanjaMIKE said the percentage of plagiarism was still a bit higher.

@hectorkambow and @paulodhiambo
Yeah, @WanjaMIKE  had instructed that I work on a few texts and I did and committed the changes I made.

@paulodhiambo please proceed. The plagiarism check was completed. Thanks

Hi @Owallah
Thank you for this amazing article. Keep up the amazing job.
cc: @paulodhiambo


Container focus not working when having absolute divs inside scroll-container

Environment

Browser:
Version of smooth-scrollbar:
8.6.3

Issue Summary

The issue is that If I have an absolute div inside the scroller then scroller.focus() does not work (e.g on window load if I press key down it does not scroll the page down, I had to click with the mouse on the window then key down works and the second thing is that when I scroll to the bottom of the page or back to the top then container also lost focus and I had to click again on the window to activate it to scroll up/ down). I attached the demo for better understanding and reproduction of the issue.
Current Behavior

Page does not scroll down with down key press and lost focus when get back to top.
Expected Behavior

It should start scroll down on pressing down key on window load as a normal behavior.
Steps to Reproduce

Online demo

https://codepen.io/ShehzadAsif/pen/wvdYymO?editors=0010

Just found the solution: add continuousScrolling: false to scrollbar options will fix this, i.e.
const bodyScrollBar = Scrollbar.init(scroller, {
  damping: 0.1,
  continuousScrolling: false,
});

Thanks, it solved the problem
On Mon, Aug 23, 2021 at 2:24 PM Daofeng Wu @.***> wrote:

Just found the solution: add continuousScrolling: false to scrollbar
options will fix this, i.e.
const bodyScrollBar = Scrollbar.init(scroller, {
damping: 0.1,
continuousScrolling: false,});
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/idiotWu/smooth-scrollbar/issues/370#issuecomment-903595648,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/APQYRBU5WV65LM4EEAV7W6TT6IHUDANCNFSM5BWI3UGQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email
.