Possible Issue with Change Tracker Caching Entity State EF Core 2.0.2

When pulling the Entries from the change tracker, state isn't updated properly. I have this test:
        [Fact]
        public void ShouldTrackChangesByDefault()
        {
            Product prod = _context.Product.First();

            List<EntityEntry<Product>> entries =
                _context.ChangeTracker.Entries<Product>().ToList();

            Assert.Single(entries);

            var name = prod.Name;
            prod.Name = "foo";

            Assert.Equal(name,
                entries[0].OriginalValues[nameof(Product.Name)].ToString());

            Assert.Equal(prod.Name,
                entries[0].CurrentValues[nameof(Product.Name)].ToString());

            Assert.Equal(EntityState.Modified,entries[0].State);
        }

This fails n the last asset - state is reported as Unchanged.
However, if I access the state, it then gets properly reported as Modified, as in this test:
        [Fact]
        public void ShouldTrackChangesByDefault()
        {
            Product prod = _context.Product.First();

            List<EntityEntry<Product>> entries =
                _context.ChangeTracker.Entries<Product>().ToList();

            Assert.Single(entries);

            var name = prod.Name;
            prod.Name = "foo"; //TODO: Log issue with EF Core Team 05_07

            Assert.Equal(name,
                entries[0].OriginalValues[nameof(Product.Name)].ToString());

            Assert.Equal(prod.Name,
                entries[0].CurrentValues[nameof(Product.Name)].ToString());

            //added this line and it fixes the test
            var foo = _context.Entry(prod).State;
            Assert.Equal(EntityState.Modified,entries[0].State);
        }

Is this by design, am I doing something stupid, or is this an issue?
Steps to reproduce
Include a complete code listing (or project/solution) that we can run to reproduce the issue.
Partial code listings, or multiple fragments of code, will slow down our response or cause us to push the issue back to you to provide code to reproduce the issue.
Further technical details
EF Core version: 2.0.1 and 2.0.2
Database Provider: Microsoft.EntityFrameworkCore.SqlServer
Operating system: Windows 10
IDE: Visual Studio 2017 15.6.5
PossibleBugSample.zip

@skimedic This is because in order to know whether or not a property has been changed, EF needs to run DetectChanges, which does the diff to determine what has changed. This can be quite expensive, and so EF only calls it automatically as part of some top-level methods. In this case, it is called for Entry and Entries, but it is not called every time you access a property through the entry due to the perf cost. There are some blog posts that describe DetectChanges in detail for EF6, and the concepts are basically the same for EF Core.

@ajcvickers Thank you for the explaination. I figured there was a reason for it.


ILB behavior with AWS ELB

I setup two AWS instances running 1.12.0-rc2, and ran a simple service as such:
docker service create --replicas 2 --network mynet --name whoami -p 8000:8000 jwilder/whoami
I then configured an ELB to point to these instances. I can see load balancing happening:
~$curl -sSL whoami-584842023.us-west-1.elb.amazonaws.com
I'm 474cbb74a3ed
~$curl -sSL whoami-584842023.us-west-1.elb.amazonaws.com
I'm 46b89a421ff7
However, to testdrive ILB, I removed one of the instances from ELB configuration. My assumption is that it shouldn't matter, i.e., routing mesh should still ensure load balancing across containers. But to my bewilderment, it is routing traffic only to one container after that:
~$curl -sSL whoami-584842023.us-west-1.elb.amazonaws.com
I'm 46b89a421ff7
~$curl -sSL whoami-584842023.us-west-1.elb.amazonaws.com
I'm 46b89a421ff7
~$curl -sSL whoami-584842023.us-west-1.elb.amazonaws.com
I'm 46b89a421ff7
more surprisingly, this is the container on the instance that I removed from LB configuration!
Can someone please explain what could be going on here?

@rhim you are trying a valid scenario and it should work as you expect. This seems like a bug. If you can try adding the other node to the ELB (and remove the faulty node), does it load-balance properly ?

@mavenugo I also noticed something funny. When I tried after a long inactive period (~20 hrs), I saw that it did route to both containers once, before resorting again to routing to a single container.

~$curl -sSL whoami-584842023.us-west-1.elb.amazonaws.com
I'm 46b89a421ff7
~$curl -sSL whoami-584842023.us-west-1.elb.amazonaws.com
I'm 474cbb74a3ed
~$curl -sSL whoami-584842023.us-west-1.elb.amazonaws.com
I'm 46b89a421ff7
~$curl -sSL whoami-584842023.us-west-1.elb.amazonaws.com
I'm 46b89a421ff7
~$curl -sSL whoami-584842023.us-west-1.elb.amazonaws.com
I'm 46b89a421ff7

I then removed the "faulty" node (say node-1), and replaced it with the other node (node-2). Results are similar, as in the requests are now only routed to the container on node-1.

~$curl -sSL whoami-584842023.us-west-1.elb.amazonaws.com
I'm 474cbb74a3ed
~$curl -sSL whoami-584842023.us-west-1.elb.amazonaws.com
I'm 474cbb74a3ed
~$curl -sSL whoami-584842023.us-west-1.elb.amazonaws.com
I'm 474cbb74a3ed

So, in effect, the problem remains irrespective of the node.
Putting both nodes back in ELB configuration restores the correct load balancing behavior.
Seems like the packets are being routed to the node other than the one that receives the request. I.e, when ELB routes to node-1, routing mesh then routes it to node-2's container, and vice-versa. That would explain the behavior I am seeing. However, that's not the intended behavior I expected from ILB.

@rhim It has been detected that this issue has not received any activity in over 6 months. Can you please let us know if it is still relevant:

For a bug: do you still experience the issue with the latest version?
For a feature request: was your request appropriately answered in a later version?

Thank you!
This issue will be automatically closed in 1 week unless it is commented on.
For more information please refer to https://github.com/docker/libnetwork/issues/1926


AttributeError: 'str' object has no attribute 'get_info'

Hi,
I am using the latest revision from the GIT repo, I have tested it with the aggregation option but am receiving the error below. I have tried a few different rule types and default alerting or custom alerting. I can't work out what it is, I removed the get_info() function off the 'rule['alert'][0]' and it works without an error. But this doesn't fix the bug, it also does not appear to be an issue when not using aggregation.
customer ERROR:root:Traceback (most recent call last):   File "/usr/local/lib/python2.7/dist-packages/elastalert-0.1.21-py2.7.egg/elastalert/elastalert.py", line 1079, in run_all_rules     num_matches = self.run_rule(rule, endtime, self.starttime)   File "/usr/local/lib/python2.7/dist-packages/elastalert-0.1.21-py2.7.egg/elastalert/elastalert.py", line 839, in run_rule     self.add_aggregated_alert(match, rule)   File "/usr/local/lib/python2.7/dist-packages/elastalert-0.1.21-py2.7.egg/elastalert/elastalert.py", line 1594, in add_aggregated_alert     alert_body = self.get_alert_body(match, rule, False, alert_time)   File "/usr/local/lib/python2.7/dist-packages/elastalert-0.1.21-py2.7.egg/elastalert/elastalert.py", line 1350, in get_alert_body     'alert_info': rule['alert'][0].get_info(), AttributeError: 'str' object has no attribute 'get_info'
Many thanks in advanced

Looks like it only does it when running --debug


iOS platform views throws thread checker error on hot restart with local unopt engine

Error: (This is all the information that shows, couldn't get an engine stack trace)
[VERBOSE-2:thread_checker.h(61)] IsCreationThreadCurrent expected thread: '`\M^BU\^A' actual thread:'io.flutter.1.raster'
[VERBOSE-3:weak_ptr.h(122)] Check failed: (checker_.checker).IsCreationThreadCurrent().

device: iPhone 7 simulator, iOS 13.3
Steps to reproduce

Build local engine with "ios_debug_sim_unopt"
Run an app with platform views. For example: the example app in https://github.com/flutter/plugins/tree/master/packages/webview_flutter
After app loads, hit R to hot restart, see crash with above error message.

Confirmed that it is also crashing on devices.

It seems that launching the camera plugin example app with local unopt engine results this failure too.
CC @iskakaushik

https://github.com/flutter/flutter/issues/70577 is likely unrelated.

I've noticed this same issue with engine commit 45f6e009110df4f34ec2cf99f63cf73b71b7a420 for linux-x64 with --runtime-mode debug --unoptimized --no-goma built on my own machine. When using a flutter_infra_release build, I don't get this issue.


⚠ Bofrost.be has degraded performance

In e1d5db1, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 3395 ms

Resolved: Bofrost.be performance has improved in f1c9b81.


fix(amplify-appsync-simulator): fixes util.isNullOrEmpty

fixes util.isNullOrEmpty returning incorrect values when invoked with a boolean or string value
fix  #4393
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@yuth @attilah Are we any closer to assessing whether this can be merged. There are two linked issues depending on this?

I tested the behavior of using non string type in AppSync service and it looks like the service renders the literal as is if the parameter passed to $util.isNullOrEmpty is of any other types.
To have the same behavior there is some additional change that would be needed in Velocity template engine

I tested the behavior of using non string type in AppSync service and it looks like the service renders the literal as is if the parameter passed to $util.isNullOrEmpty is of any other types.
To have the same behavior there is some additional change that would be needed in Velocity template engine

Could you please provide more detail on the test you performed and the additional change that is required?
Also, I have updated the tests to add more test cases.

Could you please provide more detail on the test you performed and the additional change that is required?

So I used the AppSync consoles, Velocity template test feature. My test context contains these values
{
    "arguments": {
        "id": "BEoDcq9lqwVO0ShcHNR77Yk8pqjnBLu0",
        "isFalse": false,
        "isTrue": true,
        "isEmptyString": "",
        "isNonEmptyString": "foo bar",
        "isNull": null
    },
    "source": {
        "echo": "Hello, world!"
    },
    "result": {
        "id": "OrdbGHOruchIqNirE9qQ1Rd9W3mD6hiQ",
        "name": "Hello, world!",
        "description": "Hello, world!"
    }
}

and request template contains
#set($foo = $util.isNullOrEmpty(true))
{
  "version": "2017-02-28",
  "operation": "GetItem",
  "key": #if( $modelObjectKey ) $util.toJson($modelObjectKey) #else {
  "id": $util.dynamodb.toDynamoDBJson($ctx.args.id),
  "isTrueNotNull": $util.isNullOrEmpty($ctx.args.isTrue),
  "isFalseNotNull": $util.isNullOrEmpty($ctx.args.isFalse),
  "isNonEmptyString": $util.isNullOrEmpty($ctx.args.isNonEmptyString)
  "isEmptyString": $util.isNullOrEmpty($ctx.args.isEmptyString)
  "isStringNotNull": $util.isNullOrEmpty(12),
  "isNull": $util.isNullOrEmpty($ctx.args.isNull)
  #if($ctx.args.isTrue)
   isTrue = true
  #end

  #if($ctx.args.isFalse)
     isFalse = false
  #end
} #end
}
${foo}

and this is what gets rendered by VTL on AppSync service

Closing the PR as we can't implement the exact behavior of appsync with this implementation of mock.


Feature request for improve browsing、searching and description management for column.

I have applied Amundsen to our company since 3 months ago, my colleagues become favorite user of Amundsen (one hundred people).
Here are some feedbacks from daily usage of them, hope it’s helpful for reference of feature updating.
Expected Behavior or Use Case

All table browsing：Hoping some where can browse all tables.
Browsing by tag level or multiple tag searching：Hoping there’s feature like tag level or multiple tag searching.
A Column description can be used for multiple table and editing synchronize.

Service or Ingestion ETL

Frontend
Elasticsearch

Possible Implementation

All table browsing
Add a button in navigation, maybe call it Browsing, and show all table with schema by list.
Browsing by tag level
For example, feature Follow can be using follow against 3 object：
Forum > Follow > Post
Forum > Follow > User
Forum > Follow > Topic
Multiple tag searching
Follow + Post + DailySnapshot(Grain of data)
Follow + Post + RawData(Grain of data)
Common column with editing synchronize
User can create a common column description by Web UI.
A common column description can be added to multiple table.
Context
All table browsing
Because there’s no keyword in mind of some user, especially for new co-worker .
Browsing by tag level
It's an efficiency way for information browsing.
Multiple tag searching
Get more specific result by input multiple Tag.
Common column with editing synchronize
Some column had a lot of attributes, appears in multiple table, updating frequently, and generated by non-technical guys.

It will be good to have All table browsing

All table browsing it's awesome! I'm needing this feature.
The others features that u said it's very good too

For the all table browsing, this issue is related: https://github.com/amundsen-io/amundsen/issues/138

There is multiple tag searching since we added comma separated filtering in the search page


Check failed: !the_factory_methods.empty() KV260

Hi i am struggling to run a custom DPU application through xmutil on petalinux 2021.2. I have attached my application. when loading the application i get this output:
xilinx-k26-som-2021_2:~/Vitis-AI/demo/VART/resnet50$ sudo xmutil loadapp user-firmware
DFX-MGRD> daemon loading accel user-firmware

Jan  3 04:56:56 xilinx-k26-som-2021_2 kernel: [ 1362.791145] OF: overlay: WARNING: memory leak will occur if overlay removed, property: /fpga-full/fpga-config-from-dmabuf
Jan  3 04:56:56 xilinx-k26-som-2021_2 kernel: [ 1362.791159] OF: overlay: WARNING: memory leak will occur if overlay removed, property: /fpga-full/firmware-name
Jan  3 04:56:56 xilinx-k26-som-2021_2 kernel: [ 1362.791165] OF: overlay: WARNING: memory leak will occur if overlay removed, property: /fpga-full/resets
Jan  3 04:56:56 xilinx-k26-som-2021_2 kernel: [ 1362.791305] OF: overlay: WARNING: memory leak will occur if overlay removed, property: /__symbols__/overlay0
Jan  3 04:56:56 xilinx-k26-som-2021_2 kernel: [ 1362.791320] OF: overlay: WARNING: memory leak will occur if overlay removed, property: /__symbols__/overlay1
Jan  3 04:56:56 xilinx-k26-som-2021_2 kernel: [ 1362.791335] OF: overlay: WARNING: memory leak will occur if overlay removed, property: /__symbols__/afi0
Jan  3 04:56:56 xilinx-k26-som-2021_2 kernel: [ 1362.791351] OF: overlay: WARNING: memory leak will occur if overlay removed, property: /__symbols__/clocking0
Jan  3 04:56:56 xilinx-k26-som-2021_2 kernel: [ 1362.791366] OF: overlay: WARNING: memory leak will occur if overlay removed, property: /__symbols__/overlay2
Jan  3 04:56:56 xilinx-k26-som-2021_2 kernel: [ 1362.791381] OF: overlay: WARNING: memory leak will occur if overlay removed, property: /__symbols__/DPUCZDX8G_0
Jan  3 04:56:56 xilinx-k26-som-2021_2 kernel: [ 1362.791396] OF: overlay: WARNING: memory leak will occur if overlay removed, property: /__symbols__/misc_clk_0
DFX-MGRD> Successfully loaded base design.

Accelerator loaded to slot 0
Socket 6 closed by client

my lsmod output is:
lsmod
Module                  Size  Used by
usb5744                16384  0
dmaproxy               16384  0
mali                  233472  0
uio_pdrv_genirq        16384  0

if i run example vitis ai resnet50 i get the output:

sudo ./resnet50 /usr/share/vitis_ai_library/models/resnet50.xmodel
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0103 04:57:04.826944  1532 main.cc:292] create running for subgraph: subgraph_conv1
F0103 04:57:04.844861  1532 dpu_controller.cpp:44] Check failed: !the_factory_methods.empty()
*** Check failure stack trace: ***
Aborted

packagegroup-petalinux-vitisai is enabled in petalinux.
I have also loaded the build target_vart_setup.sh
My block Design is attached and so is my application -> user-firmware.zip

Hi @jackbonnellssa
I don't  konw where and how do you build your xilinx-k26-som rootfs system in 2021.2 version, and I found that there is no zocl driver in your lsmod command output, in my opinion, you need to add zocl driver to your rootfs system. by the way, zocl is also used for XRT, so your rootfs system doesn't enable xrt either. you can run petalinux-config -c rootfs command to enable zocl and xrt into your rootfs system.

Hi @jackbonnellssa
Since we haven't received your reply for a long time, we assume you have solved this issue and I'm going to close it. If you still have any questions, please feel free to reopen it. Thank you very much.

Hi @qianglin-xlnx,
I can see that I have installed ZOCL and XRT on my system.

However, I am getting the same error message when running the model. The lsmod output follows:

If the drivers are not installed properly, how to install them?

I Running the command“./demo_yolov3 /usr/share/vitis_ai_library/models/yolov3_voc/yolov3_voc.xmodel
demo_yolov3.jpg”error“Check failed: !the_factory_methods.empty() ”
but running “sh build.sh”is ok

。

Hi @iivanov1996,
Have you ever resolved this issue?
I ran in exactly the same error with Vitis-AI 3.5 and KV260.
Thank you


Bluetooth: Iso: Do not remove iso data path on disconnect

We previously removed the iso data path when the iso channel
disconnected. Since the iso data path is unique for a given
iso channel (by handle), it does not make sense to remove
it for a disconnected channel, as the channel is, in
a sense, not existing anymore.
This update is to better comply with the bluetooth
core spec, and to avoid getting errors from the
controller on disconnect.
Rather than removing the implementation of being able
to remove the data path, the function was made non-static
and moved to the internal header file, in case we ever
want to use it. This should not affect compile size.
Signed-off-by: Emil Gydesen<EMAIL_ADDRESS>fixes #34829

The specification explicitly states that "If the Host issues [the LE Setup ISO Data Path] command more than once for the same Connection_Handle and direction before issuing the HCI_LE_Remove_ISO_Data_Path command for that Connection_Handle and direction, the Controller shall return the error code Command Disallowed (0x0C)." without considering the state or state changes of whatever is behind the Connection_Handle. I think the behavior actually intended by the specification should be clarified.

The specification explicitly states that "If the Host issues [the LE Setup ISO Data Path] command more than once for the same Connection_Handle and direction before issuing the HCI_LE_Remove_ISO_Data_Path command for that Connection_Handle and direction, the Controller shall return the error code Command Disallowed (0x0C)." without considering the state or state changes of whatever is behind the Connection_Handle. I think the behavior actually intended by the specification should be clarified.

I'm not sure I follow completely here. Are you suggesting that we amend the core specification to more clearly describe the behavior of the setup/remove iso data paths?
The paragraphs that that are related to this particular change are

The HCI_LE_Remove_ISO_Data_Path command is used to remove the input
and/or output data path(s) associated with a CIS or BIS identified by the
Connection_Handle parameter.

and

If the Host issues this command with a Connection_Handle that does not exist
or is not for a CIS or a BIS, the Controller shall return the error code Unknown
Connection Identifier (0x02).

Since the connection identifier for a iso channel is invalid after receiving the disconnect event (for connected) or the terminate BIG complete event (for broadcast), we cannot use the handle of that particular channel to remove the ISO data path.
The Zephyr host is, however, explicit when the iso channel are connected, and should ensure that the the data path is in the correct state, regardless of what state the controller may or may not have set the data path to during disconnect.

I suggest to ask BT SIG how the specification is supposed to be interpreted. I understand the current wording as follows: If you remove the ISO data path after disconnection, you get an "Unknown Connection Identifier" error (the paragraph you cited). If you do not remove the ISO data path, you get a "Command Disallowed" error when setting up the next time with the same handle (the paragraph I cited). I do not think that is what the specification really intends.

I suggest to ask BT SIG how the specification is supposed to be interpreted. I understand the current wording as follows: If you remove the ISO data path after disconnection, you get an "Unknown Connection Identifier" error (the paragraph you cited). If you do not remove the ISO data path, you get a "Command Disallowed" error when setting up the next time with the same handle (the paragraph I cited). I do not think that is what the specification really intends.

You are absolutely right.
I'll create an erratum.

Added the DNM label while I investigate if this has been changed for the upcoming BT Core 5.3 release; a lot of errata has been raised about this particular issue.

Update:
It has been a bit more clarified in BT Core 5.3:
If a CIS disconnects on a peripheral, the iso data paths are automatically removed by the controller. They remain valid for centrals.
If a host (central) removes the CIG, this also removed the ISO data paths.
The iso dath paths are automatically removed when a BIG or BIG sync is delete, terminated or lost (i.e. broadcast iso data paths are always automatically removed).
Since the Zephyr host removes the CIG when all the CIS are disconnected (either by ACL disconnect, of if they were otherwise all disconnected), we still ensure that all the iso data paths are removed with this PR.


Schema Hook cannot generate,  entc/load: load schema dir: -: import cycle not allowed

[x] The issue is present in the latest release.
[x] I have searched the issues of this repository and believe that this is not a duplicate.

Current Behavior 😯
Unable to generate code normally
Expected Behavior 🤔
Normal code generation
Steps to Reproduce 🕹

Steps:

Download compressed files  ent-test.zip
Use this project
Execute   ent generate .\ent\schema\
Mainly three files in the schema folder

Your Environment 🌎

Tech
Version

Go
1.17.2

Ent
0.10.0

Database
PostgreSQL

Driver
github.com/jackc/pgx/v4/stdlib

I'm so sorry to bother you

Maybe you can clone this project directly to the local computer  https://github.com/medivh-jay/enttest

git clone https://github.com/medivh-jay/enttest.git
cd enttest
ent generate ./ent/schema/

I use a schema hook in schema/member.go.

this issue may help you https://github.com/ent/ent/issues/892#issuecomment-717782525

I'm sorry, this is because I defined the internal structure inside the schema package myself.


If the incarnation project was existing in gitlab, foxops UI should support create a new incarnation for it

A gitlab project has been created, and I want to use foxops to manage to incarnate it, then I tried to create a incarnation using the foxops console, but the foxops threw errors like "Incarnation at '/a/b/c' and target directory './' already initialized"

actually - it's already possible via the API.
you can call the 'create incarnationAPI with anallow_import` URL parameter.


Added About us page for aqua sort and aero combat game

PR Description 📜
I have added the about us page for aero combat and aqua sort game .
Fixes #4588

Mark the task you have completed ✅

[x] I follow CONTRIBUTING GUIDELINE & CODE OF CONDUCT of this project.
[x] I have performed a self-review of my own code or work.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] My changes generates no new warnings.
[x] I have followed proper naming convention showed in CONTRIBUTING GUIDELINE
[x] I have added screenshot for website preview in assets/images
[ ] I have added entries for my game in main README.md
[ ] I have added README.md in my folder
[ ] I have added working video of the game in README.md (optional)
[x] I have specified the respective issue number for which I have requested the new game.

Add your screenshots(Optional) 📸
https://github.com/kunjgit/GameZone/assets/118350936/53f92bf5-25f9-4d90-85d3-24cfe503f8dd
https://github.com/kunjgit/GameZone/assets/118350936/c3ceec97-a756-4dd1-8d18-a0723e103eb6

Thank you soo much for contributing to our repository 💗

Thank you @Ayushmaanagarwal1211 ,for creating the PR and contributing to our GameZone 💗
Review team will review the PR and will reach out to you soon! 😇
Make sure that you have marked all the tasks that you are done with ✅.
Thank you for your patience! 😀

@kunjgit  Please review this

Thank you  @Ayushmaanagarwal1211 , for your valuable time and contribution in our GameZone 💗.
It’s our GameZone, so Let’s build this GameZone altogether !!🤝
Hoping to see you soon with another PR again 😇
Wishing you all the best for your journey into Open Source🚀


Limitations of auto-generated projectors for inductive types

This is a minimized example:
module Bug

assume val t: v:(int -> Type) -> (i:int -> v i -> Type0) -> Type

(* Works if not marked as irreducible *)
irreducible type w #v #r (x:t v r) = True

noeq type u =
| U:
  k: (i:int -> Type) ->
  x: t k (fun _ _ -> True) ->
  f: (i:int -> Tot (y:k i{w x})) ->
  u

(*
Subtyping check failed;
 expected type i:Prims.int -> Prims.Tot ((y:U?.k projectee i{ Bug.w (U?.x projectee) }));
 got      type i:Prims.int -> Prims.Tot ((y:k i{ Bug.w x }))
*)

The original issue occurs naturally in mitls-fstar where the irreducible predicate is MR.witnessed (marked as abstract in MR). Again, minimized:
module M

open FStar.HyperStack
open FStar.HyperStack.ST

module MR = FStar.Monotonic.RRef
module MM = FStar.Monotonic.Map

assume val t:eqtype

assume val r:MR.rid

noeq type pkg =
| Pkg:
  key: (i:t -> Type) ->
  define_table: MM.t r t key (fun _ -> True) ->
  create: (i:t -> ST (key i)
    (requires fun _ -> True)
    (ensures  fun _ _ h -> MR.witnessed (MM.defined define_table i))) ->
  pkg

In this case, the workaround is to either define the third parameter of MM.t before the inductive, or to add a field for it in the inductive.

This is an smt encoding issue. F* generates projectors for the inductive and fails for the projector for f. More explicitly:
module Test

#set-options "--__temp_no_proj  Test --log_queries --max_fuel 0 --max_ifuel 1 --initial_fuel 0 --initial_ifuel 1"
assume val t: v:(int -> Type0) -> (i:int -> v i -> Type0) -> Type0

assume val w (#v:(int -> Type0)) (#r:(i:int -> v i -> Type0)) (x:t v r) :Type0

noeq type test :Type u#1 =
| C:
  k: (i:int -> Type0) ->
  x: t k (fun _ _ -> True) ->
  f: (i:int -> Tot (y:k i{w x})) ->
  test

let proj_k (elem:test) :(int -> Type0) =
  match elem with
  | C k _ _ -> k

let proj_x (elem:test) :(t (proj_k elem) (fun _ _ -> True)) =
  match elem with
  | C _ x _ -> x

let proj_f (elem:test) :(i:int -> Tot (y:(proj_k elem) i{w (proj_x elem)})) =
  match elem with
  | C k x f -> f

In the above sample, typechecking proj_f fails.
The reason is that, the lambda term that F* infers as an argument to w in the return type of proj_f is not same as the lambda term that it infers for w specified in the type of  f in the data constructor C. The term is extensionally equal, but smt hashconsing does not realize it, and encodes two different lambda terms, after which it is not able to prove the refinement.
I don't think there is an easy solution to this. Lifting the lambda term to the top-level would be the best solution.

There are other cases where F* cannot typecheck the automatically generated projectors. Jean Joseph Marty recently reported the following example:
module Test
module B   = LowStar.Buffer

assume val struct (a:Type0) :Type0

noeq type t =
  | C: x:int -> fld:B.pointer (struct (u:int{u < x})) -> t

Because of a limitation of the SMT encoding, F* fails to typecheck the projector for fld. The detailed reason is:
(*
 * This straightforward implementation fails to typecheck
 *
 * Essentially, fld has type: B.pointer (struct (u:int{u < x}))
 * which we have to prove  <: B.pointer (struct (u:int{u < proj_x a}))
 *
 * Now B.pointer is a buffer with length = 1 in the refinement
 * So, F* has to prove: length #(struct (u:int{u < x})) b == 1
 *                  ==> length #(struct (u:int{u < proj_x a})) b == 1
 *
 * Because of the different type arguments here, smt runs out of luck
 * and assertions also don't help
*
* The proof needs an application of predicate extensionality
 *)

let proj_fld_fails (a:t) :B.pointer (struct (u:int{u < proj_x a}))
  = match a with
    | C x fld -> assert (proj_x a == x); fld

In any case it seems unavoidable that for some such cases F* will fail to typecheck the auto generated projectors.
For what it's worth, for the specific case above, one workaround is to disable auto projector generation (using the __temp_no_proj flag), and manually write the projector using tactics to typecheck it:
open FStar.Tactics

let coerce (a:t) (x:int) (fld:B.pointer (struct (u:int{u < x}))) (h:proj_x a == x)  //need to get hands on this equality hypothesis
  :B.pointer (struct (u:int{u < proj_x a}))

  by (match binders_of_env (cur_env ()) with
      | [_; _; _; h] -> rewrite' h; smt ()
      | _ -> fail "Impossible")

  = fld

let proj_fld (a:t) :B.pointer (struct (u:int{u < proj_x a}))
  = match a with
    | C x fld -> coerce a x fld (Squash.return_squash Refl)

We should do the following:

When typechecking an auto-generated projectors fails, we should tell the programmer that the error is in auto-generated projectors (similar to how we do for hasEq failures).

Get rid of the __temp_no_proj flag, and instead provide an attribute that when slapped on an inductive disables the projector generation for that inductive. And then the programmer has to manually write the projectors and use them.

When some inductive has that attribute, we should not even generate the declarations of the projectors (which we do currently in the case of __temp_no_proj flag). Otherwise extraction would give errors.

But this is low priority.

@tahina-pro reports more instances of problems with auto-generated projectors:
noeq type t = | T :
  (u: (nat -> Type0)) ->
  (f: ((x: nat & u x) -> Type0)) ->
  (x: nat) -> (y: u x) -> (z: f (| x, y |)) -> t

let g
  (u: (nat -> Type0))
  (f: ((x: nat & u x) -> Type0))
  (x: nat)
  (y: u x)
: Tot Type0
= f (| x, y |)

noeq type t = | T :
  (u: (nat -> Type0)) ->
  (f: ((x: nat & u x) -> Type0)) ->
  (x: nat) -> (y: u x) -> (z: g u f x y) -> t

noeq type g
  (u: (nat -> Type0))
  (f: ((x: nat & u x) -> Type0))
  (x: nat)
  (y: u x)
= | G : (r: (f (| x, y |))) -> g u f x y

All these work with the __temp_no_proj flag.

Here's another example:
noeq
type monad (m:Type0 -> Type0) : Type = {
  bind   : #a:_ -> #b:_ -> m a -> (a -> m b) -> m b;
  assoc  : #a:_ -> #b:_ -> #c:_ -> x:m a -> f:(a -> m b) -> g:(b -> m c) ->
			 squash (bind (bind x f) g == bind x (fun y -> bind (f y) g));
}

We discussed this today on the call. I will try making these projectors typecheck via a tactic, and if that works smoothly then see if we can make that happen by default everywhere (which involves some bootstrapping to do).
Regarding the suggestions by Aseem above (Oct 26 2018), I agree with all of them. And further, I think this one:

When typechecking an auto-generated projectors fails, we should tell the programmer that the error is in auto-generated projectors (similar to how we do for hasEq failures).

Would be useful for the declarations on interfaces that get interleaved into an implementation file. When one of those fails (for whatever reason), it's usually hard to realize. The ranges can tell you, but sometimes they are wrong and point to the .fst. Maybe a generic wrapper Sig_loc : string -> sigelt -> sigelt for sigelts would do, where typechecking would ignore the wrapper, except that if the sigelt raises an error, it would prefix it with the string (or similar), so the errors would be more like in generated projector t.foo: (old error), in declaration interleaved from interface: (old error).

A simple tactic seems enough to make these work. In fact we could also generate the entire projector via tactics.
let tcproj () : Tac unit =
  destruct (nth_binder (-1));
  ignore (intros ());
  rewrite (nth_binder (-1));
  compute ();
  ()

let proj_f2 (ee:u)
  : (i:int -> Tot (y:((proj_k ee) i){w (proj_x ee)}))
  by (tcproj ())
  = match ee with | U k x f -> f

noeq
type monad (m:Type0 -> Type0) : Type = {
  bind   : #a:_ -> #b:_ -> m a -> (a -> m b) -> m b;
  assoc  : #a:_ -> #b:_ -> #c:_ -> x:m a -> f:(a -> m b) -> g:(b -> m c) ->
			 squash (bind (bind x f) g == bind x (fun y -> bind (f y) g));
}

let proj_bind #m (ee:monad m)
  : #a:_ -> #b:_ -> m a -> (a -> m b) -> m b
  = match ee with | Mkmonad bind assoc -> bind

let proj_assoc #m (ee:monad m)
  : #a:_ -> #b:_ -> #c:_ -> x:m a -> f:(a -> m b) -> g:(b -> m c) ->
			 squash ((proj_bind ee) ((proj_bind ee) x f) g == (proj_bind ee) x (fun y -> (proj_bind ee) (f y) g))
  by (tcproj ())
  = match ee with | Mkmonad bind assoc -> assoc

Regarding bootstrapping I think we can just write this tactic internally, and call it in the typechecker when checking a projector, no need to have anything fancy with a snapshot of fstartaclib or similar. I'll look more into it and prepare a PR (I've observed some other weirdnesses with projectors that I should diagnose and fix.)

Here's another example from Fangyi Zhou on zulip.
module Test
open FStar.Ghost
noeq type t = {
  x: erased int;
  y: erased (y:int{y>= reveal x});
  y_: (y_:int{y_ = reveal y});
}

@Kachoc hit this today. Notably his example worked, but typechecking the projector incurred in a huge intermediate term taking >30mins to even get a subtyping predicate from the unifier. A variant of the tactic above made it work instantly, posting the tactic here for posterity. I'd like to make this work soon.
module Proj

open FStar.Tactics

let mkproj (n:pos) : Tac unit =
  let r = t_destruct (nth_binder (-1)) in
  match r with
  | [(cons, arity)] -> begin
    if (n > arity) then
      fail "bad arity";
    let bs = repeatn (arity+1) (fun () -> intro ()) in
    //dump "1";
    rewrite (nth_binder (-1));
    //dump "2";
    norm [iota; delta_only ["Proj.proj_bind"]];
    //dump "3";
    match List.Tot.nth bs (n-1) with
    | None -> fail "impossible, but I should be able to prove it"
    | Some b -> exact b
    end
  | _ ->
    fail "no"

#set-options "--__temp_no_proj Proj"

noeq
type monad (m:Type0 -> Type0) : Type = {
  bind   : #a:_ -> #b:_ -> m a -> (a -> m b) -> m b;
  assoc  : #a:_ -> #b:_ -> #c:_ -> x:m a -> f:(a -> m b) -> g:(b -> m c) ->
			 squash (bind (bind x f) g == bind x (fun y -> bind (f y) g));
}

#set-options (* Look mom, *)"--no_smt" (* ! *)

let proj_bind #m (r : monad m)
  : #a:_ -> #b:_ -> m a -> (a -> m b) -> m b
  = _ by (mkproj 1)

let proj_assoc #m (r : monad m)
  : #a:_ -> #b:_ -> #c:_ -> x:m a -> f:(a -> m b) -> g:(b -> m c) ->
	 squash ((proj_bind r) ((proj_bind r) x f) g == (proj_bind r) x (fun y -> (proj_bind r) (f y) g))
  = _ by (mkproj 2)

One thing to fix is the need for the delta_only which needs to know the relevant projectors to unfold. Using delta here works fine, but not so in a potentially huge type.

I went ahead and implemented most of this in a splice tactic. It's a little monstruous, but does the job. @Kachoc this works for your type too (at least for my slightly modified version). If you tweak this a bit to give proper names and not clash with itself it will probably be quite useful (currently it names everything proj_i, so they clash). I could help if needed.
module Proj

open FStar.Tactics

let mkproj (unf:list string) (np:nat) (i:nat) : Tac unit =
  let _ = ignore (repeatn (np+1) intro) in
  let b = nth_binder (-1) in
  let r = t_destruct b in
  dump "GGG";
  // FIXME: if this is just t_destrcut (nth_binder (-1)), this tactic
  // fails when loaded from another file... ????
  match r with
  | [(cons, arity)] -> begin
    if (i >= arity) then
      fail "bad index";
    let bs = repeatn (arity+1) (fun () -> intro ()) in
    dump "1";
    rewrite (nth_binder (-1));
    dump "2";
    norm [iota; delta_only unf];
    dump "3";
    match List.Tot.nth bs i with
    | None -> fail "impossible, but I should be able to prove it"
    | Some b -> exact b
    end
  | _ ->
    fail "no"

exception NotFound

let subst_map (ss : list (bv * fv)) (r:term) (t : term) : Tac term =
  dump ("before = " ^ term_to_string t);
  let t = fold_left (fun t (x, fv) -> subst x (mk_e_app (Tv_FVar fv) [r]) t) t ss in
  dump ("after  = " ^ term_to_string t);
  t

let rec embed_list_string (e : list string) : Tac term =
  match e with
  | [] -> `Nil
  | s::ss ->
    (`(`#(Tv_Const (C_String s))) :: (`#(embed_list_string ss)))

let mk_proj_decl (tyqn:name) ctorname univs indices (params:list binder) (idx:nat) (fieldty : typ)
                 (unfold_names : list string)
                 (smap : list (bv & fv))
: Tac (sigelt & fv)
=
  let np = List.Tot.length params in
  let ni = List.Tot.length indices in
  let tyfv = pack_fv tyqn in
  let fv = pack_fv (cur_module () @ ["proj_" ^ string_of_int idx]) in
  let rty : binder = fresh_binder (mk_e_app (pack (Tv_FVar tyfv))
                                   (map binder_to_term (params @ indices)))
  in
  let projty = mk_tot_arr (List.Tot.map (binder_set_qual Q_Implicit) params
                           @ List.Tot.map (binder_set_qual Q_Implicit) indices
                           @ [rty])
                          (subst_map smap (binder_to_term rty) fieldty)
  in
  let sv : sigelt_view =
    Sg_Let false fv univs projty (`(_ by (mkproj (`#(embed_list_string unfold_names)) (`#(Tv_Const (C_Int (np+ni)))) (`#(Tv_Const (C_Int idx))))))
  in
  dump ("returning : " ^ term_to_string (quote sv));
  (pack_sigelt sv, fv)

let mk_projs (tyname:string) : Tac decls =
  let tyqn = explode_qn tyname in
  match lookup_typ (top_env ()) tyqn with
  | None ->
    raise NotFound
  | Some se ->
    match inspect_sigelt se with
    | Sg_Inductive ctorname univs params typ ctors ->
      if (List.Tot.length ctors <> 1) then
        fail "expected a record";
      let indices = fst (collect_arr_bs typ) in
      let [ctor] = ctors in
      let (fields, _) = collect_arr_bs (snd ctor) in
      let (decls, _, _, _) =
        fold_left (fun (decls, smap, unfold_names, idx) field ->
                     let (d, fv) = mk_proj_decl tyqn ctorname univs indices params idx (type_of_binder field) unfold_names smap in
                     (d::decls,
                      (bv_of_binder field,fv)::smap,
                      (implode_qn (inspect_fv fv))::unfold_names,
                      idx+1))
                  ([], [], [], 0)
                  fields
      in
      List.Tot.rev decls
    | _ ->
      fail "not an inductive"

#set-options "--__temp_no_proj Proj"

noeq
type monad (rr : Type) (m:Type0 -> Type0) : Type = {
  ret    : #a:_ -> a -> m a;
  bind   : #a:_ -> #b:_ -> m a -> (a -> m b) -> m b;
  assoc  : #a:_ -> #b:_ -> #c:_ -> x:m a -> f:(a -> m b) -> g:(b -> m c) ->
			 squash (bind (bind x f) g == bind x (fun y -> bind (f y) g));
  id1    : #a:_ -> #b:_ -> x:a -> f:(a -> m b) ->
           squash (bind (ret x) f == f x);
  id2    : #a:_ -> c:m a ->
           squash (bind c ret == c);
}

#set-options (* Look mom, *)"--no_smt" (* ! *)

//%splice[proj_0; proj_1; proj_2; proj_3; proj_4] (mk_projs (`%monad))

#set-options "--print_bound_var_types --print_implicits"

type test : int -> Type =
 | MakeThisGuy :
  i:int ->
  x:int ->
  test i

%splice[] (mk_projs (`%test))

FYI, rather than the tactic-based solution, now that we also have dependent pattern matching, here's another possible fix.
Taking the example from #2460
module Bug2460

open FStar.List.Tot

#push-options "--print_implicits --print_effect_args"

let rec for_all_range (n: nat) (f: (i: nat{i < n}) -> bool) =
  if n = 0 then
    true
  else (
    f (n - 1) && for_all_range (n - 1) f
  )

#push-options "--__temp_no_proj Bug2460" //use YourModuleName instead of Test9
noeq
type test_t =
  | MkTestT:
    s: list nat ->
    lem: squash (for_all_range (length s) (fun (i: nat{i < length s}) -> true)) ->
    test_t
#pop-options

let test_t_s (x:test_t)
  : list nat
  = match x with
    | MkTestT s _  -> s

//This is what we generate and it fails
//because at higher order, the SMT solver fails to prove that
//     (for_all_range (length (test_t_s x)) (fun (i: nat{i < length (test_t_s x)}) -> true))
// ==  (for_all_range (length this_s) (fun (i: nat{i < length (test_t_s x)}) -> true))
// Even though these things are extensionally equal
[@@expect_failure]
let test_t_lem (x:test_t)
  : squash (for_all_range (length (test_t_s x)) (fun (i: nat{i < length (test_t_s x)}) -> true))
  = let MkTestT this_s lem = x in lem

// But, recent additions to F* make it possible to typecheck this
// The `returns` clause makes this a "dependent pattern match",
// requiring the branch to have the type
//    (squash (for_all_range (length (test_t_s (MkTestT s lem))) (fun (i: nat{i < length (test_t_s x)}) -> true)))
// and this does unify with the expected type without needing SMT intervention
let test_t_lem2 (x:test_t)
  : Tot (squash (for_all_range (length (test_t_s x)) (fun (i: nat{i < length (test_t_s x)}) -> true)))
  = match x returns (squash (for_all_range (length (test_t_s x)) (fun (i: nat{i < length (test_t_s x)}) -> true)))
    with
    | MkTestT s lem -> lem

@aseemr @mtzguido and also fyi @jaylorch

And, btw, even this works
let test_t_lem2 (x:test_t)
  : Tot (squash (for_all_range (length (test_t_s x)) (fun (i: nat{i < length (test_t_s x)}) -> true)))
  = match x returns _
    with
    | MkTestT s lem -> lem


typo: rersion -> region

@cynthn - Would you review the proposed changes? Thanks.

@axayjo is this word supposed to be "version" or "region"? Thanks :)

@cynthn Version :)

@nschonni Hey, thanks for contributing! The word in question is actually supposed to be "version". If you want to make the change, I can get it merged. But, if you don't want to update your PR I can make the fix separately and have this PR closed. Just let me know what you prefer :)

I probably won't get to it for a few hours at least. You should be able to push changes to this PR if you want, or just make it on your internal repo if that's easier

@GitHubber17 : Thanks for your contribution! The author, @axayjo, has been notified to review your proposed change.

Adjusted text per feedback from @cynthn.


Allow custom titles for boilerplate sections

CSS 2 needs custom titles for its boilerplate sections (and potentially some way to add custom section numbers), given we need Appendix B: Bibliography etc.

Note that #1677 could potentially complicate this.


Failing to download covers...again

Hi,
I'm using the 0.10.7 plugin in Calibre [Ver: 2.51.0] and it just quit downloading covers on me a couple days ago.
Log is as follows:
Running identify query with parameters:
{u'identifiers': {u'comicvine': u'356772', u'comicvine-volume': u'35835', u'isbn': u'Issue ID 356772'}, u'timeout': 30, u'title': u'Uncanny X-Force #31: Final Execution', u'authors': [u'Alan Fine', u'Axel Alonso', u'Cory Petit', u'Dan Buckley', u'Dean White', u'Frank Martin Jr.', u'Jared K. Fletcher', u'Jerome Ope\xf1a', u'Joe Quesada', u'Jordan D. White', u'Nick Lowe', u'Phil Noto', u'Rick Remender']}
Using plugins: Comicvine
The log from individual plugins is below
****************************** Comicvine ******************************
Request extra headers: [('User-agent', 'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)')]
Found 1 results
Downloading from Comicvine took 7.6779999733

Title               : Uncanny X-Force #31: Final Execution
Author(s)           : Alan Fine & Axel Alonso & Cory Petit & Dan Buckley & Dean White & Frank Martin Jr. & Jared K. Fletcher & Jerome Opeña & Joe Quesada & Jordan D. White & Nick Lowe & Phil Noto & Rick Remender
Publisher           : Marvel
Series              : Uncanny X-Force #31
Published           : 2012-09-12T04:00:00+00:00
Identifiers         : comicvine-volume:35835, comicvine:356772
Comments            : The FINAL EXECUTION kicks into high gear. What is left of X-Force go up against the new Brotherhood of Evil Mutants for the last time.
Looking up Issue(356772)
Adding Issue(356772) to queue
Added Issue(Uncanny X-Force #31: Final Execution) to queue

The identify phase took 7.84 seconds
The longest time (7.678000) was taken by: Comicvine
Merging results from different sources and finding earliest  publication dates from the worldcat.org service
We have 1 merged results, merging took: 0.00 seconds
Starting cover download for: Uncanny X-Force #31: Final Execution
Query: Uncanny X-Force #31: Final Execution [u'Alan Fine', u'Axel Alonso', u'Cory Petit', u'Dan Buckley', u'Dean White', u'Frank Martin Jr.', u'Jared K. Fletcher', u'Jerome Ope\xf1a', u'Joe Quesada', u'Jordan D. White', u'Nick Lowe', u'Phil Noto', u'Rick Remender'] {u'comicvine': u'356772', u'comicvine-volume': u'35835'}
****************************** Comicvine Covers ******************************
Request extra headers: [('User-agent', 'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)')]
Failed to download valid cover
Took 3.53800010681 seconds
Downloading cover from: http://static.comicvine.comhttp://static.comicvine.com/uploads/scale_large/7/71975/2586319-prev_img.jpeg
Failed to download cover from: http://static.comicvine.comhttp://static.comicvine.com/uploads/scale_large/7/71975/2586319-prev_img.jpeg
Traceback (most recent call last):
File "calibre_plugins.comicvine.source", line 173, in download_cover
File "site-packages\mechanize-0.2.5-py2.7.egg\mechanize_mechanize.py", line 199, in open_novisit
File "site-packages\mechanize-0.2.5-py2.7.egg\mechanize_mechanize.py", line 230, in _mech_open
File "site-packages\mechanize-0.2.5-py2.7.egg\mechanize_opener.py", line 193, in open
File "site-packages\mechanize-0.2.5-py2.7.egg\mechanize_urllib2_fork.py", line 344, in _open
File "site-packages\mechanize-0.2.5-py2.7.egg\mechanize_urllib2_fork.py", line 332, in _call_chain
File "site-packages\mechanize-0.2.5-py2.7.egg\mechanize_urllib2_fork.py", line 1142, in http_open
File "site-packages\mechanize-0.2.5-py2.7.egg\mechanize_urllib2_fork.py", line 1118, in do_open
URLError: <urlopen error [Errno 11001] getaddrinfo failed>
Downloading cover from: http://static.comicvine.comhttp://static.comicvine.com/uploads/scale_medium/7/71975/2586319-prev_img.jpeg
Failed to download cover from: http://static.comicvine.comhttp://static.comicvine.com/uploads/scale_medium/7/71975/2586319-prev_img.jpeg
Traceback (most recent call last):
File "calibre_plugins.comicvine.source", line 173, in download_cover
File "site-packages\mechanize-0.2.5-py2.7.egg\mechanize_mechanize.py", line 199, in open_novisit
File "site-packages\mechanize-0.2.5-py2.7.egg\mechanize_mechanize.py", line 230, in _mech_open
File "site-packages\mechanize-0.2.5-py2.7.egg\mechanize_opener.py", line 193, in open
File "site-packages\mechanize-0.2.5-py2.7.egg\mechanize_urllib2_fork.py", line 344, in _open
File "site-packages\mechanize-0.2.5-py2.7.egg\mechanize_urllib2_fork.py", line 332, in _call_chain
File "site-packages\mechanize-0.2.5-py2.7.egg\mechanize_urllib2_fork.py", line 1142, in http_open
File "site-packages\mechanize-0.2.5-py2.7.egg\mechanize_urllib2_fork.py", line 1118, in do_open
URLError: <urlopen error [Errno 11001] getaddrinfo failed>
Downloading cover from: http://static.comicvine.comhttp://static.comicvine.com/uploads/scale_small/7/71975/2586319-prev_img.jpeg
Failed to download cover from: http://static.comicvine.comhttp://static.comicvine.com/uploads/scale_small/7/71975/2586319-prev_img.jpeg
Traceback (most recent call last):
File "calibre_plugins.comicvine.source", line 173, in download_cover
File "site-packages\mechanize-0.2.5-py2.7.egg\mechanize_mechanize.py", line 199, in open_novisit
File "site-packages\mechanize-0.2.5-py2.7.egg\mechanize_mechanize.py", line 230, in _mech_open
File "site-packages\mechanize-0.2.5-py2.7.egg\mechanize_opener.py", line 193, in open
File "site-packages\mechanize-0.2.5-py2.7.egg\mechanize_urllib2_fork.py", line 344, in _open
File "site-packages\mechanize-0.2.5-py2.7.egg\mechanize_urllib2_fork.py", line 332, in _call_chain
File "site-packages\mechanize-0.2.5-py2.7.egg\mechanize_urllib2_fork.py", line 1142, in http_open
File "site-packages\mechanize-0.2.5-py2.7.egg\mechanize_urllib2_fork.py", line 1118, in do_open
URLError: <urlopen error [Errno 11001] getaddrinfo failed>

Please investigate

It looks like the request is doubling the url again:
Failed to download cover from: http://static.comicvine.comhttp://static.comicvine.com/uploads/scale_small/7/71975/2586319-prev_img.jpeg
I don't remember what the resolution was, I think eventually ComicVine reverted whatever it was they did and it just started working again.

Something in this function, maybe?
def download_cover(self, log, result_queue, abort,
title=None, authors=None, identifiers=None,
timeout=30, get_best_cover=False):
if identifiers and 'comicvine' in identifiers:
for url in utils.cover_urls(identifiers['comicvine'], get_best_cover):
url = 'http://static.comicvine.com' + url
browser = self.browser
log('Downloading cover from:', url)
try:
cdata = browser.open_novisit(url, timeout=timeout).read()
result_queue.put((self, cdata))
except:
log.exception('Failed to download cover from:', url)

Yeah, here's the commit from the last time:
                  timeout=30, get_best_cover=False):
 if identifiers and 'comicvine' in identifiers:
   for url in utils.cover_urls(identifiers['comicvine'], get_best_cover):

   url = 'http://static.comicvine.com' + url
   browser = self.browser
   log('Downloading cover from:', url)
   try:

So in theory if we were to comment out that "+" line it might start working again.

Just,... can't,.... stop,... commenting,....  :)
So, yes.  Go to C:\Users<username>\AppData\Roaming\calibre\plugins, extract source.py, open in a text editor and comment the url = 'http://static.comicvine.com' + url line (use a # at the beginning), save and import it in to the zip, overwriting the existing file.  When they revert this change again, you'll just need to go back and revert the comment and you'll be good again.
Until they revert it again.  Lather, rinse, repeat, wipe hands on pants.

That worked Mikey!!!
Thank you so much! I totally owe you a beer...


Support environment variables for paths in .code-workspace files

Issue Type
Feature Request
Description
I'm attempting to use { "path": "${env:APPDATA}/Code/User" } in my workspace to be able to easily open settings.json and keybindings.json as raw files (the custom panels take up too much space and I rarely need them) and it appears variables are not substituted when loading the workspace file.
VS Code Info
VS Code version: Code 1.20.1 (f88bbf9137d24d36d968ea6b2911786bfe103002, 2018-02-13T15:34:36.336Z)
OS version: Windows_NT x64 10.0.16299

Related: It's also very hard or impossible to ignore files in a specific folder from the workspace settings. Consider the following scenario:
I have a directory structure
root/
  engine/
  game/
    log/
      <various log files>

I configure a multi-root workspace that encompasses both the engine and the game. When I'm working solely on the game I want to see the log files. When I'm working on the engine I want to see the game, but I don't want to see the log files.
I've tried various patterns in settings.files.exclude in the workspace file and I can't find a way to get it to apply only to the game folder.
"game/log": true,
"../game/log": true,
"${workspaceFolder:game}/log": true,

I thought I was doing it wrong.
I've also been frustrated with the settings panels, which might be useful, although extensions are not sorted, and the resizer between the panels is too thin and hard to find, and panels wont resize enough, and search right panel should be separate,  but aside from that, $appdata and $username are sometimes helpful with multiple users.
Workarounds...
1) Junctions,  C:\cod => %appdata%\Code\
2) Move everything to one path, C:\cod
Maybe this example will help escalate:
Files:
C:\Users\FOO\TheAnswer\42.txt
C:\Users\BAR\TheAnswer\muhuhuh.txt
C:\Users\BAR\Desktop\nothingtoseehere.code-workspace.lnk
Workspace:
C:\nothingtoseehere.code-workspace
Workspace contains:
{   "folders": [

{ "path": "C:\\Users\\BAR\\TheAnswer" },
{ "path": "C:\\Users\\${UserName}\\TheAnswer" }

] }
Problem
You are Bar.  Foo wanted to share.  Foo's last words were his password.  doh.
Warning: Recreation may involve Vogons.

Another use case: I am developing an app which writes to files in the user's home directory ~/myapp/cache/.... Thus I want to watch this cache folder via the workspace path.
{
   "name": "cache",
   "path": "${env:USERPROFILE}\\.powowshell\\cache"
}

Neither does:
{
   "name": "cache",
   "path": "~/.powowshell/cache"
}

@bpasero Any milestones for this feature request?
I am also having trouble creating a template workspace for our company dev team, and I currently don't have any workarounds.
For example, in our case given envioronment variable points to a disk where we have our common (company guidelines) folder structure and folders I would like to add to template workspace.

@bpasero As this is one of the most "blocking" basic features... I am thinking of investing my time into a fix for this issue. It seems that at least expanding process environment variables would cover a lot of use cases. Has this issue completely fall off the radar?

I can currently not provide the time to be a mentor of this feature through a PR, sorry. I would search for issues labeled "help wanted" to make sure you get team members that are willing to review the change.
Since this issue is on the "Backlog" it may be considered in the future, but our "Backlog" is also very large.

Hi,
I also like and find useful this feature.
I also suggest to allow to specify different paths according with the OS.
For example:
{
   "name": "cache",
   "Win32": {
       "path": "${env:USERPROFILE}\\work\\myproject"
    },
    "Linux": {
       "path": "${env:HOME}/open-source/myproject"
     }
}

This would cover basically any portability issue between OS and environments.
The split of config according with current OS has been proposed in #117661.
Finally also #87889 would be of great help.
Regards.

I'd be really cool to be able to set linting configurations using these substitutions. Be it ${env:XXX} or ${workspaceFolder:XXX}.
I would want to do something like:
{
  "folders": [
    {"path": "common"},
    {"path": "app1"},
    {"path": "app2"}
  ],
  "settings": {
    "python.linting.pylintArgs": ["--rcfile=${workspaceFolder:common}/pylint.cfg"]
  }
}

I also want this feature. I have two clones of the same (large, multi-project) repo, and I want to use the same workspace file to filter out folders I don't care about. My repo has a setup script that sets the repo root environment variable, so if this feature worked, I'd substitute the variable for the hardcoded path.

It is a very interesting feature. $HOME or ${env.UserProfile} will help a lot.

I have resort to not checking in the workspaces bug checking in templates and having the user run a script that creates rehomed versions of the workspaces from the templates.
#!/usr/bin/env python

import os
import shutil

# Standardized Python Version
PYTHON_VERSION = "python3.8"

THIS_DIR = os.path.abspath(os.path.dirname(__file__))

REPOSITORY_FOLDER = os.path.abspath(os.path.join(THIS_DIR, ".."))
AUTOMATION_FOLDER = os.path.abspath(os.path.join(REPOSITORY_FOLDER, "automation"))

VENV_FOLDER = os.path.join(AUTOMATION_FOLDER, ".venv")
VENV_BIN_FOLDER = os.path.join(VENV_FOLDER, "bin")
VENV_SITEPACKAGES_FOLDER = os.path.join(VENV_FOLDER, "lib", PYTHON_VERSION, "site-packages")

WORKSPACES_FOLDER = os.path.join(REPOSITORY_FOLDER, "workspaces")

def replace_macros(template_line):
    """
        Perform a simple replacement any macros found in the template line passed to us.
    """

    home = os.path.expanduser("~")

    filled_line = template_line

    filled_line = filled_line.replace(r"${TMPLT:AUTOMATION_FOLDER}", AUTOMATION_FOLDER)
    filled_line = filled_line.replace(r"${TMPLT:HOME}", home)
    filled_line = filled_line.replace(r"${TMPLT:PYTHON_VERSION}", PYTHON_VERSION)
    filled_line = filled_line.replace(r"${TMPLT:REPOSITORY_FOLDER}", REPOSITORY_FOLDER)
    filled_line = filled_line.replace(r"${TMPLT:VENV_FOLDER}", VENV_FOLDER)
    filled_line = filled_line.replace(r"${TMPLT:VENV_BIN_FOLDER}", VENV_BIN_FOLDER)
    filled_line = filled_line.replace(r"${TMPLT:VENV_SITEPACKAGES_FOLDER}", VENV_SITEPACKAGES_FOLDER)
    filled_line = filled_line.replace(r"${TMPLT:WORKSPACES_FOLDER}", WORKSPACES_FOLDER)

    return filled_line

def generate_devemopment_env_file():

    cache_dir = os.path.join(AUTOMATION_FOLDER, ".cache")
    if not os.path.exists(cache_dir):
        os.makedirs(cache_dir)

    dev_env_file = os.path.join(cache_dir, "development.evn")
    with open(dev_env_file, 'w') as envf:
        envf.write("PYTHONPATH={}/automation/python3/testroots:{}/automation/python3/packages:{}".format(
            REPOSITORY_FOLDER,
            REPOSITORY_FOLDER,
            VENV_SITEPACKAGES_FOLDER
        ))
        envf.write(os.linesep)

        #TODO: Add the setting of any other 'development.env' environment variables here.

        envf.write(os.linesep)

    return

def generate_vscode_workspace_files():
    # Go through all of the VSCODE workspace templates and generate the 'code-workspace' files homed to the
    # location of this cloned repository

    print("Scanning workspaces folder:")
    print(WORKSPACES_FOLDER)
    print("")

    workspace_template_files = []

    for root, dirs, files in os.walk(WORKSPACES_FOLDER):
        for np in files:
            np_full = os.path.join(root, np)
            if os.path.isfile(np_full):
                _, npext = os.path.splitext(np_full)
                if npext == ".template":
                    workspace_template_files.append(np_full)

    for template_file in workspace_template_files:

        template_file_base, template_file_ext = os.path.splitext(os.path.basename(template_file))
        template_file_dir = os.path.dirname(template_file)
        workspace_file = os.path.join(template_file_dir, "{}.code-workspace".format(template_file_base))

        print("Processing template: {}".format(template_file))

        with open(template_file, 'r') as tf:
            template_lines = tf.read().splitlines(True)

            with open(workspace_file, 'w') as wf:
                print("Generating code-workspace: {}".format(workspace_file))
                for tline in template_lines:
                    fline = replace_macros(tline)
                    wf.write(fline)

def rehome_repository_main():

    generate_devemopment_env_file()
    generate_vscode_workspace_files()

    return

if __name__ == "__main__":
    rehome_repository_main()

Thanks for sharing @myronsonos. But I still think that native support is very welcome.

HI @bpasero. Any milestones for this feature request?

Hi indeed that would be nice to implement that.

It would make sense if environment variables were to be referenced in the same way as in Debugging and Task configuration files, i.e.
${env:USERNAME}
https://code.visualstudio.com/docs/editor/variables-reference#_environment-variables

Thanks for sharing @myronsonos. But I still think that native support is very welcome.

I am all for the native support.  But some of us don't want to wait. :)

Example
{
	"settings": {
		"ruff.interpreter": [
			"${platform:home}\\virtualenvs\\kivy_venv\\Scripts\\python.exe"
		],
	}
}

should be configurable to support a profile variable defined by VSCode (so it is OS-dependent using USERPROFILE or HOME) as opposed to ${env:USERPROFILE} which is Windows-only.

3.5 years after this feature put into backlog. Any progress yet 😢

Related on Stack Overflow: VSCode code-workspace file - use environment variables for project home path

More than 4 years since issue created. Is any progress here?

It would be great if we could use not only environment variables but also setting values.
{
  "folders": [
    { "path": "${config:super-project-base-path}/test", "name": "test"}
  ]
}

Like others, I ended up here because I need to use ${env:localappdata} in a workspace folder, which for me cascades in that folder being used in "python.defaultInterpreterPath".  Today, this is the 114th-most upvoted open issue in vscode.

Was this working at some point and got disabled?

This would help a lot if my team could reference workspace folders via environment variables.

Mr Microsoft, using env vars for variables could help migrate between linux and windows 😉 (or the other way)


code version

could you release the code version only use python?

In addition to Python code, this code repository also contains a basic library composed of some C++ codes. These libraries are designed to achieve efficient nearest neighbor search, downsampling, and other functions, and have been widely adopted by many open-source repositories for point cloud analysis.
I'm sorry that I don't get the intention of your question. Are you encountering difficulties in compiling these libraries?

I have some problems so that I cannot understand this network architecture, so I do not know how to change it in other dataset.  I do not know how to change .sh file and openpoints. May I get your email to describe my problem?    maybe so ridiculous..


A malformed type literal passed to the -is or -as operator is quietly ignored at the prompt

Note that the problem only surfaces at the interactive prompt, whether or not PSReadLine is loaded.
In a script / script block, an error occurs, as expected.
This is a regression from v6.2.3 / WinPS.
Steps to reproduce
Submit at the prompt:
'foo' -is [string
'foo' -as [string

Expected behavior
A statement-terminating error with message:
Missing ] at end of attribute or type literal.

Actual behavior
Both commands fail quietly, and $? even contains $true.
Environment data
PowerShell Core 7.0.0-rc.1

Present in preview6 too, so something happened somewhere before that. 🤔

I can not repo with latest build.

It works fine in 6.2.3 so it's definitely a regression 7.

Works well in 7.0 Preview4.

It's a bug with the concise error view.  The exception is still in $Error and if you retrieve it the formatting is blank.  @iSazonov if it doesn't repro with the latest build, it was probably fixed with #11415.

@mklement0 Can you confirm with nightly build that the issue was fixed?

Thanks for diagnosing, @SeeminglyScience.
Indeed, @iSazonov, the problem has been fixed in the latest daily build.


MdDialogRef.afterClosed() emits before dialog refocuses on an underlying element

Bug, feature request, or proposal:
The MdDialogRef.afterClosed() observable emits an event before the MdDialogContainer has restored focus to an underlying element. This can cause problems if subscriptions to MdDialogRef.afterClosed() involve triggering focus events themselves.
This happens because MdDialogContainer waits for the NgZone's onMicrotaskEmpty to emit before executing its refocus operation.
What is the expected behavior?
If this is the intended behaviour then I believe it should be made explicit in the documentation. Otherwise, the MdDialogRef.afterClosed() observable should only emit once all dialog cleanup has been completed.
Currently, the documentation says:

afterClosed - Gets an observable that is notified when the dialog is finished closing.

What is the current behavior?
See above
What are the steps to reproduce?
See this plunker. AppComponent wants to focus its input element after the dialog has closed but this event is overwritten by MdDialogContainer, which restores focus to the button element.
What is the use-case or motivation for changing an existing behavior?
The contract with MdDialogRef.afterClosed() should be clear.
I am using MdDialog within the context of an application that refocuses elements in a non-trivial way so it would not be possible to control this behaviour through tabindex attributes.
Which versions of Angular, Material, OS, browsers are affected?
Tested on Angular 4.0.1/Material head f40296e/Windows 7/Chrome 57
Is there anything else we should know?
The workaround I am using is to perform the focus event inside another NgZone.onMicrotaskEmpty subscription:
this.modal.afterClosed().subscribe(() => {
    this.ngZone.onMicrotaskEmpty.first().subscribe(() => {
        // Focus operation here
    });
});

I have a related(?) problem when I have two dialogs open (bad UX maybe but it's not my design), and I try to close one as soon as the other is closed.
onClickRemove() {
  this.dialog.open(/* open a confirmation dialog */).afterClosed().subscribe((result?: boolean) => {
    if (result) { // user clicked ok
      this.dialogRef.close(); // close original dialog
    }
  });
}

When this runs, the page will get stuck in a weird state where the first .cdk-global-overlay-wrapper element is still in the DOM. The dialog is invisible, but the underlying page ignores pointer events and doesn't scroll.
The onMicrotaskEmpty thing worked for me. Thanks!

Sounds weird @dbandstra, I'll take a look. Also, if you want to close all your open dialogs, you can use MdDialog.closeAll.

I wasn't able to reproduce your issue @dbandstra. Here's what I tried:
openDialogs() {
  this.dialogRef = this.dialog.open(JazzDialog, {
    data: { message: 'dialog 1' }
  });

  this.otherDialogRef = this.dialog.open(JazzDialog, {
    data: { message: 'dialog 2' }
  });

  this.otherDialogRef.afterClosed().subscribe(() => {
    this.dialogRef.close();
  });
}

Can you post a Plunkr that shows it. Also I'll close this issue since it should've been fixed by https://github.com/angular/material2/pull/3875.

Hi @crisbeto, I think the visibly glitchy behaviour was a result of some styles I had overridden.
Anyway, I made a Plunkr: http://plnkr.co/edit/0tU7JMSnYYO9Urm9xryB
You can see the 'problem' by inspecting the DOM. The .cdk-overlay-container element still has stuff in it after you click 'Open dialog' -> 'Remove' -> 'Yes'. I guess it's up to you guys whether this is still an issue.
Also, I used NoopAnimationsModule. The problem doesn't occur with BrowserAnimationsModule because the dialogs take time to close.

Thanks, that's definitely a bug. I'll look into it.

The issue seems to be with the animations module @dbandstra. You can work around it either by using closeAll to close all of the dialogs, or wrapping the final dialogRef.close call in a setTimeout.


Resolve twitter handles

Purpose
Currently, when editing social fields, users are able to enter Twitter handles containing '@' (e.g. '@example'). When someone then clicks on their Twitter handle from their profile, they are sent to their Twitter page (e.g. https://twitter.com/@example). This is not a problem when the handle contains only one leading '@', since pages like the above work correctly on Twitter.
The problem occurs when a user enters a Twitter handle containing more than one leading '@' (e.g. '@@@example'). In this case, this will send whoever clicks on the handle to a bad address like https://twitter.com/@@@example.
Changes
This PR removes all instances of leading '@' from user-entered Twitter handles. Twitter profiles accessed by both the user profile and the search page will lead to the correct Twitter page without any leading '@' (e.g. https://twitter.com/example for a user-entered handle of '@@@example').
This PR also includes a migration script that any leading '@' from existing user Twitter handles, and a test of this migration script.
[OSF-3617]

via Jira ticket:

Twitter handles do not always resolve properly. Example: osf.io/@BrianNosek
[...]
They will resolve properly if you include the extra @ symbol.
Example: osf.io/@@BrianNosek [...]
I believe at some point, we required the @ symbol in user profile.

I'm unable to reproduce this issue as described. On develop, twitter handles can be entered with or without an @ and resolved correctly when clicked. Two or more @'s failed to resolve.
Regardless, these changes are good going forward, but it doesn't solve the "existing users included @@ at some point in the past" issue. A migration script is needed for that. I have a script template locally, here's that but with an appropriate query for this case:
import logging
import sys

from modularodm import Q

from framework.transactions.context import TokuTransaction
from website.app import init_app
from website.models import User
from scripts import utils as script_utils

logger = logging.getLogger(__name__)

def get_targets():
    return User.find(Q('social.twitter', 'contains', '@'))

def migrate(targets, dry_run=True):
    # iterate over targets
    # log things

    if dry_run:
        raise RuntimeError('Dry run, transaction rolled back.')

def main():
    dry_run = False
    if '--dry' in sys.argv:
        dry_run = True
    if not dry_run:
        script_utils.add_file_logger(logger, __file__)
    init_app(set_backends=True, routes=False)
    with TokuTransaction():
        migrate(targets=get_targets(), dry_run=dry_run)

if __name__ == "__main__":
    main()

Also, please follow our standard PR description template

I love that you have tests on your migration. Let's see if we can add in as many crazy tests as we can think of (migrations are very tricky on production data).  Whatever you can think of that might cause problems, go ahead and check to make sure your script does what you would expect it to.

For the issue description, it's nice when we can show screenshots of what changes so QA can test. Could you do a couple of sets of screenshots, one with the current functionality, and one with what will be different with your changes?

Pass done. 🐧

Adding screenshots to clarify before and after behavior!
Before:
I enter a Twitter handle in 'Profile Settings' that I know won't resolve:

I click 'Save'.
I go to my profile, and see that my profile's Twitter handle is exactly the same as I entered it:

When I click on the Twitter link (https://twitter.com/@@@khewtart), I get:

After:
I enter my Twitter handle exactly the same as before:

I click 'Save', go to my profile, and see that my profile's Twitter handle has been resolved:

When I click on this link, it goes to my actual Twitter page, https://twitter.com/khewtart and no 404 error
The above process shows how clicking 'Save' when editing profile settings and social fields automatically resolves Twitter handles. The migration script covers those cases where users have already entered Twitter handles that can't be resolved.

The script fails on unicode characters in twitter handles, but none of the targets that would be hit on prod contain unicode (and I'm fairly certain that Twitter doesn't support non-ascii characters in handles).
Otherwise, LGTM. Dropping this into RTM 👓

Pass finished.

@sloria my original plan was to try to modify socialRules.twitter's regex to disallow @ signs, but I don't have a lot of experience with regex and couldn't get it to work. I'll go back now and try and make the change there instead of in the serializedData function.
Besides that change and moving the migration tests to the scripts/tests, that's all I need to fix correct?

@kch8qx Yep, just those two minor changes, and this should be good to go.

Note: migration script test has been moved to scripts/tests.
To do: use regex to disallow '@' in socialRules.twitter

I'm going to hold off on merging this for now. Updating this many users' profile info will result in many nodes getting updated in elastic search, which will queue many jobs in celery. Let's wait until we have multiple celery queues in production before this is merged.

Punchlist

[ ] back out no-op changes—https://github.com/CenterForOpenScience/osf.io/pull/5400#discussion_r68682014

As I read this PR, we don't need any logic changes after all, we're just doing the migration. Ya?

It's Autumn Cleaning time, so I'm closing this PR. Feel free to re-open when


Asteroid disappears sometimes

Not exactly sure what is going on but sometimes asteroid just disappears while it is falling. I think it might have something to do with the enemy field?

This was caused by asteroids being overwritten by enemies in the function that manages the enemy array. Easy fix


Feature/571 add buyer agent profiles

Changes in this PR

Screenshots of UI changes

Before
After
Next steps

@montychristo we need to catch up ASAP because the image I shared and its relevance to this ticket have not been understood.

@montychristo I have added the following to the test DSI config for you:

https://localhost:3000/support/auth/dfe/callback
https://localhost:3000/support/auth/dfe/signout


Unable to use braces when calling a TagUI Workflow inside another script - use indentation instead of curly braces

I am receiving the error below when I am trying to call another tagui script within the flow where I have a for loop.
ERROR - number of step { does not tally with with }
ERROR - automation aborted due to above

e.g.
child.tag
echo hello

parent.tag
tagui child.tag
for n from 1 to 5
{
     echo n
}

Hi @chialiyun you can use indentations instead of using braces
tagui child.tag
for n from 1 to 5
    echo n

https://tagui.readthedocs.io/en/latest/main_concepts.html#for-loops

hi @ruthtxh
does indentation work if there's multiple steps to be executed in the for loop? e.g.
for n from 1 to 5
   do step 1
   do step 2

do out of loop step 3

@ruthtxh it did not work for me.
here's my code:
child.tag
UrlDict = {}

csv_file = 'file.csv'
load `csv_file` to lines

array_lines = lines.split('\n')

for (n=1; n<array_lines.length; n++)
  items = array_lines[n].split(',')
  country = items[0].trim()
  UrlDict[country] = items[4].trim()

url = UrlDict[p1]
echo `url`

parent.tag
tagui child.tag
echo selected `UrlDict['sg']`

for n from 1 to 4
    echo n

@ruthtxh it did not work for me.
here's my code: child.tag
UrlDict = {}

csv_file = 'file.csv'
load `csv_file` to lines

array_lines = lines.split('\n')

for (n=1; n<array_lines.length; n++)
  items = array_lines[n].split(',')
  country = items[0].trim()
  UrlDict[country] = items[4].trim()

url = UrlDict[p1]
echo `url`

parent.tag
tagui child.tag
echo selected `UrlDict['sg']`

for n from 1 to 4
    echo n

hi chialiyun,
when the csv content contains "\n",

#,firstname,lastname
1,tony,stark
2,bruce,wayne
3,peter,parker
4,"Dr
Steve",Strange
how to do it...?

Closing but please ping back if this is still relevant of an issue.


nginx for using reverse proxy

if using the latest version of matrix-registration with nginx reverse proxy, additionally this should be added to nginx.conf inside the corresponding server to show generation link correctly
    location /api {
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_pass http://localhost:5000;
    }

the base_config should be like:
legacy_mr: false
reg_url: https://matrix.org
reg_page: /register

is this a suggestion for the documentation? or perhaps a suggested modification to the default settings in the base-config? i'm unclear what this issue is about but i'd gladly make changes for whatever it is.

is this a suggestion for the documentation? or perhaps a suggested modification to the default settings in the base-config? i'm unclear what this issue is about but i'd gladly make changes for whatever it is.

I would suggest to add a URL string, for the invite link via an external URL inside the bot. the configuration with the following works, as well, but the invite link looks weird :)
legacy_mr: false
reg_url: http://localhost:5000
reg_page: /register

this gives: http://localhost:5000/register?token=abc
also filed this issue here: https://github.com/ZerataX/matrix-registration/issues/66

i see, so you're referring to an installation of matrix-registration where the /api endpoint isn't publicly accessible, it's only available via local connections? i suppose it makes sense to more completely separate the api endpoint from the registration link, it was obviously written under the assumption that they would be the same.

new ability to write a custom message response means you can construct your own response, and use the reg_page variable to point users to the correct registration page regardless of the reg_url value.


OOM during prune causing state hash conflict on restart

Manual tests to mimic OOM during prune

Forced a panic during prune before database commit in DeleteVersionsRange iavl:

ESC[90m8:18PMESC[0m ESC[32mINFESC[0m Starting MConnection service ESC[36mimpl=ESC[0mMConn{<IP_ADDRESS>:26656} ESC[36mmodule=ESC[0mp2p
ESC[36mpeer=ESC[0m{"id":"4ab030b7fd75ed895c48bcc899b99c17a396736b","ip":"<IP_ADDRESS>","port":26656}
ESC[90m8:18PMESC[0m ESC[32mINFESC[0m Added peer ESC[36mmodule=ESC[0mp2p ESC[36mpeer=ESC[0m{"Data":{},"Logger":{}}
panic: force panic in DeleteVersionRange before commit

goroutine 105 [running]:
github.com/cosmos/iavl.(*MutableTree).DeleteVersionsRange(0x1b8c5a0, 0xc000f22880, 0xc000f228b0)
        github.com/cosmos/iavl@v0.17.3/mutable_tree.go:895 +0x59
github.com/cosmos/iavl.(*MutableTree).DeleteVersions(0xc00017a480, {0xc0016b6700, 0xc000b6d4b0, 0xe0})
        github.com/cosmos/iavl@v0.17.3/mutable_tree.go:880 +0x2ca
github.com/cosmos/cosmos-sdk/store/iavl.(*Store).DeleteVersions(...)
        github.com/cosmos/cosmos-sdk@v0.45.1/store/iavl/store.go:210
github.com/cosmos/cosmos-sdk/store/rootmulti.(*Store).pruneStores(0xc00017a480)
        github.com/cosmos/cosmos-sdk@v0.45.1/store/rootmulti/store.go:437 +0x10d
github.com/cosmos/cosmos-sdk/store/rootmulti.(*Store).Commit(0xc00017a480)
        github.com/cosmos/cosmos-sdk@v0.45.1/store/rootmulti/store.go:415 +0x20e
github.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).Commit(0xc000d09040)
        github.com/cosmos/cosmos-sdk@v0.45.1/baseapp/abci.go:308 +0x231
github.com/tendermint/tendermint/abci/client.(*localClient).CommitSync(0xc000fac6c0)
        github.com/tendermint/tendermint@v0.34.15/abci/client/local_client.go:264 +0xb6
github.com/tendermint/tendermint/proxy.(*appConnConsensus).CommitSync(0x0)
        github.com/tendermint/tendermint@v0.34.15/proxy/app_conn.go:93 +0x22
github.com/tendermint/tendermint/state.(*BlockExecutor).Commit(_, {{{0xb, 0x0}, {0xc000c05a80, 0x7}}, {0xc000c05a90, 0xb}, 0x310f7e, 0x3393e8, {{0xc002b09720, ...}, ...}, ...}, ...)
        github.com/tendermint/tendermint@v0.34.15/state/execution.go:228 +0x269
github.com/tendermint/tendermint/state.(*BlockExecutor).ApplyBlock(_, {{{0xb, 0x0}, {0xc000c05a80, 0x7}}, {0xc000c05a90, 0xb}, 0x310f7e, 0x3393e8, {{0xc002b09720, ...}, ...}, ...}, ...)
        github.com/tendermint/tendermint@v0.34.15/state/execution.go:180 +0x6ee
github.com/tendermint/tendermint/blockchain/v0.(*BlockchainReactor).poolRoutine(0xc000d01dc0, 0x0)
        github.com/tendermint/tendermint@v0.34.15/blockchain/v0/reactor.go:398 +0xb7b
created by github.com/tendermint/tendermint/blockchain/v0.(*BlockchainReactor).OnStart
        github.com/tendermint/tendermint@v0.34.15/blockchain/v0/reactor.go:110 +0x85

Outcome after fixing: restarted and continued without issues

Forced a panic during prune after database commit in DeleteVersionsRange iavl:

36A818E1B7A8CF3AEEAFEDFD7AC2AED7
ESC[90m9:56PMESC[0m ESC[32mINFESC[0m executed block ESC[36mheight=ESC[0m3381400 ESC[36mmodule=ESC[0mstate ESC[36mnum_invalid_txs=ESC[0m0
ESC[36mnum_valid_txs=ESC[0m0
ESC[90m9:56PMESC[0m ESC[1mESC[31mERRESC[0mESC[0m CONSENSUS FAILURE!!! ESC[36merr=ESC[0m"panic after Commit() in DeleteVersionsRange" ESC[36
mmodule=ESC[0mconsensus ESC[36mstack=ESC[0m"goroutine 415 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/
tendermint/tendermint/consensus.(*State).receiveRoutine.func2()\n\tgithub.com/tendermint/tendermint@v0.34.15/consensus/state.go:726 +0x4c\n
panic({0x1b0d920, 0x2749fb0})\n\truntime/panic.go:1038 +0x215\ngithub.com/cosmos/iavl.(*MutableTree).DeleteVersionsRange(0xc000d7a780, 0xc0
06abdf18, 0xc006abdf48)\n\tgithub.com/cosmos/iavl@v0.17.3/mutable_tree.go:899 +0x91\ngithub.com/cosmos/iavl.(*MutableTree).DeleteVersions(0
xc000c8c480, {0xc00197cb00, 0xc000c91a60, 0xb0})\n\tgithub.com/cosmos/iavl@v0.17.3/mutable_tree.go:880 +0x2ca\ngithub.com/cosmos/cosmos-sdk
/store/iavl.(*Store).DeleteVersions(...)\n\tgithub.com/cosmos/cosmos-sdk@v0.45.1/store/iavl/store.go:210\ngithub.com/cosmos/cosmos-sdk/stor
e/rootmulti.(*Store).pruneStores(0xc000c8c480)\n\tgithub.com/cosmos/cosmos-sdk@v0.45.1/store/rootmulti/store.go:437 +0x10d\ngithub.com/cosm
os/cosmos-sdk/store/rootmulti.(*Store).Commit(0xc000c8c480)\n\tgithub.com/cosmos/cosmos-sdk@v0.45.1/store/rootmulti/store.go:415 +0x20e\ngi
thub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).Commit(0xc0001e31e0)\n\tgithub.com/cosmos/cosmos-sdk@v0.45.1/baseapp/abci.go:308 +0x231\ngithub.com/tendermint/tendermint/abci/client.(*localClient).CommitSync(0xc000fd7680)\n\tgithub.com/tendermint/tendermint@v0.34.15/abci/client/local_client.go:264 +0xb6\ngithub.com/tendermint/tendermint/proxy.(*appConnConsensus).CommitSync(0x0)\n\tgithub.com/tendermint/tendermint@v0.34.15/proxy/app_conn.go:93 +0x22\ngithub.com/tendermint/tendermint/state.(*BlockExecutor).Commit(_, {{{0xb, 0x0}, {0xc000b7ce80, 0x7}}, {0xc000b7cea0, 0xb}, 0x310f7e, 0x339898, {{0xc001bb5ce0, ...}, ...}, ...}, ...)\n\tgithub.com/tendermint/tendermint@v0.34.15/state/execution.go:228 +0x269\ngithub.com/tendermint/tendermint/state.(*BlockExecutor).ApplyBlock(_, {{{0xb, 0x0}, {0xc000b7ce80, 0x7}}, {0xc000b7cea0, 0xb}, 0x310f7e, 0x339898, {{0xc001bb5ce0, ...}, ...}, ...}, ...)\n\tgithub.com/tendermint/tendermint@v0.34.15/state/execution.go:180 +0x6ee\ngithub.com/tendermint/tendermint/consensus.(*State).finalizeCommit(0xc000bc3180, 0x339898)\n\tgithub.com/tendermint/tendermint@v0.34.15/consensus/state.go:1635 +0x9fd\ngithub.com/tendermint/tendermint/consensus.(*State).tryFinalizeCommit(0xc000bc3180, 0x339898)\n\tgithub.com/tendermint/tendermint@v0.34.15/consensus/state.go:1546 +0x305\ngithub.com/tendermint/tendermint/consensus.(*State).addProposalBlockPart(0xc000bc3180, 0x39c, {0xc000bb43c0, 0xc000e91c00})\n\tgithub.com/tendermint/tendermint@v0.34.15/consensus/state.go:1919 +0x92e\ngithub.com/tendermint/tendermint/consensus.(*State).handleMsg(0xc000bc3180, {{0x27741a0, 0xc002b9fe30}, {0xc000bb43c0, 0x0}})\n\tgithub.com/tendermint/tendermint@v0.34.15/consensus/state.go:820 +0x17c\ngithub.com/tendermint/tendermint/consensus.(*State).receiveRoutine(0xc000bc3180, 0x0)\n\tgithub.com/tendermint/tendermint@v0.34.15/consensus/state.go:762 +0x419\ncreated by github.com/tendermint/tendermint/consensus.(*State).OnStart\n\tgithub.com/tendermint/tendermint@v0.34.15/consensus/state.go:378 +0x12f\n"
ESC[90m9:56PMESC[0m ESC[32mINFESC[0m Stopping baseWAL service ESC[36mimpl=ESC[0m{"Logger":{}} ESC[36mmodule=ESC[0mconsensus ESC[36mwal=ESC[0m/root/.osmosisd/data/cs.wal/wal
ESC[90m9:56PMESC[0m ESC[32mINFESC[0m Stopping Group service ESC[36mimpl=ESC[0m{"Dir":"/root/.osmosisd/data/cs.wal","Head":{"ID":"ghPPGW2OM86C:/root/.osmosisd/data/cs.wal/wal","Path":"/root/.osmosisd/data/cs.wal/wal"},"ID":"group:ghPPGW2OM86C:/root/.osmosisd/data/cs.wal/wal","Logger":{}} ESC[36mmodule=ESC[0mconsensus ESC[36mwal=ESC[0m/root/.osmosisd/data/cs.wal/wal

Outcome after fixing: restarted and continued without issues

This leads us to conclude that even if OOM, from which we can't recover, occurs during prune, the node should still be able to continue on restart on the latest v7.0.3 change.

Closing this issue for now

hrmm, do we know we're not starting the defer logic in these simulated panic cases?

I will test with os.Exit instead of panic just to make sure. os.Exit doesn't allow the option to recover and exits immediately, similar to how OOM would work

Sweet, ty!

Tested several times with extra logs, every time there is no issue on restart:
This is how it logs normally:
3:31AM INF commit start, height - 3385399
3:31AM INF commit end, height - 3385399
3:31AM INF flushing metadata
3:31AM INF flushing metadata finished
...

With panic() in DeleteVersionsRange(...) (after committing to database during prune):
4:46AM INF commit start, height - 3386300
4:46AM INF commit end, height - 3386300
4:46AM INF prune start, height - 3386300
4:46AM INF flushing metadata
4:46AM INF flushing metadata finished
4:46AM ERR CONSENSUS FAILURE!!! err="panic in DeleteVersionsRange" module=consensus stack="goroutine 264 [running]:\nruntime/debug.Stack()\

Note that it flushes. On restart it currently works fine

With os.Exit(1) in DeleteVersionsRange(...) (after committing to database during prune):
3:31AM INF executed block height=3385400 module=state num_invalid_txs=0 num_valid_txs=0
3:31AM INF commit start, height - 3385400
3:31AM INF commit end, height - 3385400
3:31AM INF prune start, height - 3385400
3:31AM INF os.Exit(1)

Note that it does not flush. When restarted, it still works as expected.

I think that tendermint has a mechanism to work around the possibility of not flushing the commit version in the SDK. This mechanism is in replay.go.
We previously thought that not flushing the commit was the main issue. However, it wasn't because tendermint detects if app version does not match its store version and catches up the app if needed overwriting the already written but unflushed commit.
The main issue IMO was iterating over fast storage on replay. It was not fixed at the time we made that assumption but it is now.
To verify that my guess is valid I logged all of storeBlockHeight, stateBlockHeight and appBlockHeight in replay.go under 3 circumstances:

On normal restart: storeBlockHeight: 3385963, stateBlockHeight: 3385963, appBlockHeight: 3385963

note that they are all equal, as a result, there is no need to catch them up to match them
The replay logic takes this branch and never actually replays because all heights are matching

On restart with panic() and flush: storeBlockHeight: 3386300, stateBlockHeight: 3386299, appBlockHeight: 3386300

note that stateBlockHeight differs from the other 2
Here, the replay logic takes this branch. It detects that the app has already written the commit so it uses a mock app to replay the commit and save state.

On restart with os.Exit and no flush: storeBlockHeight: 3386100, stateBlockHeight: 3386099, appBlockHeight: 3386099

note that storeBlockHeight differs from the other 2
In this case, the replay logic takes this branch and replays the last block on the app.

In conclusion, tendermint has a mechanism to replay the commit even if we fail to flush. When we manage to flush the committed metadata after prune failure, it replays the latest commit on the mock app before committing to state. When we don't flush the committed metadata after prune failure in commit, tendermint replays the latest commit on the actual app. That's why it works in both cases.
I'm not sure how big of an issue it is overwriting an already existing commit in the app. I think it should preserve idempotency.


Migrating from npm to yarn

The readme did already state "yarn" before. Was that ignored?

@asousame-capgemini remember to use signed commits please


Malware? What is this info?

This report seems like a false positive to me.  The SHA256 digest in your screenshot matches that of uncap.exe at https://github.com/susam/uncap/releases/tag/0.3.0.  This EXE was compiled using the source code tagged 0.3.0 on a pristine Windows VM on Azure.
If some kind community member who understands that scanning tool well could investigate and share exactly why this tool flags this binary, it would be very helpful.  From my limited investigation, it does look like this tool reports false positives:

https://www.reddit.com/r/antivirus/comments/14zijs6/
https://www.reddit.com/r/antivirus/comments/189d1mz/

In any case, you can always compile this executable from its source code shared in this project if you have a Windows machine and a compiler. The steps to do so are available at https://github.com/susam/uncap/tree/master/dev.

By the way, the analysis report is 3 months old (see top right corner of the page).  Can you try hitting the "Reanalyze" link (top right corner of the page again) and see if it still reports this digest as a problem?  I am hoping they might have fixed the false positive since then.

On hitting, it passes the checks.


Update quickstart.md

Added a check for troubleshooting "Nginx Unavailable" message. I had used the AWS VPC wizard to setup a VPC and security group, but the "docker-machine" security group did not have port 80 opened as part of that setup. This edit includes a check for that, in the event that an Nginx Unavailable message is observed.

Thanks @stephenmcnicholas , I tweaked the instructions to be slightly less AWS-centric.  Hopefully someone will accept this PR tomorrow!


raftpicker: make raftpicker Stop blocking

ping @aaronlehmann

LGTM
Sorry about the extra complexity, but this should definitely fix the issue I was concerned about.

@aaronlehmann it can create goroutine leak though. let me push small fix in this PR.

@aaronlehmann please, take a look at last commit

Current coverage is 54.77% (diff: 0.00%)

Merging #1258 into master will increase coverage by 0.03%

@@             master      #1258   diff @@
==========================================
  Files            78         78
  Lines         12422      12430     +8
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits           6799       6808     +9
  Misses         4679       4679
+ Partials        944        943     -1

Powered by Codecov. Last update e56300f...6b1d087

I don't understand why isShutdown should be necessary. I probably missed something...

@aaronlehmann sorry, I made no explanation. If Status() is called after Stop it will hang forever. And Stop can be called on automatic shutdown when you demoting a node.

Ah, I get it now.
LGTM


parseTree - Option to keep comments

Thanks for this library! I wrote a jsonc code formatter using it.
Right now I'm using the scanner to get the comments around the nodes in the tree. It would be slightly faster to not rescan these out and instead get all the comments when calling parseTree.
A simple solution would may be to have an option that returns all the comments in an array. Maybe something like...
export function parseTree(text: string, errors: ParseError[] = [], options: ParseOptions = ParseOptions.DEFAULT): Node;

Goes to:
export function parseTree(text: string, options: ParseOptions = ParseOptions.DEFAULT): ParseResult;

interface ParseResult {
	file: Node;
	errors: ParseError[];
	/** Included when `options.includeComments` is true. **/
	comments?: Comment[];
};

Then:
export interface ParseOptions {
	disallowComments?: boolean;
	allowTrailingComma?: boolean;
	allowEmptyContent?: boolean;
}

Goes to:
export interface ParseOptions {
	disallowComments?: boolean;
	includeComments?: boolean;
	allowTrailingComma?: boolean;
	allowEmptyContent?: boolean;
}

Thoughts? I could submit a PR for this, but will just need some direction on how the public API should look.

Proposal looks fine with, if you want to make a PR.

I want to make a vscode extension to sort the json fileds. I try this package, but can't find a good way to get the comment node info

Hey, I also want to simply keep the comments in the parsed jsonc, modify a property and insert the content back while preserving the comments. Can somebody help?

+1 I also would like to patch jsonc files without loosing comments


Adding hasOwnProperty check to stop lookup on the prototype

I had a key called 'values' in my data object but findInScope was finding Array.prototype.values, causing my template to not render properly. See jsfiddle: http://jsfiddle.net/h8v2goby/
Added a hasOwnProperty check to fix this.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Sarah Hayman seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Go client marshals ChatRequest.KeepAlive into wrong data type

As described by @jmorganca in https://github.com/ollama/ollama/issues/2343, the keep_alive field can be 0, -1 or another value indicating how long the model shall stay in memory.
The following Go code defines the keep_alive field as another struct with a Duration in it.
https://github.com/ollama/ollama/blob/7ed3e941058a47464a1ee97cd16f464eb788e396/api/types.go#L45
The current implementation results in the keep_alive being marshaled into {"Duration":86400000000000}
Here's a Playground example on how the ChatRequest struct is marshaled into JSON in the current implementation.
// You can edit this code!
// Click here and start typing.
package main

import (
	"encoding/json"
	"fmt"
	"time"
)

type Duration struct {
	time.Duration
}

type ChatRequest struct {
	Model     string    `json:"model"`
	KeepAlive *Duration `json:"keep_alive,omitempty"`

	Options map[string]interface{} `json:"options"`
}

func main() {
	request := ChatRequest{
		Model: "model-name",
		// keep alive for 24 hours
		KeepAlive: &Duration{Duration: time.Hour * 24},
	}

	jsonData, err := json.Marshal(request)
	if err != nil {
		fmt.Println(err)
	}

	fmt.Println(string(jsonData))
}

The output of this is as follows.
{"model":"model-name","keep_alive":{"Duration":86400000000000},"options":null}
A possible solution could be to change the keep_alive field to an int64.

@jmorganca is correct but he's missing some context. keep_alive has type Duration which has a custom JSON unmarshaler that transforms a number, e.g. -1, into a Duration. Since you're using the Go type directly, this does not apply. Duration also doesn't not have a custom marshaler which is why it's gets marshaled into "keep_alive": {"Duration": ...}


reafactor: reimplement Queue, separate Queue from Deque

What kind of change does this PR introduce? (check at least one)

[ ] Bugfix
[ ] Feature
[ ] Code style update
[x] Refactor
[ ] Build-related changes
[ ] Other, please describe:

Does this PR introduce a breaking change? (check one)

[ ] Yes
[x] No

If yes, please describe the impact and migration path for existing applications:
The PR fulfills these requirements:

[x] It's submitted to the dev branch, not the main branch
[x] When resolving a specific issue, it's referenced in the PR's title (e.g. fix #xxx[,#xxx], where "xxx" is the issue number)
[x] All tests are passing
[x] New/updated tests are included

If adding a new feature, the PR's description includes:

[x] A convincing reason for adding this feature (to avoid wasting your time, it's best to open a suggestion issue first and wait for approval before working on it)

Other information:

Coverage: 100.0%. Remained the same when pulling 1b3e2646878e3b0e6f097064df9d789dd7e2c056 on perf/queue into ff7e4b186bf4767f8cdcee98e05af779cf51fdee on dev.

This PR will solve the problem of duplicate code imported mentioned in https://github.com/js-sdsl/js-sdsl/pull/36, because we no longer have dependencies between different containers.

The delete operator make queue so slowly. https://github.com/mariadb-corporation/mariadb-connector-nodejs/issues/209