non additive capacity Constraint

I'm working on a problem where I want to apply a capacity constraint, but the capacity at each node depend on the previous nodes.  Let me try to explain it with an example
Assuming that we need to visit three nodes A, B , C

At A we have two products: (p1,p2)
At B we have two products: (p2,p3)
At C we have one product: (p3).
The capacity at each node is computed as the length of the union of all unique products ab to this node:

If the route started at A the capacity is defined by the length of the unique products
`Len(unique (p1,p2))=2

Then if we go to B, then the capacity is increased as follows
`Len(unique(p1,p2,p2,p3))=3

Then at C the capacity does not increase since p3 is already in the set
`Len(unique(p1,p2,p2,p3,p3))=3

The following example solved all my problems except that point.

https://github.com/google/or-tools/blob/stable/examples/python/cvrptw_plot.py

any advice will be appreciated.

How do you use the len() after it has been computed ?
Laurent Perron | Operations Research |<EMAIL_ADDRESS>| (33) 1 42 68 53
00
Le jeu. 30 mai 2019 à 22:34, KaramallahG<EMAIL_ADDRESS>a
écrit :

I'm working on a problem where I want to apply a capacity constraint, but
the capacity at each node depend on the previous nodes. Let me try to
explain it with an example
Assuming that we need to visit three nodes A, B , C

At A we have two products: (p1,p2)
At B we have two products: (p2,p3)
At C we have one product: (p3).
The capacity at each node is computed as the length of the union of
all unique products ab to this node:

If the route started at A the capacity is defined by the length of the
unique products
`Len(unique (p1,p2))=2
2.
Then if we go to B, then the capacity is increased as follows
`Len(unique(p1,p2,p2,p3))=3
3.
Then at C the capacity does not increase since p3 is already in the set
`Len(unique(p1,p2,p2,p3,p3))=3
The following example solved all my problems except that point.
https://github.com/google/or-tools/blob/stable/examples/python/cvrptw_plot.py
any advice will be appreciated.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/1313?email_source=notifications&email_token=ACUPL3IQI4YWD3GGDGXDIMTPYA25FA5CNFSM4HRK2JB2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4GWZ74IQ,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACUPL3PZQGNQULYUSD2T5CDPYA25FANCNFSM4HRK2JBQ
.

1- It is routing problem with almost 50 city and 4 vehicles.
2- Indeed, this is my problem which I could not not solve. At each step the len should be re-computed, so the list of products should be stored during the iterations to be able to recompute it?

 How do you use the len() after it has been computed ? Laurent Perron | Operations Research |<EMAIL_ADDRESS>| (33) 1 42 68 53 00 Le jeu. 30 mai 2019 à 22:34, KaramallahG<EMAIL_ADDRESS>a écrit :
…

1- It is routing problem with almost 50 city and 4 vehicles.
2- Indeed, this is my problem which I could not not solve. At each step the len should be re-computed, so the list of products should be stored during the iterations to be able to recompute it?

I meant what are the constraints on this len ?
Laurent Perron | Operations Research |<EMAIL_ADDRESS>| (33) 1 42 68 53
00
Le ven. 31 mai 2019 à 10:15, KaramallahG<EMAIL_ADDRESS>a
écrit :

How big is your model? How do you use the len() after it has been computed
? Laurent Perron | Operations Research |<EMAIL_ADDRESS>| (33) 1 42
68 53 00 Le jeu. 30 mai 2019 à 22:34, KaramallahG<EMAIL_ADDRESS>a écrit :
… <#m_3546821671216343635_>
1- It is routing problem with almost 50 city and 4 vehicles.
2- Indeed, this is my problem which I could not not solve. At each step
the len should be re-computed, so the list of products should be stored
during the iterations to be able to recompute it?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/1313?email_source=notifications&email_token=ACUPL3IMYXGVFFMOG4AMZWLPYDNANA5CNFSM4HRK2JB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWURLDA#issuecomment-497620364,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACUPL3IJ4QPQJB7HVP4OTK3PYDNANANCNFSM4HRK2JBQ
.

I meant what are the constraints on this len ? Laurent Perron | Operations Research |<EMAIL_ADDRESS>| (33) 1 42 68 53 00 Le ven. 31 mai 2019 à 10:15, KaramallahG<EMAIL_ADDRESS>a écrit :
…

It should not exceed a certain given value X
len (    all nodes in a route     ) < X
Thanks for your reply

How many different items ?
what is the value of X ?
Can you send me some data ? (distance matrix, items for each nodes) ?

Thanks for reply. Hope this can explain the problem
#!/usr/bin/env python3
from six.moves import xrange
from ortools.constraint_solver import pywrapcp
from ortools.constraint_solver import routing_enums_pb2
import random
stops = 20
vehicles = 5
depot = 0

# generate nodes
products=['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n' , 'o', 'p', 'q', 'r', 's', 't', 'u','v', 'w', 'x', 'y', 'z']
nodes={}
for i in range(stops):
    random_number=random.sample([2,3,4],1)[0]
    nodes[i]=random.sample(products, random_number)

def nodes_fn(node1):
    return len(set(node1))

## max cost
max_cost = 12 # the route cost should not excced this number

random.sample(products, 3)
# Create the routing index manager.
manager = pywrapcp.RoutingIndexManager(stops+1, vehicles, depot)

# Create Routing Model.
routing = pywrapcp.RoutingModel(manager)

def distance_callback(from_index, to_index):
    del from_index, to_index
    return 1

transit_callback_index = routing.RegisterTransitCallback(distance_callback)
routing.SetArcCostEvaluatorOfAllVehicles(transit_callback_index)

slack_max = 20
capacity = 20
fix_start_cumul_to_zero = True
name = 'time'
routing.AddDimension(
        transit_callback_index,
        slack_max,
        capacity,
        fix_start_cumul_to_zero,
        name)

time_dimension = routing.GetDimensionOrDie(name)
time_dimension.SetGlobalSpanCostCoefficient(1000)

search_params = pywrapcp.DefaultRoutingSearchParameters()

search_params.log_search = True
search_params.time_limit.seconds = 10
search_params.local_search_metaheuristic = (
        routing_enums_pb2.LocalSearchMetaheuristic.GUIDED_LOCAL_SEARCH)
assignment = routing.SolveWithParameters(search_params)

def print_solution(manager, routing, assignment):
    """Prints assignment on console"""
    print('Objective: {}'.format(assignment.ObjectiveValue()))
    total_distance = 0
    for vehicle_id in xrange(vehicles):
        index = routing.Start(vehicle_id)
        plan_output = 'Route for vehicle {}:\n'.format(vehicle_id)
        distance = 0
        union_elements=[]
        while not routing.IsEnd(index):
            union_elements +=nodes[manager.IndexToNode(min(index,19))]
            plan_output += ' {} -> '.format(manager.IndexToNode(min(index,19)))
            plan_output += ' (cost= {} )'.format(nodes_fn(union_elements))
            previous_index = index
            index = assignment.Value(routing.NextVar(index))
            distance += routing.GetArcCostForVehicle(previous_index, index, vehicle_id)
        plan_output += ' {}\n'.format(manager.IndexToNode(index))
        plan_output += 'Time of the route: {}h\n'.format(distance)
        print(plan_output)
        total_distance += distance
    print('Total Time of all routes: {}h'.format(total_distance))

print_solution(manager, routing, assignment)

The output
Objective: 8023
Route for vehicle 0:
 0 ->  (cost= 4 ) 14 ->  (cost= 6 ) 15 ->  (cost= 8 ) 16 ->  (cost= 11 ) 17 ->  (cost= 12 ) 18 ->  (cost= 13 ) 19 ->  (cost= 15 ) 19 ->  (cost= 15 ) 0
Time of the route: 8h

Route for vehicle 1:
 19 ->  (cost= 2 ) 13 ->  (cost= 6 ) 9 ->  (cost= 8 ) 10 ->  (cost= 11 ) 12 ->  (cost= 11 ) 11 ->  (cost= 15 ) 1 ->  (cost= 18 ) 0
Time of the route: 7h

Route for vehicle 2:
 19 ->  (cost= 2 ) 0
Time of the route: 0h

Route for vehicle 3:
 19 ->  (cost= 2 ) 0
Time of the route: 0h

Route for vehicle 4:
 19 ->  (cost= 2 ) 8 ->  (cost= 6 ) 7 ->  (cost= 8 ) 6 ->  (cost= 9 ) 5 ->  (cost= 10 ) 4 ->  (cost= 13 ) 3 ->  (cost= 16 ) 2 ->  (cost= 17 ) 0
Time of the route: 8h

Total Time of all routes: 23h

In this example I want to force the total cost to be less than 12

total cost == sum of costs of each route ? or max cost of each route ?
Laurent Perron | Operations Research |<EMAIL_ADDRESS>| (33) 1 42 68 53
00
Le lun. 3 juin 2019 à 21:01, KaramallahG<EMAIL_ADDRESS>a
écrit :

Thanks for reply. Hope this can explain the problem
#!/usr/bin/env python3
from six.moves import xrange
from ortools.constraint_solver import pywrapcp
from ortools.constraint_solver import routing_enums_pb2
import random
stops = 20
vehicles = 5
depot = 0
generate nodes
products=['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n' , 'o', 'p', 'q', 'r', 's', 't', 'u','v', 'w', 'x', 'y', 'z']
nodes={}
for i in range(stops):
random_number=random.sample([2,3,4],1)[0]
nodes[i]=random.sample(products, random_number)
def nodes_fn(node1):
return len(set(node1))
max cost
max_cost = 12 # the route cost should not excced this number
random.sample(products, 3)
Create the routing index manager.
manager = pywrapcp.RoutingIndexManager(stops+1, vehicles, depot)
Create Routing Model.
routing = pywrapcp.RoutingModel(manager)
def distance_callback(from_index, to_index):
del from_index, to_index
return 1
transit_callback_index = routing.RegisterTransitCallback(distance_callback)
routing.SetArcCostEvaluatorOfAllVehicles(transit_callback_index)
slack_max = 20
capacity = 20
fix_start_cumul_to_zero = True
name = 'time'
routing.AddDimension(
transit_callback_index,
slack_max,
capacity,
fix_start_cumul_to_zero,
name)
time_dimension = routing.GetDimensionOrDie(name)
time_dimension.SetGlobalSpanCostCoefficient(1000)
search_params = pywrapcp.DefaultRoutingSearchParameters()
search_params.log_search = True
search_params.time_limit.seconds = 10
search_params.local_search_metaheuristic = (
routing_enums_pb2.LocalSearchMetaheuristic.GUIDED_LOCAL_SEARCH)
assignment = routing.SolveWithParameters(search_params)
def print_solution(manager, routing, assignment):
"""Prints assignment on console"""
print('Objective: {}'.format(assignment.ObjectiveValue()))
total_distance = 0
for vehicle_id in xrange(vehicles):
index = routing.Start(vehicle_id)
plan_output = 'Route for vehicle {}:\n'.format(vehicle_id)
distance = 0
union_elements=[]
while not routing.IsEnd(index):
union_elements +=nodes[manager.IndexToNode(min(index,19))]
plan_output += ' {} -> '.format(manager.IndexToNode(min(index,19)))
plan_output += ' (cost= {} )'.format(nodes_fn(union_elements))
previous_index = index
index = assignment.Value(routing.NextVar(index))
distance += routing.GetArcCostForVehicle(previous_index, index, vehicle_id)
plan_output += ' {}\n'.format(manager.IndexToNode(index))
plan_output += 'Time of the route: {}h\n'.format(distance)
print(plan_output)
total_distance += distance
print('Total Time of all routes: {}h'.format(total_distance))
print_solution(manager, routing, assignment)
The output
Objective: 8023
Route for vehicle 0:
0 ->  (cost= 4 ) 14 ->  (cost= 6 ) 15 ->  (cost= 8 ) 16 ->  (cost= 11 ) 17 ->  (cost= 12 ) 18 ->  (cost= 13 ) 19 ->  (cost= 15 ) 19 ->  (cost= 15 ) 0
Time of the route: 8h
Route for vehicle 1:
19 ->  (cost= 2 ) 13 ->  (cost= 6 ) 9 ->  (cost= 8 ) 10 ->  (cost= 11 ) 12 ->  (cost= 11 ) 11 ->  (cost= 15 ) 1 ->  (cost= 18 ) 0
Time of the route: 7h
Route for vehicle 2:
19 ->  (cost= 2 ) 0
Time of the route: 0h
Route for vehicle 3:
19 ->  (cost= 2 ) 0
Time of the route: 0h
Route for vehicle 4:
19 ->  (cost= 2 ) 8 ->  (cost= 6 ) 7 ->  (cost= 8 ) 6 ->  (cost= 9 ) 5 ->  (cost= 10 ) 4 ->  (cost= 13 ) 3 ->  (cost= 16 ) 2 ->  (cost= 17 ) 0
Time of the route: 8h
Total Time of all routes: 23h
In this example I want to force the total cost to be less than 12
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/1313?email_source=notifications&email_token=ACUPL3LZYQFNOBVXQITR2YTPYVS73A5CNFSM4HRK2JB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODW2LYOQ#issuecomment-498383930,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACUPL3O3JHW6WDLXCXWZIZTPYVS73ANCNFSM4HRK2JBQ
.

Second question, is the distance matrix realistic ? Or just for the sake of
the example ?
Meaning is it a real delivery problem, or another problem mapped to a vrp
one ?
Laurent Perron | Operations Research |<EMAIL_ADDRESS>| (33) 1 42 68 53
00
Le lun. 3 juin 2019 à 21:17, Laurent Perron<EMAIL_ADDRESS>a écrit :

total cost == sum of costs of each route ? or max cost of each route ?
Laurent Perron | Operations Research |<EMAIL_ADDRESS>| (33) 1 42 68
53 00
Le lun. 3 juin 2019 à 21:01, KaramallahG<EMAIL_ADDRESS>a
écrit :

Thanks for reply. Hope this can explain the problem
#!/usr/bin/env python3
from six.moves import xrange
from ortools.constraint_solver import pywrapcp
from ortools.constraint_solver import routing_enums_pb2
import random
stops = 20
vehicles = 5
depot = 0
generate nodes
products=['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n' , 'o', 'p', 'q', 'r', 's', 't', 'u','v', 'w', 'x', 'y', 'z']
nodes={}
for i in range(stops):
random_number=random.sample([2,3,4],1)[0]
nodes[i]=random.sample(products, random_number)
def nodes_fn(node1):
return len(set(node1))
max cost
max_cost = 12 # the route cost should not excced this number
random.sample(products, 3)
Create the routing index manager.
manager = pywrapcp.RoutingIndexManager(stops+1, vehicles, depot)
Create Routing Model.
routing = pywrapcp.RoutingModel(manager)
def distance_callback(from_index, to_index):
del from_index, to_index
return 1
transit_callback_index = routing.RegisterTransitCallback(distance_callback)
routing.SetArcCostEvaluatorOfAllVehicles(transit_callback_index)
slack_max = 20
capacity = 20
fix_start_cumul_to_zero = True
name = 'time'
routing.AddDimension(
transit_callback_index,
slack_max,
capacity,
fix_start_cumul_to_zero,
name)
time_dimension = routing.GetDimensionOrDie(name)
time_dimension.SetGlobalSpanCostCoefficient(1000)
search_params = pywrapcp.DefaultRoutingSearchParameters()
search_params.log_search = True
search_params.time_limit.seconds = 10
search_params.local_search_metaheuristic = (
routing_enums_pb2.LocalSearchMetaheuristic.GUIDED_LOCAL_SEARCH)
assignment = routing.SolveWithParameters(search_params)
def print_solution(manager, routing, assignment):
"""Prints assignment on console"""
print('Objective: {}'.format(assignment.ObjectiveValue()))
total_distance = 0
for vehicle_id in xrange(vehicles):
index = routing.Start(vehicle_id)
plan_output = 'Route for vehicle {}:\n'.format(vehicle_id)
distance = 0
union_elements=[]
while not routing.IsEnd(index):
union_elements +=nodes[manager.IndexToNode(min(index,19))]
plan_output += ' {} -> '.format(manager.IndexToNode(min(index,19)))
plan_output += ' (cost= {} )'.format(nodes_fn(union_elements))
previous_index = index
index = assignment.Value(routing.NextVar(index))
distance += routing.GetArcCostForVehicle(previous_index, index, vehicle_id)
plan_output += ' {}\n'.format(manager.IndexToNode(index))
plan_output += 'Time of the route: {}h\n'.format(distance)
print(plan_output)
total_distance += distance
print('Total Time of all routes: {}h'.format(total_distance))
print_solution(manager, routing, assignment)
The output
Objective: 8023
Route for vehicle 0:
0 ->  (cost= 4 ) 14 ->  (cost= 6 ) 15 ->  (cost= 8 ) 16 ->  (cost= 11 ) 17 ->  (cost= 12 ) 18 ->  (cost= 13 ) 19 ->  (cost= 15 ) 19 ->  (cost= 15 ) 0
Time of the route: 8h
Route for vehicle 1:
19 ->  (cost= 2 ) 13 ->  (cost= 6 ) 9 ->  (cost= 8 ) 10 ->  (cost= 11 ) 12 ->  (cost= 11 ) 11 ->  (cost= 15 ) 1 ->  (cost= 18 ) 0
Time of the route: 7h
Route for vehicle 2:
19 ->  (cost= 2 ) 0
Time of the route: 0h
Route for vehicle 3:
19 ->  (cost= 2 ) 0
Time of the route: 0h
Route for vehicle 4:
19 ->  (cost= 2 ) 8 ->  (cost= 6 ) 7 ->  (cost= 8 ) 6 ->  (cost= 9 ) 5 ->  (cost= 10 ) 4 ->  (cost= 13 ) 3 ->  (cost= 16 ) 2 ->  (cost= 17 ) 0
Time of the route: 8h
Total Time of all routes: 23h
In this example I want to force the total cost to be less than 12
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/1313?email_source=notifications&email_token=ACUPL3LZYQFNOBVXQITR2YTPYVS73A5CNFSM4HRK2JB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODW2LYOQ#issuecomment-498383930,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACUPL3O3JHW6WDLXCXWZIZTPYVS73ANCNFSM4HRK2JBQ
.

Thanks for reply. I would say it is max cost of each route since the cost increase by adding nodes but not as sum.
Yes, it is real delivery problem, but here I just used dummy example for the distance.

Thanks for reply.

I would say max cost of each route, since the cost increase by adding nodes but not as a sum.
Yes, it is real delivery problem, but here I used dummy distance example

staring at your code, I don't think it does what you think it does.
You've set up a time dimension, and call the maximum time value "capacity".
The travel time callback function sets the travel time at 1 for each pair, regardless.  Obviously a toy example, but that is fine.
So at each node, time increases by 1.
But then in your problem output statement you start doing strange things, like calling nodes_fn which returns the length of a set made with the argument.  You haven't told the solver to do anything with this information, so you can't expect the output to be anything other than arbitrary.
I ran your code, rewriting your output function to be:
def print_solution2(manager, routing, assignment):
    """Prints assignment on console"""
    print('Objective: {}'.format(assignment.ObjectiveValue()))
    total_distance = 0
    time_dimension = routing.GetDimensionOrDie('time')
    for vehicle_id in xrange(vehicles):
        index = routing.Start(vehicle_id)
        plan_output = 'Route for vehicle {}:\n'.format(vehicle_id)
        distance = 0
        while not routing.IsEnd(index):
            time_var = time_dimension.CumulVar(index)
            plan_output += ' {} -> '.format(manager.IndexToNode(index))
            plan_output += ' (time= {} )'.format(assignment.Value(time_var))
            previous_index = index
            index = assignment.Value(routing.NextVar(index))
            distance += routing.GetArcCostForVehicle(previous_index, index, vehicle_id)
        plan_output += ' {}\n'.format(manager.IndexToNode(index))
        plan_output += 'Time of the route: {}h\n'.format(distance)
        print(plan_output)
        total_distance += distance
    print('Total Time of all routes: {}h'.format(total_distance))

The output looks like:
Objective: 8023
Route for vehicle 0:
 0 ->  (time= 0 ) 14 ->  (time= 1 ) 15 ->  (time= 2 ) 16 ->  (time= 3 ) 17 ->  (time= 4 ) 18 ->  (time= 5 ) 19 ->  (time= 6 ) 20 ->  (time= 7 ) 0
Time of the route: 8h

Route for vehicle 1:
 0 ->  (time= 0 ) 13 ->  (time= 1 ) 9 ->  (time= 2 ) 10 ->  (time= 3 ) 12 ->  (time= 4 ) 11 ->  (time= 5 ) 1 ->  (time= 6 ) 0
Time of the route: 7h

Route for vehicle 2:
 0 ->  (time= 0 ) 0
Time of the route: 0h

Route for vehicle 3:
 0 ->  (time= 0 ) 0
Time of the route: 0h

Route for vehicle 4:
 0 ->  (time= 0 ) 8 ->  (time= 1 ) 7 ->  (time= 2 ) 6 ->  (time= 3 ) 5 ->  (time= 4 ) 4 ->  (time= 5 ) 3 ->  (time= 6 ) 2 ->  (time= 7 ) 0
Time of the route: 8h

Total Time of all routes: 23h

So in this case, the time horizon of 20 is respected for each vehicle, as each route caps out at 8 hrs.  And of course, the actual problem relating to products at nodes not reported because the solver doesn't care about that.
Because I can never debug things with random numbers thrown in the mix, I replaced your random node generator with the first run, namely:
nodes={0:['q', 'w', 'o'],
       1:['l', 'j', 's'],
       2:['a', 'f', 'x', 'q'],
       3:['f', 'z', 'p'],
       4:['y', 'l', 'c', 'v'],
       5:['g', 'p', 'd'],
       6:['m', 'h'],
       7:['q', 'h', 'l'],
       8:['o', 'z', 'c'],
       9:['i', 't', 'f'],
       10:['x', 'q'],
       11:['p', 'g', 'v', 'i'],
       12:['u', 'k', 'q', 'x'],
       13:['f', 'y', 'm', 'c'],
       14:['n', 't', 'u'],
       15:['v', 'm', 'k'],
       16:['i', 'e'],
       17:['d', 'h', 'm'],
       18:['r', 'n', 'a', 's'],
       19:['w', 'x', 'y']
}

which, when unioned and sorted, produces a list:
['a', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z']

(only 'b' seems to be missing).
My interpretation of what you want is you're looking for a maximum number of unique items on each vehicle.  You haven't said exactly what this capacity should be, but from the comment in the code, I'm going to assume you want at most 12 unique items per vehicle.
Here's where it gets really tricky, AND where I'm going to stop for a moment and let you respond before I do anything more.  The problem is you're asking for the capacity (how much more is loaded into the vehicle at some node i) to be dependent upon all prior nodes that were visited by the vehicle.  By just sticking with one dimension to model vehicle load, this cannot be done.
Consider 0, 1, and 2 from above, where 0 has ['q', 'w', 'o'], 1 has ['l', 'j', 's'], and 2 has ['a', 'f', 'x', 'q'].  Visit to 0 picks up 3 items, a visit to 1 picks up 3 items and a visit to 2 picks up 4.  But instead, you want the pickup to be state dependent, so after visiting 0, a subsequent visit to 2 will only add 3 more items to the vehicle's load.  Correct?
The problem is you don't get much information at all in the demand callback.  In fact, you just get the node itself.   The solver also just collects integers, so you don't even get 'a', 'c', etc, you just get the running sum of numbers, so even if you use C++ and get access to the state-dependent transit callbacks, you still can't do what you want with just one dimension.  The dimension callback does not get to know what vehicle is being used, nor what nodes have come before that node in the vehicle's trip.
It isn't full stop; rather you have to figure out a clever hack.  What I would do is define one dimension per unique load type. This can get quite large, and might blow up the complexity.  In this case 26 dimensions (one per unique letter in the alphabet).  Then each node supplies 0 or 1 in each dimension, depending on what products it has that need to be collected.  What I can't see clearly is how to specify your max load criteria.  Perhaps setting a global constraint for each vehicle at the final depot node, such that no more than "max value" of these dimensions can be non-zero.

Thank you for your response. Each node can contain more than 100 different elements, so I'll first try grouping them into smaller groups, and then I'll try your method. Is it possible to post the complete code?
Many Thanks

I'd say just try it with all the products first.  If it fails it fails, but the solver often surprises me that it can handle what I think are complex constraints fairly easily.  If it doesn't work, then you can try combining products.
The code is mostly what you posted, with minor additions as follows:
First you need a function generator for use with each dimension.  I did it this way, but probably cleaner to use partial from functools.  It is also ugly in that it uses the global nodes object, but whatever.
def demand_fn_gen(dim,manager):
    def closure(index):
        node = manager.IndexToNode(index)
        # should probably check to make sure node is in nodes too...
        if dim in nodes[node]:
            return 1
        return 0
    return closure

Then set up each dimension in a loop.  The loop calls the function generator for each dimension to get the right evaluator.  Again, the dimension is collecting product names, and the evaluator takes a node and returns whether or not (0,1) there is similarly named product on offer there.  Also, I collect the dimensions in the loop.  Again, bad use of globals, but you'll fix that in production, right?
product_dimensions = []
for p in products:
    dimname = 'dim_'+p
    p_callback = demand_fn_gen(p,manager)
    p_callback_index = routing.RegisterUnaryTransitCallback(p_callback)
    routing.AddDimension(
        p_callback_index, # same "cost" evaluator as above
        0,  # No slack
        max_cost,  # at most max_cost pickups of this dimension
        True, # set to zero for each vehicle
        dimname)
    _D = routing.GetDimensionOrDie(dimname)
    product_dimensions.append(_D)

That goes above the constraints I posted earlier.  Otherwise same as your code.
Oh, I also added a few lines and a print statement to your output, mostly just to prove it was working:
    time_dimension = routing.GetDimensionOrDie('time')
    ...
        while not routing.IsEnd(index):
            time_var = time_dimension.CumulVar(index)
            union_elements += nodes[manager.IndexToNode(index)]
            print(union_elements,sorted(set(union_elements)))
            plan_output += ' {} -> '.format(manager.IndexToNode(index))
            plan_output += ' (time= {} )'.format(assignment.Value(time_var))
            plan_output += ' (cost= {} )'.format(nodes_fn(union_elements))
   ...

Also, you're initial bit of code was off by one---you should not have any demand at the depot node
nodes={0:[]}
for i in range(1,stops+1):
    random_number=random.sample([2,3,4],1)[0]
    nodes[i]=random.sample(products, random_number)

Thanks for your reply. I will test it


fix unclosed file warning when loading templates

Fixed an unclosed file warning caused by the fix for #97.

Thanks for the contribution.


Error message on IRdemo bibcheck

Getting "ERROR: /home/ubuntu/.virtualenvs/invenio/etc/bibcheck does not exist" on the site https://irdemo.tind.io/admin/bibcheck/bibcheckadmin.py
Edit: there is a message there saying "Note: This module is running via Invenio 1 compatibility mode. If you find any troubles, please submit an issue."

@MadeleineM @remileduc Is this issue still interesting? Is it TIND specific or reproducible also on an Invenio demo site?

I only got 404s while looking for bibcheck on @remileduc's Invenio 1 site, so I can't tell. @audub @PXke, any idea?
The issue is still reproductible on TIND's sites. That said, the page is very well hidden, so I am not sure if it's still in use.

Same error on CDS https://cds.cern.ch/admin/bibcheck/bibcheckadmin.py


[C/C++] Allow IntelliSence with fieldname struct initialization

When initializing a struct with fieldnames, IntelliSence should pop up.

This is more of a C feature as I don't think C++ has this type of initialization, but the IntelliSence for C is bundled with the C/C++ extension.

Please file here: https://github.com/Microsoft/vscode-cpptools


Repetition of paragraph

Most of the content in this paragraph is repeated in the next one (see permalinks below). Remove one?
https://github.com/rocker-org/website/blob/ab636ccbfe54617f12bbe271cd59e5809c193d5d/content/use/managing_users.md?plain=1#L32-L40
https://github.com/rocker-org/website/blob/ab636ccbfe54617f12bbe271cd59e5809c193d5d/content/use/managing_users.md?plain=1#L43

Hi, thank you for catching this!
Definitely needs to be fixed...


Revert "Adds i18n to widget layer (#123620)"

This reverts commit 9e4b5fb7d986fe1a92f92d970f8021e2c1e1603d.
The previous pr breaks tests in vector_graphics. Reverting for now
Pre-launch Checklist

[ ] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[ ] I read the Tree Hygiene wiki page, which explains my responsibilities.
[ ] I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
[ ] I signed the CLA.
[ ] I listed at least one issue that this PR fixes in the description above.
[ ] I updated/added relevant documentation (doc comments with ///).
[ ] I added new tests to check the change I am making, or this PR is test-exempt.
[ ] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

We could also temporarily disable those tests too, its not breaking developer tests just the package's own tests

closing revert for now, it looks like we can fix forward


[SPARK-47714][SQL] Set spark.sql.legacy.timeParserPolicy to CORRECTED by default

What changes were proposed in this pull request?

This config was added in 3.0 and it's time to use the CORRECTED behavior by default. This is not a big deal as the CORRECTED behavior is also throwing an error, but the error message is improved to show the actual problem.
Why are the changes needed?

better error message
Does this PR introduce any user-facing change?

error message becomes different
How was this patch tested?

existing tests
Was this patch authored or co-authored using generative AI tooling?

no

migration guide?

closing in favor of https://github.com/apache/spark/pull/45859


[Bug] [知识库] 建立完知识库以后对话服务会崩溃

Search before asking

[X] I had searched in the issues and found no similar issues.

Operating system information
Linux
Python version information

=3.11

DB-GPT version
main
Related scenes

[ ] Chat Data
[ ] Chat Excel
[ ] Chat DB
[X] Chat Knowledge
[ ] Model Management
[ ] Dashboard
[ ] Plugins

Installation Information

[ ] Installation From Source

[X] Docker Installation

[ ] Docker Compose Installation

[ ] Cluster Installation

[ ] AutoDL Image

[ ] Other

Device information
GTX 3060
Models information
default embedding
What happened
当我建立完知识图谱以后，只要Chat Knowledge没服务就崩溃
1、知识图谱确认已建立
2、确认chat llm正常，chat Knowledge就立刻崩溃
2024-09-14 18:31:04 2024-09-14 10:31:04 7ba7ee03bbb8 dbgpt.rag.transformer.llm_extractor[1] INFO Using model proxyllm to extract
2024-09-14 18:31:04 2024-09-14 10:31:04 7ba7ee03bbb8 dbgpt.model.adapter.base[1] INFO Message version is v2
2024-09-14 18:31:04 2024-09-14 10:31:04 7ba7ee03bbb8 dbgpt.model.cluster.worker.default_worker[1] INFO current generate stream function is asynchronous stream function
2024-09-14 18:31:04 2024-09-14 10:31:04 7ba7ee03bbb8 dbgpt.model.proxy.llms.chatgpt[1] INFO Send request to openai(1.44.1), payload: {'stream': True, 'model': 'THUDM/glm-4-9b-chat'}
2024-09-14 18:31:04
2024-09-14 18:31:04  messages:
2024-09-14 18:31:04 [{'role': 'user', 'content': "A question is provided below. Given the question, extract up to keywords from the text. Focus on extracting the keywords that we can use to best lookup answers to the question.\nGenerate as more as possible synonyms or alias of the keywords considering possible cases of capitalization, pluralization, common expressions, etc.\nAvoid stopwords.\nProvide the keywords and synonyms in comma-separated format.Formatted keywords and synonyms text should be separated by a semicolon.\n---------------------\nExample:\nText: Alice is Bob's mother.\nKeywords:\nAlice,mother,Bob;mummy\nText: Philz is a coffee shop founded in Berkeley in 1982.\nKeywords:\nPhilz,coffee shop,Berkeley,1982;coffee bar,coffee house\n---------------------\nText: 卫宁\nKeywords:\n"}]
2024-09-14 18:31:09 2024-09-14 10:31:09 7ba7ee03bbb8 openai._base_client[1] INFO Retrying request to /chat/completions in 0.867724 seconds
2024-09-14 18:31:15 2024-09-14 10:31:15 7ba7ee03bbb8 openai._base_client[1] INFO Retrying request to /chat/completions in 1.713854 seconds
2024-09-14 18:31:17 llm_adapter:
2024-09-14 18:31:17
2024-09-14 18:31:17 model prompt:
2024-09-14 18:31:17
2024-09-14 18:31:17 human: A question is provided below. Given the question, extract up to keywords from the text. Focus on extracting the keywords that we can use to best lookup answers to the question.
2024-09-14 18:31:17 Generate as more as possible synonyms or alias of the keywords considering possible cases of capitalization, pluralization, common expressions, etc.
2024-09-14 18:31:17 Avoid stopwords.
2024-09-14 18:31:17 Provide the keywords and synonyms in comma-separated format.Formatted keywords and synonyms text should be separated by a semicolon.
2024-09-14 18:31:17 ---------------------
2024-09-14 18:31:17 Example:
2024-09-14 18:31:17 Text: Alice is Bob's mother.
2024-09-14 18:31:17 Keywords:
2024-09-14 18:31:17 Alice,mother,Bob;mummy
2024-09-14 18:31:17 Text: Philz is a coffee shop founded in Berkeley in 1982.
2024-09-14 18:31:17 Keywords:
2024-09-14 18:31:17 Philz,coffee shop,Berkeley,1982;coffee bar,coffee house
2024-09-14 18:31:17 ---------------------
2024-09-14 18:31:17 Text: 卫宁
2024-09-14 18:31:17 Keywords:
2024-09-14 18:31:17
2024-09-14 18:31:17
2024-09-14 18:31:17 async stream output:
2024-09-14 18:31:17
2024-09-14 18:31:17
2024-09-14 18:31:17 2024-09-14 10:31:17 7ba7ee03bbb8 dbgpt.model.cluster.worker.default_worker[1] INFO is_first_generate, usage: None
2024-09-14 18:31:17 Keywords: 卫宁;Weining
2024-09-14 18:31:17
2024-09-14 18:31:17 Synonyms:
2024-09-14 18:31:28 2024-09-14 10:31:28 7ba7ee03bbb8 dbgpt.util.api_utils[1] WARNING Health check failed for http://<IP_ADDRESS>:5670, error: HTTPConnectionPool(host='<IP_ADDRESS>', port=5670): Read timed out. (read timeout=10)
2024-09-14 18:31:43 2024-09-14 10:31:43 7ba7ee03bbb8 dbgpt.util.api_utils[1] WARNING Health check failed for http://<IP_ADDRESS>:5670, error: HTTPConnectionPool(host='<IP_ADDRESS>', port=5670): Read timed out. (read timeout=10)
2024-09-14 18:31:58 2024-09-14 10:31:58 7ba7ee03bbb8 dbgpt.util.api_utils[1] WARNING Health check failed for http://<IP_ADDRESS>:5670, error: HTTPConnectionPool(host='<IP_ADDRESS>', port=5670): Read timed out. (read timeout=10)
2024-09-14 18:32:13 2024-09-14 10:32:13 7ba7ee03bbb8 dbgpt.util.api_utils[1] WARNING Health check failed for http://<IP_ADDRESS>:5670, error: HTTPConnectionPool(host='<IP_ADDRESS>', port=5670): Read timed out. (read timeout=10)
2024-09-14 18:32:28 2024-09-14 10:32:28 7ba7ee03bbb8 dbgpt.util.api_utils[1] WARNING Health check failed for http://<IP_ADDRESS>:5670, error: HTTPConnectionPool(host='<IP_ADDRESS>', port=5670): Read timed out. (read timeout=10)
2024-09-14 18:32:43 2024-09-14 10:32:43 7ba7ee03bbb8 dbgpt.util.api_utils[1] WARNING Health check failed for http://<IP_ADDRESS>:5670, error: HTTPConnectionPool(host='<IP_ADDRESS>', port=5670): Read timed out. (read timeout=10)
2024-09-14 18:32:58 2024-09-14 10:32:58 7ba7ee03bbb8 dbgpt.util.api_utils[1] WARNING Health check failed for http://<IP_ADDRESS>:5670, error: HTTPConnectionPool(host='<IP_ADDRESS>', port=5670): Read timed out. (read timeout=10)
2024-09-14 18:33:13 2024-09-14 10:33:13 7ba7ee03bbb8 dbgpt.util.api_utils[1] WARNING Health check failed for http://<IP_ADDRESS>:5670, error: HTTPConnectionPool(host='<IP_ADDRESS>', port=5670): Read timed out. (read timeout=10)
2024-09-14 18:33:28 2024-09-14 10:33:28 7ba7ee03bbb8 dbgpt.util.api_utils[1] WARNING Health check failed for http://<IP_ADDRESS>:5670, error: HTTPConnectionPool(host='<IP_ADDRESS>', port=5670): Read timed out. (read timeout=10)
2024-09-14 18:33:43 2024-09-14 10:33:43 7ba7ee03bbb8 dbgpt.util.api_utils[1] WARNING Health check failed for http://<IP_ADDRESS>:5670, error: HTTPConnectionPool(host='<IP_ADDRESS>', port=5670): Read timed out. (read timeout=10)
2024-09-14 18:33:58 2024-09-14 10:33:58 7ba7ee03bbb8 dbgpt.util.api_utils[1] WARNING Health check failed for http://<IP_ADDRESS>:5670, error: HTTPConnectionPool(host='<IP_ADDRESS>', port=5670): Read timed out. (read timeout=10)
2024-09-14 18:34:13 2024-09-14 10:34:13 7ba7ee03bbb8 dbgpt.util.api_utils[1] WARNING Health check failed for http://<IP_ADDRESS>:5670, error: HTTPConnectionPool(host='<IP_ADDRESS>', port=5670): Read timed out. (read timeout=10)
What you expected to happen
正常回答
How to reproduce
当我建立完知识图谱以后，只要Chat Knowledge没服务就崩溃
Additional context
No response
Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

it looks like your llm service problem.

My model service is functioning normally, and chat with LLM is also working fine. However, the service crashes when chatting with knowledge. Additionally, an LLM returning an exception should not cause the entire app to crash.

{
"id": "019205ac0bd94ce0c4fd26572b10043d",
"object": "chat.completion.chunk",
"created":<PHONE_NUMBER>,
"model": "Qwen/Qwen2-7B-Instruct",
"choices": [
{
"index": 0,
"delta": {
"content": "云"
},
"finish_reason": null,
"content_filter_results": {
"hate": {
"filtered": false
},
"self_harm": {
"filtered": false
},
"sexual": {
"filtered": false
},
"violence": {
"filtered": false
}
}
}
],
"system_fingerprint": "",
"usage": {
"prompt_tokens": 180,
"completion_tokens": 3,
"total_tokens": 183
}
}


✨ Add flag to include Blazor Web App & pages inside slices

This is focused on the backend. Blog post could be fine


internal: Ignore unpack Pod already exist errors in the Bundle controller

The r.ensureUnpackPod(...) attempts to create or re-create a Pod that's responsible for sourcing and unpacking Bundle contents. When that method returns an error from the util.CreateOrRecreate(...) logic where the generated unpack Pod's metadata.Name is already present on the cluster:
https://github.com/operator-framework/rukpak/blob/d868cf228aecc4bcd1ff1d1a3b32ac4177ca6717/internal/provisioner/plain/controllers/bundle_controller.go#L259
We can imply that this is because we're processing an out-of-date cache resource, and ignore that error as the Bundle's manager instance watches for Pod events:
https://github.com/operator-framework/rukpak/blob/d868cf228aecc4bcd1ff1d1a3b32ac4177ca6717/internal/provisioner/plain/controllers/bundle_controller.go#L437
Signed-off-by: timflannagan<EMAIL_ADDRESS>

Out of curiosity, do you know what sequence of events is causing the bundle reconcile to be called where we end up with the unpack pod being stale? Looking at the SetupWithManager function, nothing sticks out to me as something that might result in a bunch of concurrent requests for the same bundle.
My only guess is that we're creating the pod and updating the status, and we're somehow re-reconciling due to the status update before the cache contains the new pod. Is there a chance that we're doing c.Status().Update() somewhere before we create the pod?

I was under the impression that the requeue behavior happens when the Pod has been first created:
https://github.com/operator-framework/rukpak/blob/d868cf228aecc4bcd1ff1d1a3b32ac4177ca6717/internal/provisioner/plain/controllers/bundle_controller.go#L158-L159
We get back that op == controllerutil.OperationResultCreated reconcile result, but at that point, I guess the cache should already be up to date? This was only really reproducible for me when I had a bad unpack Pod that was being generated.

at that point, I guess the cache should already be up to date?

Correct. If the event that triggers the reconcile is the "pod create" event, then the cache will contain the pod object associated with that event.

Actually I'm not sure this is much of an issue right now. Worst case is we perform unnecessary work requeuing an apierrors.IsAlreadyExists which isn't the end of the world.


threadsd crashes when attempting to add db by client.newDBFromAddr()

I'm trying to access a db by daemon 2 which was created by a different threads daemon 1. I first get the dbinfos by
dbInfo = await client.getDBInfo(dbID);
from the threads daemon 1. Then I switch to the other daemon 2 and try to call
await client.newDBFromAddr(dbInfo.address[0], dbInfo.key, collections: [newCollection]);
And get the dart error:

Unhandled exception:
gRPC Error (2, failed to dial 12D3KooWJEwjTgJUfvwSeoW7TcqTHDevnBM1huyPY9jZcj1sccag: no good addresses)

What am I doing wrong?
When I then run again:
await client.newDBFromAddr(dbInfo.address[0], dbInfo.key, collections: [newCollection]);
results in error:

Unhandled exception:
gRPC Error (2, thread already exists)

If I then try to access the DB still from daemon 2 with
await client.getDBInfo(dbID);
The daemon crashes with

2020-05-09T14:25:42.328+0200	DEBUG	net	net/net.go:360	pulling thread bafkzjg7rdddl73kg4cq7iszrnyajs22ipbamersunjqkav52ajs36qq...
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x17b7a49]
goroutine 38893 [running]:
github.com/textileio/go-threads/db.(*DB).GetDBInfo(0x0, 0xc00427db28, 0x1, 0x1, 0x22, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/home/runner/work/go-threads/go-threads/db/db.go:298 +0x99
github.com/textileio/go-threads/api.(*Service).GetDBInfo(0xc001fa6d70, 0x1f81240, 0xc0085835f0, 0xc0086a59c0, 0xc001fa6d70, 0xc0085835f0, 0xc007f87ba0)
/home/runner/work/go-threads/go-threads/api/service.go:232 +0x275
github.com/textileio/go-threads/api/pb._API_GetDBInfo_Handler(0x1c8c880, 0xc001fa6d70, 0x1f81240, 0xc0085835f0, 0xc0040e2d20, 0x0, 0x1f81240, 0xc0085835f0, 0xc008118870, 0x24)
/home/runner/work/go-threads/go-threads/api/pb/api.pb.go:2506 +0x217
google.golang.org/grpc.(*Server).processUnaryRPC(0xc001fc8340, 0x1f8ce00, 0xc008f0b500, 0xc006ec8c00, 0xc00365a180, 0x280ff70, 0x0, 0x0, 0x0)
/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.0/server.go:1082 +0x50a
google.golang.org/grpc.(*Server).handleStream(0xc001fc8340, 0x1f8ce00, 0xc008f0b500, 0xc006ec8c00, 0x0)
/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.0/server.go:1405 +0xccb
google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc008744050, 0xc001fc8340, 0x1f8ce00, 0xc008f0b500, 0xc006ec8c00)
/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.0/server.go:746 +0xa1
created by google.golang.org/grpc.(*Server).serveStreams.func1
/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.0/server.go:744 +0xa1

and dart error:

Unhandled exception:
gRPC Error (2, HTTP/2 error: Connection error: Connection is being forcefully terminated. (errorCode: 10))

Our dart clients are no longer supported to the same degree that our Javascript and Go ones are. Having said that, I suspect the issue here is that the first address in that address array is a localhost address, so it is not dialble. Are you running this on the same machine? Same network?

Closing as stale.


fix: use relative links

Using relative links throughout the repo

Unrelated test is causing this PR to fail.

We're going to close and leave this as-is for now; thank you for the contribution that we can keep in mind for future changes, here 🙏


Add entry for Semgrep

Link to project: https://github.com/returntocorp/semgrep

Hey @pabloest,
thanks for the PR. Just updated the homepage URL to point to https://semgrep.live/, which seems to be the official address next to Github. Hope that's fine. Other than that, all good to merge. 💛

Thanks so much, @mre !


fix(TabBar): dp exceptions when using TBI as ItemTemplate root

GitHub Issue (If applicable): closes #1285
PR Type
What kind of change does this PR introduce?

Bugfix

What is the current behavior?
When using TabBar::ItemTemplate with TabBarItem as the root element, there are multiple exceptions throw on item materialization causing performance degradation:

The Dependency Property [WrongDP] is owned by [Control] and cannot be used on [ContentPresenter].

What is the new behavior?
^ no more.
PR Checklist
Please check if your PR fulfills the following requirements:

[x] Tested code with current supported SDKs
[x] Tested the changes where applicable:

[ ] UWP
[ ] WinUI
[ ] iOS
[ ] Android
[ ] WASM
[ ] MacOS
[x] Skia

[ ] Updated the documentation as needed:
[x] Runtime Tests and/or UI Tests for the changes have been added (for bug fixes / features) (if applicable)
[x] Contains NO breaking changes
[x] Associated with an issue (GitHub or internal)
[x] Commits must be following the Conventional Commits specification.

Other information
regression from #1261, which added support for TabBar::ItemTemplate with TabBarItem as the root element.

yes im aware of current build issue...
there is a problem on windows, when attempting to correct that...
in TabBar::PrepareContainerForItemOverride, setting both ContentPresenter::Content and ::ContentTemplate does NOT immediately materialized the content-template like on uno... It only materialize when the CP gets measured...
which means that we cant setup the CP's TBI in PrepareContainerForItemOverride
there is an potential injection site on TabBarListPanel::MeasureOverride which we can abuse? (after: foreach (...) child.Measure(...))

⚠⚠ The build 149388 has failed on Uno Toolkit - CI.

fixme@xy: // failing in all ios/droid/wasm

Verify_ItemTemplate_Has_No_Nested_TabBarItem
Verify_ItemTemplate_Disabled_Not_Selectable

@kazo0 / @Xiaoy312 any progress on getting this issue fixed? The temporary workaround we have can't be applied more generally as it affects the layout/UX of the tabbar too much.


[WIP] Add resample implementation for Series and DataFrame

Signed-off-by: Alexey Prutskov<EMAIL_ADDRESS>
What do these changes do?

[x] passes flake8 modin
[x] passes black --check modin
[ ] signed commit with git commit -s
[ ] Resolves #1195 and #1282 ?
[ ] tests added and passing

@devin-petersohn I implement methods of Resampler object, which is returned from DataFrame.resample(...) (it's look like Series.dt or Series.str). Is this correct approach? Or I need to implement only resample function?

@devin-petersohn Hi! I implement almost all methods of Resampler object.
When I use default_to_pandas method in the quantile function without to_pandas() call (line) object PandasQueryCompiler is returned. But I wait DataFrame or Series. Is this problem in implementation of default_to_pandas method?
If I use my implementation (not default_to_pandas) I have strange errors from ray - <ValueError: buffer source array is read-only>:
E                       ray.exceptions.RayTaskError(ValueError): ray::modin.engines.ray.pandas_on_ray.frame.partition.deploy_ray_func() (pid=112719, ip=<IP_ADDRESS>)
E                         File "python/ray/_raylet.pyx", line 427, in ray._raylet.execute_task
E                         File "python/ray/_raylet.pyx", line 449, in ray._raylet.execute_task
E                       ray.exceptions.RayTaskError: ray::modin.engines.ray.pandas_on_ray.frame.axis_partition.deploy_ray_func() (pid=112719, ip=<IP_ADDRESS>)
E                         File "python/ray/_raylet.pyx", line 463, in ray._raylet.execute_task
E                         File "/localdisk/aprutsko/modin/modin/engines/ray/pandas_on_ray/frame/axis_partition.py", line 108, in deploy_ray_func
E                           result = func(*args)
E                         File "/localdisk/aprutsko/modin/modin/engines/base/frame/axis_partition.py", line 202, in deploy_axis_func
E                           result = func(dataframe, **kwargs)
E                         File "/localdisk/aprutsko/modin/modin/engines/base/frame/data.py", line 729, in _map_reduce_func
E                           series_result = func(df)
E                         File "/localdisk/aprutsko/modin/modin/backends/pandas/query_compiler.py", line 543, in map_func
E                           val = op(resampled_val, *args, **kwargs)
E                         File "/localdisk/aprutsko/miniconda3/envs/modin/lib/python3.8/site-packages/pandas/core/resample.py", line 918, in quantile
E                           return self._downsample("quantile", q=q, **kwargs)
E                         File "/localdisk/aprutsko/miniconda3/envs/modin/lib/python3.8/site-packages/pandas/core/resample.py", line 1041, in _downsample
E                           result = obj.groupby(self.grouper, axis=self.axis).aggregate(how, **kwargs)
E                         File "/localdisk/aprutsko/miniconda3/envs/modin/lib/python3.8/site-packages/pandas/core/groupby/generic.py", line 928, in aggregate
E                           result, how = self._aggregate(func, *args, **kwargs)
E                         File "/localdisk/aprutsko/miniconda3/envs/modin/lib/python3.8/site-packages/pandas/core/base.py", line 311, in _aggregate
E                           return self._try_aggregate_string_function(arg, *args, **kwargs), None
E                         File "/localdisk/aprutsko/miniconda3/envs/modin/lib/python3.8/site-packages/pandas/core/base.py", line 267, in _try_aggregate_string_function
E                           return f(*args, **kwargs)
E                         File "/localdisk/aprutsko/miniconda3/envs/modin/lib/python3.8/site-packages/pandas/core/groupby/groupby.py", line 1904, in quantile
E                           return self._get_cythonized_result(
E                         File "/localdisk/aprutsko/miniconda3/envs/modin/lib/python3.8/site-packages/pandas/core/groupby/groupby.py", line 2291, in _get_cythonized_result
E                           func(**kwargs)  # Call func to modify indexer values in place
E                         File "pandas/_libs/groupby.pyx", line 722, in pandas._libs.groupby.group_quantile
E                         File "stringsource", line 658, in View.MemoryView.memoryview_cwrapper
E                         File "stringsource", line 349, in View.MemoryView.memoryview.__cinit__
E                       ValueError: buffer source array is read-only

../miniconda3/envs/modin/lib/python3.8/site-packages/ray/worker.py:1515: RayTaskError(ValueError)

ValueError: buffer source array is read-only happens because of pyarrow serialization that is used in Ray. Maybe we need to implement a workaround for this problem like it was done here https://github.com/modin-project/modin/pull/1534/commits/b930e3e5507df624bc3ebf9f18032fa378a7b4ef ?

Thank you @gshimansky!  I added the same workaround for this.

@devin-petersohn I think it is possible to make separate methods in query_compiler for each method of Resampler. But in this case we will have more lines of code in query_compiler. I implement this.

Thank you @prutskov, it is more lines, but gives more control to implementations like Omnisci.

@devin-petersohn Can you review again, please? If this is OK I will squash commits.

@devin-petersohn I implement several methods using groupby functionality. In this list:
get_group
nunique
mean
median
prod
std
sum
var

Part of methods from groupby give me incorrect result, when resampling was by DateTime-like column instead of index. I implemented those methods using apply_full_axis. List of this methods:
groups
indices
apply
aggregate
transform
pipe
ffill
backfill
bfill
pad
fillna
count
first
last
max
min
ohlc
size
sem
quantile

Methods nearest, asfreq and interpolate isn't present in groupby and were implemented using apply_full_axis.
Also testing was improved.

@prutskov Will you rebase on current master since CI issue was fixed? Thanks!

@devin-petersohn I rebased on current master. All checks have passed.


Missing step in instructions?

Hello, thanks for sharing this project.
I'm trying to install it "by the book":
I cloned the Bedrock repo to my local environment and set up my virtual host to point to the folder.
I ran composer install.
I  copied the .env.example to .env and provided the DB, ENV, HOME and SITEURL vars, but omitted the salts.
When I try load the site at "my.site.dev" the url switches to "my.site.dev/wp/wp-admin/install.php" which looks good, but then I get connection timeout or "not available". I can however load load the readme.html so the setup seems to be working in itself.
So then I decided to try add the salts with wp dotenv generate salts (after installing the https://github.com/aaemnnosttv/wp-cli-dotenv-command)
But I get
Error: The site you have requested is not installed.
Run wp core install.
Any ideas where I went wrong?

Ok, so reading in #184 I discover salts are not needed on local install...
I still find the instructions slightly confusing, sorry if I'm being thick, but I'm unclear about what the sequence is

Clone bedrock to your machine
Execute composer install
Copy .env.example to .env and add your case specific settings (db name, dblogin etc)
Add a theme to themes folder
Configure your local machine (hosts, vhosts)
Log in at wp/wp-admin

What I don't understand is how/when the variables in .env will be transformed into a wp-config file. There is no command executed after the .env is created and given the db settings

They're not transformed into wp-config.php. They're parsed by DotEnv and then retrieved as environment variables in application.php.
https://github.com/roots/bedrock/blob/master/config/application.php
The Bedrock issue tracker is for ongoing Bedrock development, not for support. Please use https://discourse.roots.io/ for support requests.

QWp6t, thanks for your answer and I can see your point about this not being the right forum for support, but I still feel the instructions seem to be missing some step or steps, which is a legitimate issue for the project.

Perhaps the requirements need to be clarified if it is not the installation instructions? I have PHP 5.5 (5.5.29) and Composer installed and followed the installation instructions.

@leanderlindahl documentation can always be improved, but in this I don't see anything you've missed.
If you can get to my.site.dev/wp/wp-admin/install.php but then the install doesn't work due to a bad DB connection then something must be wrong with your DB/the credentials you used.

@swalkinshaw Thanks for taking the time to reply. I really like the entire Roots project and "suite" and it's a great resource and opportunity to pick up good coding and workflow practices. I use it whenever there's an opportunity.
Now to the actual issue: I know the DB credentials are valid. I run 30-40 local installs with the same credentials. And I know the virtual host i correctly configured because I can load loc.mysite.dev/readme.html... I even started to think perhaps the .env file doesn't have the right permissions and can't be read and put the entire project (including .env) to chmod 777, just to make sure.
I guess this is something we would have discovered right away if we were in the same room but obviously it can't be conveyed or identified in this format. I believe you when if you say it is working both in theory and in practice. But it doesn't for me. (And I've hade the entire stack with Trellis installed and working several times, so I'm not completely lost with this stuff).
My guess is that there is one piece of the puzzle somehow missing and that's why I suggested  that there might be some "implicit" knowledge that could be missing in the requirements or instructions.
Following QWp6t suggestion I have continued the discussion here https://discourse.roots.io/t/missing-step-installing-bedrock/5542


Update middleware gem

Mitchell's original project is stale
New fork that's currently maintained instead!

Coverage remained the same at 98.006% when pulling e13ba30426cfe6f7c4e219d4464b0aebf7663de4 on update_middleware_gem into e69c3f08372ce9236caf3eee47ccc757ad88650b on master.


Implement AbandonedConfig for GenericKeyedObjectPool

Hi, we have added AbandonedConfig for GenericKeyedObjectPool.
Would be nice to see this in the next official release

Coverage decreased (-0.08%) to 84.066% when pulling 4edad40a26b482f43126e395d3154e436e2f97d7 on vitecde:abandonkeyed into a468ef3eae85043f58dc8e8fcb296058c8fe2ccf on apache:master.

Sorry for taking so long to review this.  Looks good to me.  Thanks and nice work.  I only have one comment.  The iteration in the top loop in removeAbandoned is over poolMap.entrySet.  This set is may be modified during execution of the loop (due to adding / dropping keyed pools).  I am not sure the behavior will be consistent / predictable in all cases.  Could be I am wrong on this in which case this commend can be ignored.  It might be better to take some kind of snapshot at the top of the loop.   I don't think any additional sync is needed (beyond what you have using the victims' monitors), and if the loop behaves predictably (skipping removals, possibly missing adds), the current code is OK.  Missing adds is not likely a problem since instances in new pools are not likely to have timed out and skipping removals similarly is no problem, since pools don't get dropped unless they are empty.  I am just worried about strange things happening due to behavior being undefined.

@psteitz Thats an interesting question. From what I understand ConcurrentHashMap seems at least make sure that the iterator itself does not fail and already does some kind of "snapshot" behaviour. It may or may not show concurrent changes depending on the implementation but it should not fail
https://stackoverflow.com/questions/3768554/is-iterating-concurrenthashmap-values-thread-safe

Looking at this again, I think both outer and inner loops are OK, using iterators that won't fail from CHM.  So this looks good to merge to me.


Configure the UATs to run behind a proxy

Context
This is the tracking issue for configuring the UATs behind proxy.
We need to be able to run the UATs behind proxy in order to test the bundle in that environment.
What needs to get done

#95
#96
Fix https://github.com/canonical/knative-operators/issues/204

Definition of Done
UATs can be run behind proxy

Summary
Completed work

#76 - The exploration issue contains a lot of context on the design and the issues discovered.
Issue https://github.com/canonical/knative-operators/issues/204 affecting KNative and KServe, the fixes are in:

https://github.com/canonical/kserve-operators/pull/257
https://github.com/canonical/knative-operators/pull/208

Work in progress

backporting of https://github.com/canonical/kserve-operators/pull/257 and https://github.com/canonical/knative-operators/pull/208
https://github.com/canonical/charmed-kubeflow-uats/issues/95
https://github.com/canonical/charmed-kubeflow-uats/issues/96

Todo

fix issue https://github.com/canonical/training-operator/issues/182 blocking TFJobs UATs - I expect this is an issue with our current UATs for TFJob not being able to pick up the proxy envs, not an issue with training operator itself.


Run-Time and File-Size of extracting method calls from multi-dex applications such as instagram

First of all, I would like to thank you for answering my previous questions:)
and then, I would ask you to read this issue carefully because your answer is too important for me. I target the instagram app which has 3 DEX files.
I have written two different codes for extracting method call from dex file, as you may have seen it before. Code No.1, uses apk, dvm sub-module and code No.2 uses analysis sub-module. I decide to print the output of the codes in a txt file, as you see in both codes. I also test the codes in both 'w', 'a' mode which stands for write to file, append to file.
Code No.1 :
import os
import time
from androguard.core.bytecodes import apk,dvm
from androguard.core.analysis import analysis

os.chdir('H:\ReverseEngineering\Sample\APK')

def PRINT_METHOD_CALLS(m):
    file = open('H:\ReverseEngineering\Sample\MethodCalls_instagram2.txt','a' )
    app = apk.APK(m)
    app_dex = dvm.DalvikVMFormat(app.get_dex())
    app_x = analysis.Analysis(app_dex)
    app_x.create_xref()
    classes = app_dex.get_classes()
    for class_ in classes:
        cx = app_x.get_class_analysis(class_.get_name())
        for method in cx.get_methods():
            file.write("inside Method {} ".format(method.name) + ':' + '\n')
            for _, call, _ in method.get_xref_to():
               file.write("    calling -> {} -- {}".format(call.class_name, call.name) + '\n')
    file.close()

begin = time.time()

for app in os.listdir():
    PRINT_METHOD_CALLS(app)

end = time.time()

print(end-begin)

Code No.2 :
from androguard.misc import AnalyzeAPK
import os
import time

os.chdir('H:\ReverseEngineering\Sample\APK')

def EXTRACT_METHOD_CALLS(a):
    app, list_of_dex, dx = AnalyzeAPK(a)
    for method in dx.get_methods():
        file.write("inside Method {} ".format(method.name) + ':' + '\n')
        for _, call, _ in method.get_xref_to():
            file.write("    calling -> {} -- {}".format(call.class_name, call.name) + '\n')

begin = time.time()

for apk in os.listdir():
    file_name , file_extension = os.path.splitext(apk)
    file = open(file_name + '.txt', 'w')
    EXTRACT_METHOD_CALLS(apk)
    file.close()

end = time.time()

print(end-begin)

The results I want to talk about with you, are the run-time and txt file's size.
on my computer, they are like below:
| Run time |   | File Size |
-- | -- | -- | -- | --
  | "w" | "a" | "w" | "a"
Code No.1 | 197 sec | 217 sec | 11.8 MB | 11.8 MB
Code No.2 | 571 sec | 503 sec | 31.7 MB | 31.7 MB
First of all, I thought because this is a multi-dex application, File Size must be different in "w", "a" mode, as a clear explanation, I thought in write mode codes will overwrite the Method call's of second dex file on first and also overwrite the third on the second. and in the append mode, it will just add them and nothing will overwrite. I really wonder when I see the content and the size of files in both "w" and "a" are equal. what's your idea, what really happens...?
Second, why does Run-Time increase in append mode...?
Third, the difference in content(last part) and increase in size, show that  Code No.2 deals with multi-dex file better. but I want to sure that whole the method calls in all dex file have written in txt file or Not...? please guide me about this.
Sorry, it was a bit long:)

The first code is wrong for multi dex files.
You are just adding the first dex and running analysis on it.
This is also the reason why the first code is much faster, as you are not taking into account all the DEX files.
A note: Use a session and reset it after each call of AnalyzeAPK - otherwise you will have all analyzed files in memory (see last code example here: https://androguard.readthedocs.io/en/latest/intro/sessions.html)

Good point, So the code No.2 must be like below:

from androguard.misc import AnalyzeAPK, get_default_session

def EXTRACT_METHOD_CALLS(a):

# get a default session
sess = get_default_session()

# Use the session
app, list_of_dex, dx = AnalyzeAPK(a , session = sess)

# Extract Method Calls
for method in dx.get_methods():
file.write("inside Method {} ".format(method.name) + ':' + '\n')
for _, call, _ in method.get_xref_to():
file.write("    calling -> {} -- {}".format(call.class_name, call.name) + '\n')

#reset the session
sess.reset()

Right....?

Would you please answer me...!
[as a master] I need your approve if code No.2 is compatible with multi-dex files.

that looks good


Mangas.in  updated web address

Source information
Mangasin 1.4.14
Source language
Spanish
Source new URL
https://m440.in/
Other details
No response
Acknowledgements

[X] I have updated all installed extensions.
[X] I have opened WebView and checked that the source URL has not changed yet.
[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a short but informative title.
[X] I will fill out all of the requested information in this form.

Duplicate https://github.com/keiyoushi/extensions-source/issues/5154 perhaps?


Re-introduce "CSDS Implementation"

Related #25038 #25745
The first CSDS attempt #25038 was rolled back because Envoy protos are not considered "core" components, or included in any components. But technically, we use generated upb files which means we depend on Envoy protos transitively anyway.
The most correct solution is adding Envoy protos to "core" component. But there are 2 blockers:

Adding Envoy proto packages means adding hundreds of new paths to the "core" component list, which includes Envoy's proto folder, protoc-gen-validate's proto folder, opencensus's proto folder, googleapis's proto folder;
Blaze's compatible_with tag "non_prod" is not instrumented for Envoy protos, which means we have to update the entire build chain's compatible_with tag, including: envoy, protoc-gen-validate, opencensus, googleapis, and part of internal ProtoBuf ("non_prod" is a superset of "gce"/"mobile", but ironically as a compatible tag it isn't compatible with existing tags) (all above packages are compatible with "gce", so there is no technical reason they won't be compatible with "non_prod").

With the release date set as next Tuesday, I don't think we can clear this two blockers quick enough.
Thanks to @gnossen. I just learnt that our staging test spawns GCE, pulls gRPC from GitHub, and test the OSS version. This means we can temporarily hide the CSDS feature in google3, without impacting the ability to test CSDS in production and in staging.
The internal counterpart of this PR is cl/363946493.
cl/363758897, the example produced import CL looks like cl/363759740.

FYI, there's already been some work on moving the xDS protos into core components internally.  See b/169268726 for details.
I'll take a look at the CL internally.

@markdroth PTALA. I have updated the generic hack with a more specific hack.

@nicolasnoble PTAL to see if the added Bazel rule is okay.

cl/364351425 is reviewed by Nico. This PR is cherry-picked as cl/364390547. Merging.


page about storage accounting

moving info from https://wiki.egi.eu/wiki/APEL/Storage

it can be put in production, so we can ask the product teams to update further things


request: Rename validator parameter of REQ to json_validator

Merged after adding an extra sentence of explanation to the second commit message, noting the confusion this rename helps with.  Thanks @andersk!


You mispelled error on the README.md

"function errorCallback(errror) {" should be "error"

This can close since it has been fixed

@muaz-khan can be closed.


Mars theme.ts

🔴 Mars Theme was inspired by a recent quote by Grimes on Instagram saying, she was "Ready to die with the red dirt of mars beneath my feet." I thought also of the movies Dune and Total Recall. I felt it might make our favorite Teslans feel more at home on Bitclout with a native theme inspired by their aspiring future home planet. I went with a high tech led type of red link and button to employ the new button class and give that studio-esq Red-Light-Special look.
[It might also be a nice Valentines Day type of theme for Valentines screenshots. 💖]
I can't do a screenshot with the new merge of the button class, but it's red to match the links and I would love if anyone could make a screenshot for me if it's rejected or merged.
Welcoming any core team adjustments to logo or whatever else.

@teambrixmusic i think there are some visibility issue with this theme ?
maybe a better color scheme ?

This one is a bit much for me too @teambrixmusic.
Maybe consider looking at some of the colour schemes for something like https://styles.ulysses.app/themes?order=likes could give inspiration of a good combination of colours.

Here's suggestions

Use background color more dark
Text and links color to be lighter
Don't use too bright colors.

Thanks for the feedback @harpreet9629 @tijno @AdityaChaudhary0005 😊
Since this one was too bold for main, I can move it to my stylus repo.
Is it rude to close a pull here on my end once submitted?
I have 2 others I can commit.
Do you think I should bring these up in a CIP in the future?


[BUG] Backup-target show old error message when connecting to new target

Describe the bug
when set up backup-target with a invalid server, error message popped; then trying another invalid server, the error message still shows old one.
To Reproduce
Steps to reproduce the behavior:

install Harvester with any nodes
navigate to Advanced -> Settings
edit backup-target and input an invalid NFS like nfs://<IP_ADDRESS>/invalid/endpoint
wait until the error message shown
edit backup-target and input another invalid NFS like nfs://localhost/invalid/endpoint

Expected behavior
Error message should show the latest one.
Environment:

Harvester ISO version: master-d05a6cc7-head
Underlying Infrastructure (e.g. Baremetal with Dell PowerEdge R630): Baremetal DL360 3nodes
ui-source Option: Bundled

Additional context
[similar case]

Root cause of this bug is, backup-target still in connecting state (as #1498 described), and the old error message box didn't set to invisible.

moved from #1498, also close #1498
The problem
when we establish connection to the backup-target, in some case, connection spend a long waiting for server response, and the UI didn't show any message to display current state is succeed or connecting.
The solution
Add an Info message box to display current state.
The Alternatives
An loading icon for alternative.
Additional Context
This case can be reproduced when you connect to an NFS Type, and connect to an server not provide NFS service, such like localhost.

ref to #1051

hi，@lanfon72   now save the configuration backend will check, there is no way to configure the invalid backup-target,    I think this issue has been fixed.    Can you help verify it

close this issue as it is fixed following #1051 , and unable reproduce the original case.


Update the dataformatID of BED-JSON

Update the dataFormatId of BED-JSON to ”9ca9e8f5-94bf-4fdd-81e3-31a58d7ca708” according to https://www.buildingenvelopedata.org/data-formats .
Priority = 2

Looks good. Thanks, @StephenCzarnecki !


MassTransit Quartz service error.

I am getting a Object reference not set to an instance of a object on the MassTransitQuartz Service project that you have .
I have narrowed it down to this function
static string TranslateJsonBody(string body, string destination)
        {
            JObject envelope = JObject.Parse(body);

            envelope["destinationAddress"] = destination;

            JToken message = envelope["message"];

            JToken payload = message["payload"];

            JToken payloadType = message["payloadType"];

           envelope["message"] = payload;

            envelope["messageType"] = payloadType;

            return JsonConvert.SerializeObject(envelope, Formatting.Indented);

        }
```
`

If i change the code above so that them message has correct case , it works

Any idea what is happening ? Has there been a change in the structure of the Message which are not reflected in the Quartz service code ?

```
  static string TranslateJsonBody(string body, string destination)
        {
            JObject envelope = JObject.Parse(body);

            envelope["destinationAddress"] = destination;

            JToken message = envelope["Message"];

            JToken payload = message["Payload"];
            JToken payloadType = message["PayloadType"];

            envelope["message"] = payload;
            envelope["messageType"] = payloadType;

            return JsonConvert.SerializeObject(envelope, Formatting.Indented);
        }
```

Hmm, this might be bad - it should be case-insensitive, and it should be using camelCase in the JSON tags...

@phatboyg i have also noticed that the MassTransit.QuartzIntegration project is using very old Nuget packges for MassTransit.
Also noticed that ScheduleMEssageService class in the MassTransit.QuartzService project still uses the API configuration based on MT 2.X  aka ServiceBusFactory.New
Will you be updating those projects ?
If you like I could do a pull request with case insensitive value retreival  for the Translate functions above .

I think you're on the wrong package: https://www.nuget.org/packages/MassTransit.Quartz
The other one is for the 2.x codebase.


nodePackages.prisma: 4.1.1 -> 4.2.1

Description of changes
Prisma 4.2.0 and 4.2.1 were released this week. This commit updates the version in nixpkgs to the latest one.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.


cf curl shoud add trailing new line to the output

[x] I tried updating to the latest version of the CF CLI to see if it fixed my problem.
[x] I attempted to run the command with CF_TRACE=1 to help debug the issue.
[x] I am reporting a bug that others will be able to reproduce.

Describe the bug and the command you saw an issue with
The output of cf curl ... > /some/file and cf curl ... --output /some/file differs.
Expected behavior
I would expect the the output of both cf curl ... > /some/file and cf curl ... --output /some/file being the same. While it is not a big problem for json (still can be confusing though) this is a problem when you download binary blobs (such as app bits or droplet).
To Reproduce
cf curl /v2/apps/$(cf app myapp --guid)/droplet/download --output /tmp/droplet.out.tgz
cf curl /v2/apps/$(cf app myapp --guid)/droplet/download > /tmp/droplet.redir.tgz
$ diff -u <(xxd /tmp/droplet.out.tgz) <(xxd /tmp/droplet.redir.tgz)
--- /dev/fd/63	2018-08-31 10:21:24.355472752 +0900
+++ /dev/fd/62	2018-08-31 10:21:24.355472752 +0900
@@ -186977,4 +186977,4 @@
 02da600: 2be3 95e6 f565 dcef fff9 d2e4 7fe9 ff95  +....e..........
 02da610: 8ff7 b556 ab72 d5ff 88fe 03c9 fa0f 4208  ...V.r........B.
 02da620: 2184 1042 0821 8490 42d8 03b9 48f6 6d00  !..B.!..B...H.m.
-02da630: 4060 00                                  @`.
+02da630: 4060 000a                                @`..

$ gunzip -vt /tmp/droplet.out.tgz
/tmp/droplet.out.tgz:	 OK

$ gunzip -vt /tmp/droplet.redir.tgz
/tmp/droplet.redir.tgz:
gzip: /tmp/droplet.redir.tgz: unexpected end of file

$ cf -v
cf version 6.38.0+7ddf0aadd.2018-08-07

@giner Thanks for reporting this. We can see how it would be confusing, since regular curl does not do append a new line. We are in the process of improving and refactoring the cf curl command, and will try to incorporate a fix to this in those changes.
As an aside, in situations where cf curl is missing some functionality that ordinary curl has, we have had some luck using curl -H "Authorization: $(cf oauth-token)" ....

@giner We are not planning to make this change right now.
If you need to pipe binary output, you can usecurl -H "Authorization: $(cf oauth-token)"..., and if you are saving a file, the cf curl --output flag is available.
There are two reasons for this decision. First, we don't want to prioritize an issue when easy workarounds are readily available.
Second, users with default PS1 who want to manually inspect a JSON document returned from CC API, which is the primary use case of cf curl, will be annoyed to see the last } of the JSON run into their prompt, as in:
cloudfoundry@host:~$cf curl /v2/info
{
   "name": "",
   "build": "",
   ....
}cloudfoundry@host:~$

While I agree being able to pipe binary files directly would be cool, users who wish to do that can use regular curl without much difficulty.
We are closing the issue for now. In general, cf curl is not meant to provide feature parity with curl, but rather just an easier-to-use command for common use cases. Feel free to open new issues in the future. Thanks!


Error when height width not specified

px-vis 1.2.1
Dashboard team is seeing a console error with the px-vis-timeseries component when creating a new component that has timeseries as one of it's elements.  More specifically the following error is seen when removing then adding the timeseries component:

Uncaught TypeError: Cannot read property 'push' of null
at HTMLElement.generateRandomID (px-vis-behavior-common.html:1885)
at HTMLElement._drawElement (px-vis-clip-path.html:94)
at HTMLElement._complexObserverEffect (polymer.html:1613)
at HTMLElement._effectEffects (polymer.html:1448)
at HTMLElement._propertySetter (polymer.html:1432)
at HTMLElement.__setProperty (polymer.html:1441)
at HTMLElement._applyEffectValue (polymer.html:1924)
at HTMLElement._annotationEffect (polymer.html:1587)
at HTMLElement._effectEffects (polymer.html:1448)
at HTMLElement._propertySetter (polymer.html:1432)

The error is in px-vis-behavior-common.html generateRandomID().
The line is: this._uniqueIdsUsed.push(id);
Theory:
I believe this is happening because the PxVisBehavior.uniqueIds.detached() method is being called which sets _uniqueIdsUsed to null.  When the px-vis-clip-path is created again it inherits the old data bindings instead of creating the new property.
I tried creating a codepen to reproduce the issue but polygit was not working.
This is the sequence of events where we see it:

Create custom polymer component with  as an element.

px-vis-timeseries should NOT have height or width properties set.

Create custom component using document.createElement()/.appendChild()

The timeseries properties look like this:
        <px-vis-timeseries
          hidden={{middleError}}
          display-threshold-title="true"
          include-all-series="true"
          show-tooltip="true"
          selection-type="xy"
          chart-vertical-alignment="top"
          id="apmChart"
          disable-navigator="{{!data.navigator}}"
          tooltip-config="{}"
          threshold-data="{{threshold}}"
          series-config="[[seriesConf]]"
          register-config='{"type":"horizontal"}'
          chart-data="[[chartData]]"
          >
        </px-vis-timeseries>

The default values for the above properties look like this.
      data: {
        type: Object,
        notify: true,
        reflectToAttribute: true,
        observer: '_widgetDataChanged'
      },
      seriesConfig: {
        type: Object,
        notify: true
      },
      chartData: {
        type: Object,
        notify: true
      },
      threshold: {
        type: Array,
        notify: true
      },
      middleError: {
        type: Boolean,
        notify: true,
        value: false,
      }

Observations:
PxVisBehavior.uniqueIds.detached() (px-vis-behavior-common) is setting _uniqueIdsUsed to null (as expected).
The size observer in px-vis-clip-path.html is triggering when _uniqueIdsUsed is null.
This can be mitigated by specifying height and width properties e.g. <px-vis-timeseries height="400" width="500" ...>
Looking at the logic I'm still not entirely sure why this happens but it did block 2 teams for too long so it would be good to save time for anyone else who might encounter it.
Will be submitting a PR for possible fix.

Hello,
thanks for taking the time of providing a detailed issue. I'm not entirely sure what the underlying problem is in your case but can't seem to be able to reproduce when moving charts around the dom.
However taking the approach of resetting the array to an empty array instead of null when detaching sounds safe to me, so if that fixes your issue I'm going to release a patch version with that fix

Thanks, much appreciated!

Now available in vis v2.0.7


Validate specified timezone against IANA database

Case 1: Site Configuration
If an invalid timeZone is specified in the site configuration, the failure is silent. Example:
timeZone = "America/LosAngeles"   # Should be America/Los_Angeles

Case 2: Time Function
If an invalid timezone is specified as the optional second argument to the time function, the failure is also silent (the build continues without error or warning). However, the error is rendered on the page (easy to miss and then inadvertently deploy). Example:
{{ time "2020-07-31T09:29:26" "America/LosAngeles" }}

This renders:
unknown time zone America/LosAngeles

The second case above is #8835 -- I separated that into its own issue, because that is more general and technical. We do return an error on invalid timezone in time.AsTime, but due to some technical mumbo jumbo, that error does not transform into a real error when you use the short form time.


Update/5.2.10 win mingw

xz 5.2.10
jira: https://anaconda.atlassian.net/browse/PKG-855
upstream: https://git.tukaani.org/?p=xz.git;a=tree (GitHub mirror)
license
release notes
Changes

Update download URL
Bump version and SHA
Use mingw build flow in favor of CMAKE/Ninja on Windows

Notes

INSTALL indicates that CMAKE/Visual Studio build flows cannot build the command line tools on Windows, despite this seemingly working on previous versions.
[win] The MSVC toolchain is used to produce the necessary dll and .lib files we had previously
[win] mingw is used only to create the command line tools

Linter check found the following problems:
The following problems have been found:
WARNING: /tmp/abs_80gtbfzl2j/clone/recipe/meta.yaml:59: missing_doc_source_url: The recipe is missing a doc_source_url
WARNING: /tmp/abs_80gtbfzl2j/clone/recipe/meta.yaml:59: missing_license_url: The recipe is missing a license_url
Warnings were found

Linter check found the following problems:
The following problems have been found:
WARNING: /tmp/abs_3ewiaghf93/clone/recipe/meta.yaml:59: missing_license_url: The recipe is missing a license_url
WARNING: /tmp/abs_3ewiaghf93/clone/recipe/meta.yaml:59: missing_doc_source_url: The recipe is missing a doc_source_url
Warnings were found

Linter check found the following problems:
The following problems have been found:
WARNING: /tmp/abs_04bnx205mx/clone/recipe/meta.yaml:59: missing_doc_source_url: The recipe is missing a doc_source_url
WARNING: /tmp/abs_04bnx205mx/clone/recipe/meta.yaml:59: missing_license_url: The recipe is missing a license_url
Warnings were found

Linter check found the following problems:
The following problems have been found:
WARNING: /tmp/abs_d62n9x3k0j/clone/recipe/meta.yaml:59: missing_doc_source_url: The recipe is missing a doc_source_url
WARNING: /tmp/abs_d62n9x3k0j/clone/recipe/meta.yaml:59: missing_license_url: The recipe is missing a license_url
Warnings were found

Hey team, @dbast and I have found an issue with this package while testing constructor. (see https://github.com/conda/constructor/pull/614 and https://github.com/conda/constructor/pull/616)
This package now contain files in lib/ (lowercase) under Windows, which creates some issues with Lib\site-packages on certain case-sensitive contexts (e.g. Bash on Windows):
./lib:
total 168K
-rw-r--r-- 2 jrodriguez 1049089 168K Jan 17 16:06 liblzma.a
drwxr-xr-x 1 jrodriguez 1049089    0 Jan 18 10:20 pkgconfig

On Python 3.10 packages, since xz is a Python dependency, it gets installed pretty early, "stealing" lib from Python's Lib. This seems to be creating issues with conda list, where packages are reported as installed from pypi instead of its originating channel.
What's worse, this would propagate to all packages built with this xz on $PREFIX. Those archives will have lib on the files metadata, instead of the most-common-on-Windows Lib.


Cache images instead of using cdns

In addition to tmp folder, we need to have a CDN cache. Since official albion CDN is too slow and responds pooly, find a way to cache images in a file or custom service, preferabily a s3 bucket or a cache service.

Done in v2.2.0


i18n: Add support for internationalisation

Our current website is available in different languages, so we'll need to support that in the new site too.
This falls apart in three aspects:

Routing
Translation of content
Translation of the interface

Routing
Languages are currently determined by  a language code in the URL, with the default (en) not using a code. For example /about/ is English, whereas /fr/about/ is the French version of that same page.

Routing should take this into account. Links to /about/ should switch to /fr/about if the user is browsing the French site.
We could/should detect the language requested by the browser, and serve that language (if we have it)
Going to /en/about/ should also work (here we could just redirect to /about/)
When you're on /about and you change your language to French, we should probably send you to /fr/about.
When /about is not translated, people visiting /fr/about should get to see the English version with a notice that this content is not available in French. However, we should not redirect them to /about as that would change the interface language.

Unfortunately, while there is an i18n plugin for Gatsby, it doesn't seem to be all that great and this might require some work.
Translation of content
This is relatively straightforward. /about can be an English markdown page, and /fr/about a French one.
Translation of the interface
Here, there's a number of options. We can use a fully-featured package such as react-intl or implement something more minimalistic.
A few requirements from the top of my head:

Support for loading translations from YAML files (as that's what our translators are used to)
Support for purals, date formatting, and variable interpolation (this would suggest we're better of with react-inlt)

I think we've covered everything. There might be some more subtle issues, but I'm closing this as the current site is working with multi-lingual markdown content, and react-intl for translation


Added Share Soft Delete

@tg-msft, could you look at this PR next?  It's relatively small.

The only major question I have is about the name (also in ApiView). We called this Undelete for Blobs and we're using Restore here. Any reason we don't want Undelete for both?

If we want to keep the APIs consistent, I'd prefer "Restore" over "Undelete".  For the original naming of the APIs, I just copied the service.

Could we just keep calling Delete periodically until we stop getting ShareBeingDeleted or whatever the error code is? If you do keep this as-is, we've got a handy https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/storage/Azure.Storage.Common/tests/Shared/StorageTestBase.cs#L338 that encapsulates the mode check.

I'll attempt to call Delete in a loop until the error code changes and see what happens.

If we want to keep the APIs consistent, I'd prefer "Restore" over "Undelete". For the original naming of the APIs, I just copied the service.

Should we rename Undelete in Blobs to Restore (with the old method hidden by [EditorBrowsable(Never)] and calling into the new one)?  This is maybe worth having a quick chat with the right folks on your team to make a call here.  It's going to be a little weird to have ShareClient.Restore and BlobContainerClient.Restore but BlobClient.Undelete.  Resolving that question doesn't need to block this PR from getting in.

If we want to keep the APIs consistent, I'd prefer "Restore" over "Undelete". For the original naming of the APIs, I just copied the service.

Should we rename Undelete in Blobs to Restore (with the old method hidden by [EditorBrowsable(Never)] and calling into the new one)? This is maybe worth having a quick chat with the right folks on your team to make a call here. It's going to be a little weird to have ShareClient.Restore and BlobContainerClient.Restore but BlobClient.Undelete. Resolving that question doesn't need to block this PR from getting in.

Ok, I'll leave the API name as-in for now, and we can circle back around once 73 is closer to the main Arch Board review.


Rebalancing tree

Hey @nomeata,
do you plan to add an implementation of a rebalancing tree as is implemented in the Rust ic-certified-map library or are there reasons you think this is not needed?
See the issue here.

I believe it's not needed, because it's a trie that branches on the bits of the key which for uniformly distributed keys (as likely the case for example with hashes as labels) nicely yields a balanced tree. And even an unbalanced tree can't be deeper than the twice the number of bits in the key.
But it's true that the module should have some documentation on that.


fix: remove errant print

Signed-off-by: Chris Gianelloni<EMAIL_ADDRESS>

Well catched!!! Thank you. I'm going to Merge and Release a new version fixing this bug.


Crash with REI (Roughly Enough Items), Architectury , The Dev said to me

I asked him and he told me it seems like Zenith Attribute crashed the modpack,
really hope you can take a look at your mod, it's a really good mod
log: https://mclo.gs/bYdpXXj
crash log: crash-2024-11-05_05.58.10-client.txt

No idea why I wrote this mixin...
Will release a fix soon


Obsolete  GO:0046752 viral capsid precursor transport to host cell nucleus?

GO:0046752 viral capsid precursor transport to host cell nucleus is defined as 'Any process in which viral capsid precursors are transported to a specific location in the nucleus, thus accumulating the necessary components for assembly of a capsid. ' - but I dont think this is a specific transport process, this is just specifying the targets.
No annotations, no mappings
@pmasson55 @genegodbold Is this a useful term ?

@pmasson55 should comment--I don't know how common this is or if there are special cellular features that are commonly exploited to enable this transport. IF there were such features and more than a couple of viruses do it, THEN I think the term is worthwhile, though you might want to add some molecular detail to the title to distinguish it.


Receipt functionality

As per discussion in #27 with @TomAFrench I have been working on implementation for the new receipt endpoint(s).
Specifically:

Create a receipt
Retrieve a receipt
Delete a receipt

New functions are:

create_receipt
retrieve_receipt
delete_receipt.

create_receipt
The :params: for create_receipt consist of all possible parameters to create a receipt. Specifically, the parameters transaction_id, total, currency and items are required. Whereas, taxes, payments and merchant can be provided but are not required. The function accounts for this by checking wheather they are supplied and are of correct type.
The :rtype: for create_receipt is a tuple which consists of the the ... information you provided, repeated back to you. as well as the external_id which is randomly generated. This allows the end user to be able to store the external_id for later use, to query the endpoints retrieve_receipt and delete_receipt which require the external_id of the created receipt.

Update
This branch is most likely going to be idle until the devs fix the API issue.

Checking in, I haven't forgotten about this. Chasing up to see if this issue has been fixed yet...


PostAgent - weird json utf8 error

Setup
Using an PostAgent to submit to a login form. No redirection on the server, simple form structure with form content type. Config:
{
  "post_url": "https://---",
  "expected_receive_period_in_days": "1",
  "content_type": "form",
  "method": "post",
  "payload": {
    "id":"---",
    "passwd":"---",
    "Action":"---"
  },
  "emit_events": "true",
  "no_merge": "true",
  "output_mode": "clean"
}

Error
[00:00:00] ERROR -- : Exception during dry-run. source sequence is illegal/malformed utf-8: /app/vendor/bundle/ruby/2.5.0/gems/activesupport-<IP_ADDRESS>/lib/active_support/core_ext/object/json.rb:38:in `to_json'
...
/usr/lib/ruby/2.5.0/webrick/server.rb:307:in `block in start_thread'

Full error here.
Question
Is this an error in the response or request? The target site is pretty old and uses headers with:
<?xml version="1.0" encoding="ISO-8859-1"?>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
                      "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
  <head><link href="favicon.ico" rel="shortcut icon" />
    <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type" />
<!-- .... ->

In my experience errors like these are mostly caused by websites reporting a wrong encoding. Huginn does not try to guess the encoding of the data it receives, but trusts the encoding set by the HTTP response.

Hello,
the problem is - we cannot change encoding from external websites - so it would be very useful, to set/change encoding.
kind regards
Randy

Thats true what @lessrand said. I tried different HTTP headers to froce the server to send an different encoding but no success. There was also an HTTP Header to set an accepted charset for the client to the server, however it support hast been dropped and hasn't been implemented at all, as the Clients tend to do more work on their own the the responses as seen in the last years.
So any possible way, to add this to the Huginn project by default?

In cases like these the headers send by the client rarely make a difference. I am guessing that the server is "thinking" the content is encoded in - for example - utf8, thus sending it as the encoding, but the actual HTML is written in a different encoding.
The Website- and PostAgent support the force_encoding configuration option. It can be used to force Huginn to use a different encoding.

@dsander Tested but doesn't change anything. Still receiving the same error. And if you look into line 6 of the error message (/app/lib/json_with_indifferent_access.rb:10:in dump') you can see that it is actually about constructing an JSON Object for the function build_eventfromagents.rb. I'm not an Huginn expert, which means I don't know the exact structure. But force_encoding` changes nothing for me at this point.

How did you determine the actual encoding of the data returned by the website? Can you share the URL or the raw data?

@dsander See attachment.
$ curl URL >> out.html
$ file -i out.html
out.html: text/xml; charset=iso-8859-1

Attachments
out.html.zip
NOTE: needed to send .zip as GitHub blacklisted .html

Hmm yes the charlock_holmes gem also thinks it's ISO-8859-1, you did try to set force_encoding to ISO-8859-1 as a "root" configuration option of the PostAgent?

@dsander Yes that worked. I will see how this encoding will flow through the following agents, and report back later.

@dsander Issue has been solved by this setting. Messages created by this Agent do not cause any issues in receiver agents.

Just one more comment on this.
The same issue is true for CSVReader,SMTPAgent a.s.o.
If you are not in an english speaking country you always have to handle various encodings and I cannot force my mail provider or my data source to switch from Latin1 or Eastern European Encoding to UTF8. Yes in a perfect world there would be utf8 in every system, but reality is not like that you have ansi, latin...
So please make the agents aware of that and allow a encoding parameter for the sources that automatically converts from whatever encoding to utf8 and huginn agents will be way more useful.
kind regards
Randy

@lessrand Hmm the CsvAgent only accepts data that is already in Huginn, so the encoding should not be an issue because we store everything as UTF-8. I don't know to which Agent to are referring to as SMTPAgent.

Dear dsander,
CSVReader reads a file I get by ftp - so if I want to convert that ftp file to utf8 i need some conversion.
It may be that it is ansi, latin1 or something else. maybe an encoding cnversion agent might be an alternative.
SMTPAgent fails when the mail server returns non utf8 characters, because it is set to some other encoding.
A handling of mail Input that is not utf8 encoded would be very useful.
kind regards
Randy

Gotcha, what I am saying is that if we add (forced) encoding conversion it has to be done at the "edge", i.e. the Agent that first imports the data into Huginn. I just looked at the code and the FtpsiteAgent already supports force_encoding (it's even documented in the Agent description which isn't always the case 😉 ). I still don't know what the SMTPAgent is though.


tuiTd control styles implements to Dialog window generated by component inside tuiTd

In Table component we have tuiTd , tuiTd makes styles for tui inputs (tui-input / tui-text-area), i.e. hide borders, make input content fill 100%.
But these styles also incorrectly implements to dialogs generated by components those are inside tuiTd.
Example: https://stackblitz.com/edit/angular-tqgx3q-vqz7p2

@nsbarsukov @vladimirpotekhin Do you think this is a bug?
Fix:
    this.dialogService
      .open<boolean>(new PolymorpheusComponent(DialogComponent))
      .subscribe();

instead of
    this.dialogService
      .open<boolean>(new PolymorpheusComponent(DialogComponent, this.injector))
      .subscribe();

This is expected behavior, that is how DI propagates in Angular and there are easy ways to work around this.


Save training loss history

Hello, I would like to save the training loss history to see the progress using different optimizers.
I am not familiar with TensorFlow. How can I extract this information?
Kind Regards.

Hi Gollini, you may want to use add another callback function to access the loss from the model. Please check an example here: https://stackoverflow.com/questions/65302030/access-loss-and-model-in-a-custom-callback
You can consider adding a callback to the https://github.com/UW-EXP/GLOBEM/blob/6b7581d476bc1fc1f68b9488b56089563df7b900/algorithm/dl_erm.py#L71