test case

For ixgbe nic, we want to assign a tx hardware qeueue to each cpu,
and the tx softirq should use the corresponding hardware queue.

each packet will select a softqueue in dev_queue_xmit,
we rewrite ixgbe driver ndo_select_queue(ixgbe_select_queue),
which will return current cpu index(based 0) when packet select queue.
thus for each cpu use its own tx queue.

but, we found some packet had unmatched queue index when send
on specific cpu.

for example, a packet’s queue index is 5 but is sent by cpu3,
thus, cpu3 will operate tx hw queue5, which should only be done by cpu5.

Read More

data structure

1
2
3
4
5

7913 static const struct net_device_ops ixgbe_netdev_ops = {
...
7916         .ndo_start_xmit         = ixgbe_xmit_frame,
7917         .ndo_select_queue       = ixgbe_select_queue,
...

receive skb: record receive queue index

record queue_index +1, 0 is used as NOT record.

####call trace

1
2
3
4

> ixgbe_poll
> > ixgbe_clean_rx_irq
> > > ixgbe_process_skb_fields
> > > > skb_record_rx_queue

Read More

test case

####On redhat5, Why tcpdump could not work on bonding work.
OS: redhat 5.
There are two 82599 interfaces eth0 and eth1.
These two interfaces are used as slave of bond0,
eth1 is backup of eth0.

We ping the default gateway on test machine.
ping work OK, and tcpdump on bond0 show the icmp request and icmp require packets.
while on eth0 only icmp request, and eth1 has no any packet.

Read More

This question should be divided into two parts.

Part 1: adding a route

Add a direct(link) route

Add a route with gateway

create a socket

for example ‘connect’ system call

call trace

以handle_level_irq为例说明.

1
2
3
4

===> handle_level_irq
==> ==> handle_irq_event
==> ==> ==> handle_irq_event_percpu
==> ==> ==> ==>action->handler

where handler is registered

1
2
3
4
5
6

127 static inline int __must_check
128 request_irq(unsigned int irq, irq_handler_t handler, unsigned long flags,
129             const char *name, void *dev)
130 {
131         return request_threaded_irq(irq, handler, NULL, flags, name, dev);
132 }

Read More

中断处理过程：

硬件中断到中断控制器

reg value–>irq(int) —> struct irq_desc

1
2
3
4
5
6

==> 中断时的有一个寄存器会保存中断源的vector值.
==> ==> `arch/x86/kernel/entry_64.S`调用函数`do_IRQ`.
==> ==> ==> `do_IRQ`依据`vector_irq`和vector值, 找到对应的中断号,并调用`handle_irq`.
==> ==> ==> ==> `handle_irq`通过函数irq_to_descdesc,可将中断号转化为`struct irq_desc`.
==> ==> ==> ==> generic_handle_irq_desc(irq, desc);
==> ==> ==> ==> ==> `generic_handle_irq_desc`调用 desc->handle_irq(irq, desc);

注:这里的handle_irq不是真正的中断处理函数,而是几大类中断控制器处理函数.
如82599, msi等.
具体分析见:irq study1

中断控制器到具体的中断处理函数

1
2
3
4
5

==> handle_level_irq
==> ==> irqreturn_t handle_irq_event(struct irq_desc *desc)
==> ==> ==> struct irqaction *action = desc->action
==> ==> ==> ret = handle_irq_event_percpu(desc, action);
==> ==> ==> ==> action->handler(irq, action->dev_id);

这里的action->handler才是我们使用request_irq注册的中断处理函数.
具体分析见:
具体分析见:irq study2

中断处理过程：

reg value–>irq(int) —> struct irq_desc

1
2
3
4
5
6

==> 中断时的有一个寄存器会保存中断源的vector值.
==> ==> `arch/x86/kernel/entry_64.S`调用函数`do_IRQ`.
==> ==> ==> `do_IRQ`依据`vector_irq`和vector值, 找到对应的中断号,并调用`handle_irq`.
==> ==> ==> ==> `handle_irq`通过函数irq_to_descdesc,可将中断号转化为`struct irq_desc`.
==> ==> ==> ==> generic_handle_irq_desc(irq, desc);
==> ==> ==> ==> ==> `generic_handle_irq_desc`调用 desc->handle_irq(irq, desc);

注:这里的handle_irq不是真正的中断处理函数,而是几大类中断控制器处理函数.
如82599, msi等.

`do_IRQ(struct pt_regs *regs)

File: arch/x86/kernel/irq.c

arch/x86/kernel/entry_64.S
will call do_IRQ

Read More

summary

A delayed work will first start a timer,
and when timeout, the delayed work will be put a worker_pool‘s
worklist or a pool_workqueue‘s delayed_works

how to use delayed work

data structure

1
2
3
4
5
6
7
8

113 struct delayed_work {
114         struct work_struct work;
115         struct timer_list timer;
116
117         /* target workqueue and CPU ->timer uses to queue ->work */
118         struct workqueue_struct *wq;
119         int cpu;
120 };

Read More

###Summary
The struct worker is the really scheudle unit in workqueue.
Each struct worker has a corresponding thread(task) by worker->task.
A struct worker is linked to struct worker_pool->idle_list when work is idle.
and moved to struct worker_pool->busy_hash.

worker_thread

1. move worker from pool->idle_list and clear worker ‘s WORKER_IDLE flag.
2. check the pool and manage the workers(create/destory)

3. Iterate all the `struct work_struct *work` in the `struct worker_pool->worklist`,
   

   and run them in sequence with process_one_work(worker, work);.
4. move worker into idle list again.
5. schedule();

Read More

summary

Read More